Well, Tatsumi never actually showed sexual or even romantic attraction to Subaru when he was a child... but as time passed he began to understand the anticipation he feels for Subaru to come back and he understood that this unreasonable longing that has last for so long is love.
But, after this realisation, he still didn't fantasise about the child Subaru he had in mind, and he was fully aware that he has grown up and became an adult.
Eventhough, it is briefly shown in the manga, these two have been in touch for years by emails and stuff... so the emotions haven't grown only because Subaru was an important presence in Tatsumi's life in the past. It is more likely that it has developed over the course of years.
So even if the story starts a bit weird, i liked it and it wasn't off-putting for me. It is not pedophilia as many people have mentioned it, cause that term has specific things associated with it that we don't see in Tatsumi's character.
The characters and storyline were interesting... BUT, somehow the author managed to write the dramatic events in the most underwhelming way... like the trajedy that happened in the acadamy, their long lasting separation for over 10 years, their reunion, the final revenge thing. All these events felt skipped and didn't stick to me ( ̄∇ ̄") as a reader i should feel the impacts of the things that push through the plot line.
Also... i think it could have shown better how they fell so deep in love together in the first place.
Lastly, the whole not being able to escape a cave for 10 years, and rigon not being caught when he was literally in the enemy's turf most of the time seemed so unrealistic and over the top.
The better examples of plot delivering in this genre would be painter behind the curtain, mo dao zu shi, and King's maker.
Can't wait for the sequel (≧∀≦)