i think that they say so because we know Shouto has an even older brother, and something really bad happened to him. From the clues we know that he was super powerful and his fire was even stronger than his dad's so it went wrong somewhere, we just don't know where.
So people see Dabi, who is very very powerful with fire, as that, and the accident, being the fact that his powers actually burn him himself too because his body is not strong enough to withhold the power he has. He has a similar shitty attitude as Shouto but with more shitiness of his dad in it, dislike the same stuff (fish) and all that. Shouto probably doesnt reconize him because maybe he was too young to remember (we don't even know Dabi's age i think, so shouto might noteven have been born when shit went down) , or it can also be the decay of dabi's face that didnt allow it.
Why do so many people think Todoroki and Dabi are related? I'm really wondering what similarities they have aside from both being able to use fire. Todoroki didn't even acknowledge Dabi or react to seeing him when they first met sooo...how could they be brothers?