That's sort of true but it gets exaggerated, and often conflated with future outcomes. The Qin were unifying China parallel to Rome defeating Carthage. The Romans end up arguably about two centuries behind the Chinese, conquering Gaul and dominating the whole of the Med and materializing as an empire about 2 later, and collapsing about 2 later. The Han dynasty rules from that (roughly) 200bc-200ad block and expands its borders over the river areas and comes across mountains in the west, emptiness in the north, ocean to the east, and jungles to the south. In this period Rome's empire is based around the Mediterranean which is a FAR superior internal transportation system (at least until
this) with its borders also natural and better: Sahara desert instead of Mongolian steppes, and the Danube/Rhine line is a very solid potential for border over merely jungles. They both have one weak border, with China pushing a western pocket (the Han had a pretty awkward-looking setup going west) and Rome having its clients east.
Long story short, Rome had both a better internal network for connection and a better external natural border.
Further, Rome was even better at integration. The Han did a remarkable job of bringing the groups into a central framework, but the Romans were basically unmatched in stabilizing a culture with idealogy/values and common melding points such as language sharing. They were also arguably just as good at absorbing external cultures, with them becoming basically Greek and making it Roman, the way China would repeatedly just adopt outsiders and meld it into themselves.
When China collapses, it falls HARD. The depopulation takes centuries to recover. The religion is supplanted by a foreign one - Buddhism. Culture and society divides based on the nearby major river, so north/south. By Comparison, half of Rome doesn't even collapse - it would be like one of the three kingdoms being relatively unscathed for China. Christianity is established whereas Taoism flopped, then survives and even flourishes where Taoism and Confucianism are rendered beneath Buddhism. The period of ~4 to 800 in the west, the "dark ages" between the fall of the Western Roman Empire and the Carolingian rise, I would argue are actually far less severe than the 2 to 600 block in the east, between the fall of the Han and the eventual rise of the Sui and especially Tang.
What I think marks the big difference is that the Carolingian resurgence fails to reconstitute the Empire, fails to centralize at all, fails to restore the cultural appeal, etc. In contrast the Tang are often regarded as China's high point, with a restoration of the unified China and the massive population and ability to continue pushing borders from the seat of a very strong central government. The Carolingians (Charlemagne etc) were not necessarily some kind of awful failure, they were pretty great and successful in fact, but they simply were not great ENOUGH and the administration in particularly was rendered fully incapable of pulling off a return to a single unified empire. The alternative - the Eastern/Byzantine Romans, never managed to regain full control either. Justinian squandered resources trying to force it without good diplomacy and with poor use of great generals, and IMO their best hope was squelched when he went all in on Persia only to get hit by the unstoppable tide of Islam. The result is nobody ever managed to actually recreate the Roman Empire, and so only traces of it (like Romance languages & Catholicism) survived.
TL;DR China had inferior geography, weaker cultural unity and assimilation, and a far worse collapse. It was instead the fact that a future dynasty was able to reconstitute the Empire and perpetuate the concept that allowed it to continue, and easily repeat that feat once already having been done. Rome had better geography, stronger culture, superior religious unity, and half of it even survives its crash, but it ends up separating and staying separated and the idea of a perpetual "Rome" as a cultural and geographical entity dies out.
