Yazīd ibn Abī Ḥabīb differed with him [on this]:

479 — We were informed by ʿĪsā ibn Ḥammād Zughbah: Layth narrated to us, from Yazīd ibn Abī Ḥabīb, from ʿIrāk ibn Mālik, that it had reached him
that Nawfal ibn Muʿāwiyah said: I heard the Messenger of Allah, peace and blessings be upon him, say: "Among the prayers there is a prayer — whoever misses it, it is as if he has been robbed of his family and his wealth." Ibn ʿUmar said: I heard the Messenger of Allah, peace and blessings be upon him, say: "It is the ʿAṣr prayer."(1)

--------------------------------------------
= From Nawfal ibn Muʿāwiyah (similar to the ḥadīth of Abū Hurayrah which they both narrated before it), raised to the Prophet: "There will be trials; the one sitting during them is better than the one standing..." And he added: "Among the prayers there is a prayer — whoever misses it, it is as if he has been robbed of his family and his wealth."

Aḥmad (4545), Muslim (626): (200), and others have also brought forth the ḥadīth of Ibn ʿUmar via the chain of Sālim ibn ʿAbd Allāh ibn ʿUmar, from him, with similar wording, and it has other chains.
See also the two ḥadīths that follow.

Al-Sindī said: His statement "wuṭira ahlahu wa-mālahu" is narrated with the accusative [ahlahu wa-mālahu], on the basis that "wuṭira" means "was deprived of," and it takes two objects; and with the nominative [ahluhu wa-māluhu], on the basis that it means "was taken," in which case "his family" is the subject of the passive verb. The intended meaning is that one should be wary of missing it just as one is wary of losing his family and his wealth... And the correct view is that what is meant is that he incurs such a loss of reward in the Hereafter that, if weighed against the losses of this world, nothing would counterbalance it except the loss of one whose family and wealth have been taken from him. And Allah knows best. Moreover, this ḥadīth does not fall under the heading of the ʿAṣr prayer during travel; rather, this is a separate discussion... and Allah knows best.

(1) A sound ḥadīth; its narrators are trustworthy. Layth: he is Ibn Saʿd.
The difference of Yazīd ibn Abī Ḥabīb from Jaʿfar ibn Rabīʿah in the preceding narration lies in his saying: "from ʿIrāk ibn Mālik that it had reached him that Nawfal ibn Muʿāwiyah..."

Al-Khaṭīb al-Baghdādī also brought it forth in "Al-Kifāyah," pp. 413–414, via the chain of the author [al-Muṣannif] with this isnād, and he said: The ruling requires giving precedence in this ḥadīth to Jaʿfar ibn Rabīʿah, on account of the established fact that he connected the ḥadīth [in full], due to his trustworthiness and precision. Layth's narration does not constitute a refutation of him, for it is possible that ʿIrāk had this ḥadīth conveyed to him from Nawfal ibn Muʿāwiyah, then heard it from him afterward, and so narrated it both ways. And Allah knows best.
See also the ḥadīth preceding this one and the one following it.