Revert "[smart_holder] Keep parent alive when returning raw pointers (#4609)" This reverts commit 99cf27a4f5d36d7336e0340bb5ee2eef03c00407.