[DNM] Revert memory leak fixes #600

Job	Run time
ci-data	4s
Compile native gem for aarch64-linux	1m 49s
Compile native gem for aarch64-linux-musl	1m 45s
Compile native gem for arm64-darwin	1m 54s
Compile native gem for x64-mingw-ucrt	2m 1s
Compile native gem for x86_64-darwin	1m 54s
Compile native gem for x86_64-linux	5m 5s
Compile native gem for x86_64-linux-musl	1m 46s
	16m 18s

Provide feedback