trie: optimize compactToHex and cleanup encoding logic

[therainisme]

Rewrote compactToHex to allocate the exact buffer size upfront. This avoids the intermediate allocation and slicing overhead previously caused by keybytesToHex.

Also switched to bitwise operations for nibble handling in encoding.go and extracted the 100 threshold constant in trie.go.

Benchmarks:

goos: linux
goarch: amd64
pkg: github.com/ethereum/go-ethereum/trie
cpu: 12th Gen Intel(R) Core(TM) i5-12500

BenchmarkCompactToHex-12    	34527297	        36.71 ns/op	      16 B/op	       1 allocs/op  (baseline)
BenchmarkCompactToHex-12    	39204872	        31.94 ns/op	      16 B/op	       1 allocs/op  (optimized)