Migration towards base32 CIDv1

[kevina]

This issue is to form a migration plan to base32 CIDv1 specific to go-ipfs. It is separate from #4143 which was more of a discussion or https://github.com/ipfs/ipfs/issues/337 which is more of a meta issue for all of go-ipfs, js-ipfs, and ipfs-companion.

Phase 1

• Add ability to retrieve blocks even if given using a different CID version Add ability to retrieve blocks even if given using a different CID version #5285
• Add flag to ipfs add to return CIDs in alternative base Create ipfs.add flag for specifying base of produced CID #5233 p.r Add global option to specify the multibase encoding. #5464
• Provide utility to convert between multibase encoding and CID versions Tool for converting hashes #5229 p.r. Provide new "cid" sub-command. #5385

Phase 2 -- full migration.

• Refactor blockstore to work with bare multihashes instead of CIDs Refactor blockstore to work with bare multihashes instead of CIDs #5231
• Possible Add ability to retrieve using a different CID version from legacy nodes RFC: Handling of CidV1/V0 at the bitswap level #5378
• Figure out how to handle CID outputs in non-default multibase encoding RFC: Handling of Alternative Multibases #5349 (also ipfs resolve should respect CID base #5234 and WIP: Add global option to specify the multibase encoding. #5289). This includes if the multibase should be preserved in the CID and the addition of a global flag to specify the CID base.
• Create config option for specifying full default cidv1 parameters Create config option for specifying full default cidv1 parameters #5230
• Figure out how to handle existing CidV0 RFC: How to handle CIDv0 as we migrate to CIDV1 #5291

cc @whyrusleeping @Stebalien @lidel @magik6k @kyledrake.

[kyledrake]

This is great, thank you for setting this up. LMK if you need anything from me.

[kevina]

Issues that I currently need feedback on from the core developers:

• Handling of Alternative Multibases RFC: Handling of Alternative Multibases #5349
• How to handle CIDv0 as we migrate to CIDV1 RFC: How to handle CIDv0 as we migrate to CIDV1 #5291
• Create config option for specifying full default cidv1 parameters Create config option for specifying full default cidv1 parameters #5230

[hsanjuan]

(subscribing)

[daviddias]

(subscribing)

[alanshaw]

Kind request 🙏 - there's a button on the right you can press to subscribe - commenting "(subscribing)" sends me and everyone else mentioned or participating a notification.

[lidel]

Some relevant updates from Golang Core Dev Team Weekly Sync 2019-02-11:

Base32/CIDv1

[..]

@eingenito - Need to consider next steps and who/when this can be picked up.
@Stebalien indicates that the next work to be done is converting the blockstore to use raw multihashes (Issue, WIP). This is a complicated task that needs some context for whoever picks it up.

Q: What's the relative priority of this task compared to others on this list?
A: Most important next milestone is storing/fetching [raw] multihashes in the DHT in libp2p.

[DonaldTsang]

Idea: Use https://github.com/kstenerud/safe-encoding as a better alternative for safe encoding

[Stebalien]

On technical merits alone, those encoding schemes look superior (and the order-preserving property looks quite nice). Unfortunately,

1. We're far enough into this switch that changing the default will require a pretty strong motivation.
2. The motivation for Base32 is wide (often built-in) language support.

[lidel]

Discussion that lead us to picking Base32 as the new default can be found in #4143 (comment):

[..] chose RFC because it's a standard, it's been around a while, nginx-misc-module supports it, and it probably has the widest support across all programming languages.

The only reason I would chose the RFC one is because it is a standard and more likely to have an implementation available as part of the language.

@DonaldTsang encodings you linked look really interesting.
I believe picking RFC one is still a good choice when it comes to being the new default, but we surely can add order-preserving ones as alternative encodings. Thank you for opening multiformats/multibase#51, let's discuss them there.