Fixing last_known_instituion being mostly an empty field in the database snapshot

[DevinBayly]

Howdy!

Lots of great work went into all of these tools so just wanted to say thanks for all that. I had a researcher report to me that they were trying to use the last_known_institution field from my recently created snapshot and it was null with the exception of ~1500 rows. This prompted me to look at the flattener and I found this section

openalex-documentation-scripts/flatten-openalex-jsonl.py

Line 294 in 6eb823c

author['last_known_institution'] = (

It appears that this is still using the deprecated material https://docs.openalex.org/api-entities/authors/author-object#last_known_institution-deprecated. I'm going to re-write that part of the flatten_authors to make the last_known_institutions a json result (this will also require updating the spec, since I think it's still saying that last_known_institution is a text type/VARCHAR)

Would these changes be of interest for folks? I'd be happy to do a PR. If these helper scripts aren't really the main way people work with the snapshot though I understand that it's more work to review and merge changes from people.

have a good one either way!

[samemon]

yes please. the current script does not work on the current snapshot.

[DevinBayly]

Hey there! Just saw the notification in my spam. I think another PR on the repo might have code that helps?
#12 , but in case that doesn't work for you here's a couple of lines I changed

If this works for you I'll think about getting a PR together in January.