Skip to content

Document type and github loader#172

Open
matthusby wants to merge 3 commits intobrainlid:mainfrom
matthusby:add-github-data-loader
Open

Document type and github loader#172
matthusby wants to merge 3 commits intobrainlid:mainfrom
matthusby:add-github-data-loader

Conversation

@matthusby
Copy link
Copy Markdown
Contributor

This is a proof of concept, I did a quick dump of how I thought this should work.

Add LangChain.Document module, which provides a structured way to handle documents with content, metadata, and type attributes.

Add LangChain.Document.Loaders.Github module to support fetching issues from a specified repository, converting them into Document structs.

gh = LangChain.Document.Loaders.Github.new!()
docs = LangChain.Document.Loaders.Github.load(gh, %{type: :issue, repo: "brainlid/langchain"}

@brainlid
Copy link
Copy Markdown
Owner

Nice @matthusby! You describe this as a PoC. Is there anything else you think it needs or is lacking?

Would you be willing to write tests for the lib/document/loaders/github.ex file?

@matthusby
Copy link
Copy Markdown
Contributor Author

I called it PoC as its only doing issues, I thought I would add some more to it. But I have gotten busy so this can go as is and I will add more to it when I expand it. Added some tests 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants