Chatdoc API markdown issue

[spseon]

I am currently using Chatdoc API to read multiple markdown documents.

When I upload multiple markdown documents to regular Chatdoc(not API), it returns accurate answers as well as reference pages.

However, when I upload the same markdown documents to ChatdocAPI-generated platform, it's returning inaccurate answers and reference pages. The answers are so inaccurate that it seems as if it is understanding the text.

(I used the same chatgpt version(gpt 4.0) to test on both Chatdoc and Chatdoc API)

We'd like to get similar outputs using Chatdoc API as we would when using regular Chatdoc. What could be the causing factor for such a huge discrepancy? Is there any way we can improve the API output?

[spseon]

Please, give us feedback as soon as possible. Our service development has stopped due to this issue and we need this issue to be solved as soon as possible.

[spseon]

food-laws-chatbot-POC-main.zip

Here is our code for reference.

[cumtyc]

Could you please provide a sample document and corresponding questions?

[spseon]

Our documents and questions are in Korean. Would that work as well?

For reference, this is the site we created using your API: https://dsx6dsyv-poc.vercel.app

[spseon]

Here are the sample files and questions we used to test on API and Chatdoc. Let us know if files in Korean doesn't work for you.

법령모음_MD-20240927T043931Z-001.zip

• “무가당"이라고 표시하기 위한 기준을 알려줄래?
• 용량 허용 오차범위를 알려줄래?
• 나트륨 저감 표시할 수 있는 식품이 뭐가있어?

[spseon]

For reference, here is an example :

Chatdoc

• When I ask "Please tell me the reference amount for one serving of jam(잼류의 1회 섭취 참고량을 알려줘)" it returns correct answer("The reference amount for one serving of jam is 20g.)
• 

Custom API

• But when we ask the same the same question in custom API, it says that the information is not provided within the document.
• 

The same thing happens with English documents:

Chatdoc

• We uploaded a multiple English markdown documents and asked "What are the cases where milk are considered unsafe for importation?"
• It returned correct answers with several reference pages.
• 

Custom API

• But custom API did not find relevant information within the document.

• 

• Also, it gives us too many reference pages, some of which are completely irrelevant(those sentences do not even contain words from the prompt questions). This reference page problem happens for every question within custom API.

Custom API does not always return wrong answers. However, even when it does return relevant answers, it is still different from Chatdoc answers(Chatdoc returns more accurate answers). We are aware that chatGPT can give different answers for the same question, but we don't think that is the problem since each of Chatdoc and custom API consistently returns same answers for the same question. Also, custom API consistently provides too many reference pages, most of which are irrelevent.

Following are the files we uploaded for the test(we uploaded all the files in one collection):

• Korean doc: 법령모음_MD-20240927T043931Z-001.zip
• English doc: English test doc.zip

We appreciate your help. Thank you.

[cumtyc]

We have updated our API service and switched the default model to gpt-4o-mini. You can try it out to see if it resolves your issue.

[spseon]

Thank you very much for the update. It seems to work much better now!

There is an additional issue regarding detailed citation('detailed_citation):

1. The citation number is displayed at the end of each sentence. However, it is not clickable(it's not linked to the text on the Document Viewer)

2. Sometimes, the detailed citation does how show at the end of each sentence whereas it does for Chatdoc.

• Chatdoc

• Custom API

[spseon]

Hello, after doing some further testing, we noticed some issue with the answers.

When we entered the prompt "Please tell me the reference amount for one serving of jam(잼류의 1회 섭취 참고량을 알려줘)" in Chatdoc, it used to extract information from the table and return correct answer("20g"). It was only the case in custom API that the information could not be found.

However, not the answer cannot be found in both Chatdoc and custom API. Was there any update to Chatdoc? Is there a way we can get correct answers for both? (We tested with this prompt a number of times and Chatdoc returned correct answers everytime.)

• Chatdoc returned correct answer until 29th of Semptember.

• Chatdoc begins failing to find the info in the document(2nd of October) for the same question. The same happens even when we swtich the model type.