-
Notifications
You must be signed in to change notification settings - Fork 488
Open
Description
Description
I use llamasharp in kombination with KernelMemory.
When I send a question to the model it generates a response whithout any error messages but the response stops mid sentence.
I use
Embeddingmodel: "multilingual-e5-base-q8_0.gguf"
Model: "em_german_leo_mistral.Q4_K_M.gguf"
and the following code:
InferenceParams infParams = new()
{
//MaxTokens = setMaxTokens,
MaxTokens = -1,
TokensKeep = 100,
SamplingPipeline = new DefaultSamplingPipeline
{
Temperature = 0.7f,
TopP = 0.9f,
TopK = 50,
RepeatPenalty = 1.08f
},
};
LLamaSharpConfig llamasharpconfig = new(modelPath) {
DefaultInferenceParams = infParams,
ContextSize = 4096,
};
memory = new KernelMemoryBuilder()
.WithLLamaSharpTextEmbeddingGeneration(new LLamaSharpConfig(modelPathEmbedding) {
DefaultInferenceParams = infParams,
ContextSize = 4096,
})
.WithLLamaSharpTextGeneration(llamasharpconfig)
.WithPostgresMemoryDb(postgresConfig)
.WithSimpleFileStorage(@".\database\")
.Build(new KernelMemoryBuilderBuildOptions { AllowMixingVolatileAndPersistentData = false });
What am I missing? I already tried adding other parameters and changing the parameters with no success.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels