Skip to content

Commit c6539cc

Browse files
ksmin23ksmin23
authored
Merge pull request #5 from ksmin23/features/langfuse-v3
Features/langfuse v3
2 parents 2530dda + 4e0bd2a commit c6539cc

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

56 files changed

+2873
-161
lines changed

‎README.md‎

Lines changed: 8 additions & 161 deletions
Original file line numberDiff line numberDiff line change
@@ -1,177 +1,24 @@
11

22
# Hosting Langfuse on Amazon ECS with Fargate using CDK Python
33

4-
> :information_source: This guide covers Langfuse v2. For Langfuse v3, see the [v3 documentation](https://langfuse.com/self-hosting).
5-
6-
![lanfuse-on-aws-ecs-fargate-arch](./langfuse-on-aws-ecs-fargate-arch.svg)
7-
84
This repository contains the AWS CDK Python code for deploying the [Langfuse](https://langfuse.com/) application using Amazon Elastic Container Registry (ECR) and Amazon Elastic Container Service (ECS).
95

106
Langfuse is an open-source LLM engineering platform that helps teams collaboratively debug, analyze, and iterate on their LLM applications.
117

12-
The `cdk.json` file tells the CDK Toolkit how to execute your app.
13-
14-
This project is set up like a standard Python project. The initialization
15-
process also creates a virtualenv within this project, stored under the `.venv`
16-
directory. To create the virtualenv it assumes that there is a `python3`
17-
(or `python` for Windows) executable in your path with access to the `venv`
18-
package. If for any reason the automatic creation of the virtualenv fails,
19-
you can create the virtualenv manually.
20-
21-
To manually create a virtualenv on MacOS and Linux:
22-
23-
```
24-
$ python3 -m venv .venv
25-
```
26-
27-
After the init process completes and the virtualenv is created, you can use the following
28-
step to activate your virtualenv.
29-
30-
```
31-
$ source .venv/bin/activate
32-
```
33-
34-
If you are a Windows platform, you would activate the virtualenv like this:
35-
36-
```
37-
% .venv\Scripts\activate.bat
38-
```
39-
40-
Once the virtualenv is activated, you can install the required dependencies.
41-
42-
```
43-
(.venv) $ pip install -r requirements.txt
44-
```
45-
46-
> To add additional dependencies, for example other CDK libraries, just add
47-
them to your `setup.py` file and rerun the `pip install -r requirements.txt`
48-
command.
49-
50-
### Set up `cdk.context.json`
51-
52-
Then, we need to set approperly the cdk context configuration file, `cdk.context.json`.
53-
54-
For example,
55-
56-
```
57-
{
58-
"db_cluster_name": "langfuse-db",
59-
"ecs_cluster_name": "langfuse-cluster",
60-
"ecs_service_name": "langfuse-alb-service",
61-
"image_version": "2",
62-
"langfuse_env": {
63-
"NODE_ENV": "production",
64-
"NEXTAUTH_SECRET": "mysecret",
65-
"SALT": "mysalt",
66-
"TELEMETRY_ENABLED": "true",
67-
"NEXT_PUBLIC_SIGN_UP_DISABLED": "false",
68-
"LANGFUSE_ENABLE_EXPERIMENTAL_FEATURES": "true"
69-
}
70-
}
71-
```
72-
73-
:information_source: This guide covers Langfuse v2. `image_version` should be set to `2`.
74-
75-
:information_source: `NEXTAUTH_SECRET` and `SALT` can be created using `openssl rand -base64 32` on `MacOS` or `Ubuntu`.
76-
(For more information, see [**Langfuse Configuring Environment Variables**](https://langfuse.com/docs/deployment/self-host#configuring-environment-variables))
77-
78-
### Deploy
79-
80-
At this point you can now synthesize the CloudFormation template for this code.
81-
82-
```
83-
(.venv) $ export CDK_DEFAULT_ACCOUNT=$(aws sts get-caller-identity --query Account --output text)
84-
(.venv) $ export CDK_DEFAULT_REGION=$(aws configure get region)
85-
(.venv) $ cdk synth --all
86-
```
87-
88-
Use `cdk deploy` command to create the stack shown above.
89-
90-
```
91-
(.venv) $ cdk deploy --require-approval never --all
92-
```
93-
94-
We can list all the CDK stacks by using the `cdk list` command prior to deployment.
95-
96-
```
97-
(.venv) $ cdk list
98-
LangFuseECRStack
99-
LangFuseVpcStack
100-
LangFuseALBStack
101-
LangFuseAuroraPostgreSQLStack
102-
LangFuseECSClusterStack
103-
LangFuseECSTaskStack
104-
LangFuseECSAlbFargateServiceStack
105-
```
106-
107-
## Clean Up
108-
109-
Delete the CloudFormation stack by running the below command.
110-
111-
```
112-
(.venv) $ cdk destroy --force --all
113-
```
114-
115-
## Useful commands
116-
117-
* `cdk ls` list all stacks in the app
118-
* `cdk synth` emits the synthesized CloudFormation template
119-
* `cdk deploy` deploy this stack to your default AWS account/region
120-
* `cdk diff` compare deployed stack with current state
121-
* `cdk docs` open CDK documentation
122-
123-
Enjoy!
124-
125-
## Tracing for your LLM Application with Langfuse
126-
127-
After deploying all CDK stacks, you can find the **Langfuse URL** using the following command:
128-
129-
```bash
130-
aws cloudformation describe-stacks --stack-name LangFuseECSAlbFargateServiceStack | \
131-
jq -r '.Stacks[0].Outputs | map(select(.OutputKey == "LoadBalancerDNS")) | .[0].OutputValue'
132-
```
133-
134-
Next, open the **Langfuse URL** in your browser to create a new project for tracking your LLM application with Langfuse.
135-
136-
### Create a New Project in Langfuse
137-
138-
1. Create a Langfuse Account
139-
140-
![Login](./assets/01-langfuse-main-page.png)
141-
142-
![Singup](./assets/02-sign-up.png)
143-
2. Create a New Project
144-
![New-Project](./assets/03-new-project.png)
145-
3. Create New API Credentials in the Project Settings
146-
![API-Keys](./assets/04-create-api-keys.png)
147-
148-
### Log Your First LLM Call to Langfuse
149-
150-
Open the `tracing_for_langchain_bedrock` notebook in the `examples` folder and run it. (See [here](./examples/tracing_for_langchain_bedrock.ipynb) for more information)
151-
152-
You will the see the list of traces as follows:
153-
![Traces](./assets/05-traces.png)
154-
155-
You will also see the details of the selected trace as follows:
156-
157-
![Trace-detail](./assets/06-trace-detail.png)
8+
| Project | Architecture |
9+
|---------|--------------|
10+
| [langfuse-v2](./langfuse-v2/) | ![lanfuse-v2-arch](./langfuse-v2/langfuse-on-aws-ecs-fargate-arch.svg) |
11+
| [langfuse-v3](./langfuse-v3/) | ![lanfuse-v3-arch](./langfuse-v3/langfuse-v3-on-aws-ecs-fargate-arch.svg) |
15812

15913
## References
16014

161-
* [(AWS Korea Tech Blog) Hosting Langfuse on Amazon ECS with Fargate using AWS CDK Python (2024-08-09)](https://aws.amazon.com/ko/blogs/tech/hosting-langfuse-with-aws-cdk-python-using-amazon-ecs-and-aws-fargate/)
162-
* [Langfuse Official Documents](https://langfuse.com/docs) - Langfuse is an open-source LLM engineering platform that helps teams collaboratively debug, analyze, and iterate on their LLM applications.
163-
* [Langfuse Self-Hosting Guide (Docker)](https://langfuse.com/docs/deployment/self-host)
164-
* [Langfuse Configuring Environment Variables](https://langfuse.com/docs/deployment/self-host#configuring-environment-variables)
165-
* [Tracing for Langchain (Python & JS/TS)](https://langfuse.com/docs/integrations/langchain/tracing)
166-
* [Langfuse Python SDK (Low-level)](https://langfuse.com/docs/sdk/python/low-level-sdk)
167-
* [Langfuse Cookbook](https://github.com/langfuse/langfuse-docs/tree/main/cookbook)
168-
* [(GitHub) Langfuse](https://github.com/langfuse/langfuse) - 🪢 Open source LLM engineering platform: Observability, metrics, evals, prompt management, playground, datasets. Integrates with LlamaIndex, Langchain, OpenAI SDK, LiteLLM, and more.
169-
* [Langfuse ECR/ECS Deployment CDK TypeScript](https://github.com/AI4Organization/langfuse-ecr-ecs-deployment-cdk) - Deploy Langfuse to ECR and ECS with Fargate and AWS CDK TypeScript.
170-
* [cdk-ecr-deployment](https://github.com/cdklabs/cdk-ecr-deployment) - A CDK construct to deploy docker image to Amazon ECR.
15+
* [(Official) Self-host Langfuse Guide](https://langfuse.com/self-hosting)
16+
* [(GitHub) langfuse](https://github.com/langfuse/langfuse/)
17+
* [AWS CDK Reference Documentation](https://docs.aws.amazon.com/cdk/api/v2/)
17118

17219
## Security
17320

174-
See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information.
21+
See [CONTRIBUTING](./CONTRIBUTING.md#security-issue-notifications) for more information.
17522

17623
## License
17724

.gitignore renamed to langfuse-v2/.gitignore

File renamed without changes.

‎langfuse-v2/README.md‎

Lines changed: 184 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,184 @@
1+
2+
# Hosting Langfuse on Amazon ECS with Fargate using CDK Python
3+
4+
> :information_source: This guide covers Langfuse v2. For Langfuse v3, see the [v3 documentation](https://langfuse.com/self-hosting).
5+
6+
![lanfuse-on-aws-ecs-fargate-arch](./langfuse-on-aws-ecs-fargate-arch.svg)
7+
8+
This repository contains the AWS CDK Python code for deploying the [Langfuse](https://langfuse.com/) application using Amazon Elastic Container Registry (ECR) and Amazon Elastic Container Service (ECS).
9+
10+
Langfuse is an open-source LLM engineering platform that helps teams collaboratively debug, analyze, and iterate on their LLM applications.
11+
12+
The `cdk.json` file tells the CDK Toolkit how to execute your app.
13+
14+
This project is set up like a standard Python project. The initialization
15+
process also creates a virtualenv within this project, stored under the `.venv`
16+
directory. To create the virtualenv it assumes that there is a `python3`
17+
(or `python` for Windows) executable in your path with access to the `venv`
18+
package. If for any reason the automatic creation of the virtualenv fails,
19+
you can create the virtualenv manually.
20+
21+
To manually create a virtualenv on MacOS and Linux:
22+
23+
```
24+
$ git clone --depth=1 https://github.com/aws-samples/deploy-langfuse-on-ecs-with-fargate.git
25+
$ cd deploy-langfuse-on-ecs-with-fargate
26+
$ git sparse-checkout init --cone
27+
$ git sparse-checkout set langfuse-v2
28+
$ cd langfuse-v2
29+
30+
$ python3 -m venv .venv
31+
```
32+
33+
After the init process completes and the virtualenv is created, you can use the following
34+
step to activate your virtualenv.
35+
36+
```
37+
$ source .venv/bin/activate
38+
```
39+
40+
If you are a Windows platform, you would activate the virtualenv like this:
41+
42+
```
43+
% .venv\Scripts\activate.bat
44+
```
45+
46+
Once the virtualenv is activated, you can install the required dependencies.
47+
48+
```
49+
(.venv) $ pip install -r requirements.txt
50+
```
51+
52+
> To add additional dependencies, for example other CDK libraries, just add
53+
them to your `setup.py` file and rerun the `pip install -r requirements.txt`
54+
command.
55+
56+
### Set up `cdk.context.json`
57+
58+
Then, we need to set approperly the cdk context configuration file, `cdk.context.json`.
59+
60+
For example,
61+
62+
```
63+
{
64+
"db_cluster_name": "langfuse-db",
65+
"ecs_cluster_name": "langfuse-cluster",
66+
"ecs_service_name": "langfuse-alb-service",
67+
"image_version": "2",
68+
"langfuse_env": {
69+
"NODE_ENV": "production",
70+
"NEXTAUTH_SECRET": "mysecret",
71+
"SALT": "mysalt",
72+
"TELEMETRY_ENABLED": "true",
73+
"NEXT_PUBLIC_SIGN_UP_DISABLED": "false",
74+
"LANGFUSE_ENABLE_EXPERIMENTAL_FEATURES": "true"
75+
}
76+
}
77+
```
78+
79+
:information_source: This guide covers Langfuse v2. `image_version` should be set to `2`.
80+
81+
:information_source: `NEXTAUTH_SECRET` and `SALT` can be created using `openssl rand -base64 32` on `MacOS` or `Ubuntu`.
82+
(For more information, see [**Langfuse Configuring Environment Variables**](https://langfuse.com/docs/deployment/self-host#configuring-environment-variables))
83+
84+
### Deploy
85+
86+
At this point you can now synthesize the CloudFormation template for this code.
87+
88+
```
89+
(.venv) $ export CDK_DEFAULT_ACCOUNT=$(aws sts get-caller-identity --query Account --output text)
90+
(.venv) $ export CDK_DEFAULT_REGION=$(aws configure get region)
91+
(.venv) $ cdk synth --all
92+
```
93+
94+
Use `cdk deploy` command to create the stack shown above.
95+
96+
```
97+
(.venv) $ cdk deploy --require-approval never --all
98+
```
99+
100+
We can list all the CDK stacks by using the `cdk list` command prior to deployment.
101+
102+
```
103+
(.venv) $ cdk list
104+
LangFuseECRStack
105+
LangFuseVpcStack
106+
LangFuseALBStack
107+
LangFuseAuroraPostgreSQLStack
108+
LangFuseECSClusterStack
109+
LangFuseECSTaskStack
110+
LangFuseECSAlbFargateServiceStack
111+
```
112+
113+
## Clean Up
114+
115+
Delete the CloudFormation stack by running the below command.
116+
117+
```
118+
(.venv) $ cdk destroy --force --all
119+
```
120+
121+
## Useful commands
122+
123+
* `cdk ls` list all stacks in the app
124+
* `cdk synth` emits the synthesized CloudFormation template
125+
* `cdk deploy` deploy this stack to your default AWS account/region
126+
* `cdk diff` compare deployed stack with current state
127+
* `cdk docs` open CDK documentation
128+
129+
Enjoy!
130+
131+
## Tracing for your LLM Application with Langfuse
132+
133+
After deploying all CDK stacks, you can find the **Langfuse URL** using the following command:
134+
135+
```bash
136+
aws cloudformation describe-stacks --stack-name LangFuseECSAlbFargateServiceStack --region ${CDK_DEFAULT_REGION} | \
137+
jq -r '.Stacks[0].Outputs | map(select(.OutputKey == "LoadBalancerDNS")) | .[0].OutputValue'
138+
```
139+
140+
Next, open the **Langfuse URL** in your browser to create a new project for tracking your LLM application with Langfuse.
141+
142+
### Create a New Project in Langfuse
143+
144+
1. Create a Langfuse Account
145+
146+
![Login](./assets/01-langfuse-main-page.png)
147+
148+
![Singup](./assets/02-sign-up.png)
149+
2. Create a New Project
150+
![New-Project](./assets/03-new-project.png)
151+
3. Create New API Credentials in the Project Settings
152+
![API-Keys](./assets/04-create-api-keys.png)
153+
154+
### Log Your First LLM Call to Langfuse
155+
156+
Open the `tracing_for_langchain_bedrock` notebook in the `examples` folder and run it. (See [here](./examples/tracing_for_langchain_bedrock.ipynb) for more information)
157+
158+
You will the see the list of traces as follows:
159+
![Traces](./assets/05-traces.png)
160+
161+
You will also see the details of the selected trace as follows:
162+
163+
![Trace-detail](./assets/06-trace-detail.png)
164+
165+
## References
166+
167+
* [(AWS Korea Tech Blog) Hosting Langfuse on Amazon ECS with Fargate using AWS CDK Python (2024-08-09)](https://aws.amazon.com/ko/blogs/tech/hosting-langfuse-with-aws-cdk-python-using-amazon-ecs-and-aws-fargate/)
168+
* [Langfuse Official Documents](https://langfuse.com/docs) - Langfuse is an open-source LLM engineering platform that helps teams collaboratively debug, analyze, and iterate on their LLM applications.
169+
* [Langfuse Self-Hosting Guide (Docker)](https://langfuse.com/docs/deployment/self-host)
170+
* [Langfuse Configuring Environment Variables](https://langfuse.com/docs/deployment/self-host#configuring-environment-variables)
171+
* [Tracing for Langchain (Python & JS/TS)](https://langfuse.com/docs/integrations/langchain/tracing)
172+
* [Langfuse Python SDK (Low-level)](https://langfuse.com/docs/sdk/python/low-level-sdk)
173+
* [Langfuse Cookbook](https://github.com/langfuse/langfuse-docs/tree/main/cookbook)
174+
* [(GitHub) Langfuse](https://github.com/langfuse/langfuse) - 🪢 Open source LLM engineering platform: Observability, metrics, evals, prompt management, playground, datasets. Integrates with LlamaIndex, Langchain, OpenAI SDK, LiteLLM, and more.
175+
* [Langfuse ECR/ECS Deployment CDK TypeScript](https://github.com/AI4Organization/langfuse-ecr-ecs-deployment-cdk) - Deploy Langfuse to ECR and ECS with Fargate and AWS CDK TypeScript.
176+
* [cdk-ecr-deployment](https://github.com/cdklabs/cdk-ecr-deployment) - A CDK construct to deploy docker image to Amazon ECR.
177+
178+
## Security
179+
180+
See [CONTRIBUTING](../CONTRIBUTING.md#security-issue-notifications) for more information.
181+
182+
## License
183+
184+
This library is licensed under the MIT-0 License. See the LICENSE file.

app.py renamed to langfuse-v2/app.py

File renamed without changes.

assets/01-langfuse-main-page.png renamed to langfuse-v2/assets/01-langfuse-main-page.png

File renamed without changes.

assets/02-sign-up.png renamed to langfuse-v2/assets/02-sign-up.png

File renamed without changes.

assets/03-new-project.png renamed to langfuse-v2/assets/03-new-project.png

File renamed without changes.

assets/04-create-api-keys.png renamed to langfuse-v2/assets/04-create-api-keys.png

File renamed without changes.

assets/05-traces.png renamed to langfuse-v2/assets/05-traces.png

File renamed without changes.

assets/06-trace-detail.png renamed to langfuse-v2/assets/06-trace-detail.png

File renamed without changes.

0 commit comments

Comments
 (0)