babel-formatting.org

Tests

Testing sessions:

x = 1

print(x*2)

Timer formatting

Timer is off by default. These tests enable it explicitly:

print(1)
import time
time.sleep(3)

Turn off timer-show to hide it

print(1)

Set :timer-rounded to no to get the full timer. (Also modifying the timer string here so that my expect tests will skip it.)

print(1)

Table formatting

As org tables

By default dataframes are printed as org tables

import pandas as pd
data = {
'Name': ['Joe', 'Eva', 'Charlie', 'David', 'Eva'],
'Age': [44, 32, 33,33, 22],
'City': ['New York', 'San Francisco', 'Boston', 'Paris', 'Tokyo'],
'Score': [92.5, 88.0, 95.2, 78.9, 90.11111]}
df = pd.DataFrame(data)
print(df)

idx	Name	Age	City	Score
0	Joe	44	New York	92.5
1	Eva	32	San Francisco	88.0
2	Charlie	33	Boston	95.2
3	David	33	Paris	78.9
4	Eva	22	Tokyo	90.11111

This respects various pandas options:

Float formatting

pd.options.display.float_format = '{:.1f}'.format
print(df.set_index("Name"))

Name	Age	City	Score
Joe	44	New York	92.5
Eva	32	San Francisco	88.0
Charlie	33	Boston	95.2
David	33	Paris	78.9
Eva	22	Tokyo	90.1

Max rows

pd.options.display.max_rows = 10
long_df = pd.DataFrame({'A': range(200)})
print(long_df)

idx	A
0	0
1	1
2	2
3	3
4	4
5	5
6	6
7	7
8	8
9	9

Problem – hangs when printing large dataframes.

print_org_df sets max_rows to be 20 by default to avoid this issue.

import pandas as pd
long_df = pd.DataFrame({'A': range(400)})
print(long_df)

idx	A
0	0
1	1
2	2
3	3
4	4
5	5
6	6
7	7
8	8
9	9
10	10
11	11
12	12
13	13
14	14
15	15
16	16
17	17
18	18
19	19

If we make the max_rows even modestly large, we run into it, depending on computing resources.

pd.options.display.max_rows = 200
long_df = pd.DataFrame({'A': range(200)})
print(long_df)

idx	A
0	0
1	1
2	2
3	3
4	4
5	5
6	6
7	7
8	8
9	9
10	10
11	11
12	12
13	13
14	14
15	15
16	16
17	17
18	18
19	19
20	20
21	21
22	22
23	23
24	24
25	25
26	26
27	27
28	28
29	29
30	30
31	31
32	32
33	33
34	34
35	35
36	36
37	37
38	38
39	39
40	40
41	41
42	42
43	43
44	44
45	45
46	46
47	47
48	48
49	49
50	50
51	51
52	52
53	53
54	54
55	55
56	56
57	57
58	58
59	59
60	60
61	61
62	62
63	63
64	64
65	65
66	66
67	67
68	68
69	69
70	70
71	71
72	72
73	73
74	74
75	75
76	76
77	77
78	78
79	79
80	80
81	81
82	82
83	83
84	84
85	85
86	86
87	87
88	88
89	89
90	90
91	91
92	92
93	93
94	94
95	95
96	96
97	97
98	98
99	99
100	100
101	101
102	102
103	103
104	104
105	105
106	106
107	107
108	108
109	109
110	110
111	111
112	112
113	113
114	114
115	115
116	116
117	117
118	118
119	119
120	120
121	121
122	122
123	123
124	124
125	125
126	126
127	127
128	128
129	129
130	130
131	131
132	132
133	133
134	134
135	135
136	136
137	137
138	138
139	139
140	140
141	141
142	142
143	143
144	144
145	145
146	146
147	147
148	148
149	149
150	150
151	151
152	152
153	153
154	154
155	155
156	156
157	157
158	158
159	159
160	160
161	161
162	162
163	163
164	164
165	165
166	166
167	167
168	168
169	169
170	170
171	171
172	172
173	173
174	174
175	175
176	176
177	177
178	178
179	179
180	180
181	181
182	182
183	183
184	184
185	185
186	186
187	187
188	188
189	189
190	190
191	191
192	192
193	193
194	194
195	195
196	196
197	197
198	198
199	199

Printing multiple dataframes:

print(df)
print("Space between dataframes")
print(df)

idx	Name	Age	City	Score
0	Joe	44	New York	92.5
1	Eva	32	San Francisco	88.0
2	Charlie	33	Boston	95.2
3	David	33	Paris	78.9
4	Eva	22	Tokyo	90.1

Space between dataframes

idx	Name	Age	City	Score
0	Joe	44	New York	92.5
1	Eva	32	San Francisco	88.0
2	Charlie	33	Boston	95.2
3	David	33	Paris	78.9
4	Eva	22	Tokyo	90.1

In general space between dataframes requires ones below to be aligned. I have an advise function ( adjust-org-babel-results ) that does this, but it can be slow if there are many tables in the org file, so it can be disabled like this.

print(df)
print("Space between dataframes")
print(df)

idx	Name	Age	City	Score
0	Joe	44	New York	92.5
1	Eva	32	San Francisco	88.0
2	Charlie	33	Boston	95.2
3	David	33	Paris	78.9
4	Eva	22	Tokyo	90.1

Space between dataframes

idx	Name	Age	City	Score
0	Joe	44	New York	92.5
1	Eva	32	San Francisco	88.0
2	Charlie	33	Boston	95.2
3	David	33	Paris	78.9
4	Eva	22	Tokyo	90.1

Bug – tables that contain | are buggy.

Need a way to handle |’s in the string names

import pandas as pd


df = pd.DataFrame({"names": ["John \vert", "Mary", "Bob  Rob", "Alice John", "Tom"]})
print(df)

idx	names
0	John

ert \

1	Mary
2	Bob Rob
3	Alice John
4	Tom

One work around is to call to_markdown directly, as ob-python-extras converts | that are not in dataframes into \ to prevent org from incorrectly recognizing text as tables.

import pandas as pd

df = pd.DataFrame({"names": ["John", "Mary", "Bob|Rob", "Alice|John", "Tom"]})
print(df.to_markdown())

Displaying styled dataframes as pngs

Dataframes can also be displayed as styled dataframes. This is nice for exporting documents with pretty tables.

Removing because I haven’t been able to get it to work in CI. — #+name: styled_dataframes

styled_df = df.style.background_gradient()
print(styled_df)

Polars

Polars dataframes are always printed as an org table as well.

import polars as pl

df = pl.DataFrame({"x": [1, 1, 3], "y": [2, 3, 1]})
print(df)

idx	x	y
0	1	2
1	1	3
2	3	1

Cell Timer: 0:00:00

Testing Tabulate

If Tabulate is available we can use it directly to formate the dataframe. This is built into pandas and the safer option.

#+name print_with_tabulate

import pandas as pd
data = {
'Name': ['Joe', 'Eva', 'Charlie', 'David', 'Eva'],
'Age': [44, 32, 33,33, 22],
'City': ['New York', 'San Francisco', 'Boston', 'Paris', 'Tokyo'],
'Score': [92.5, 88.0, 95.2, 78.9, 90.11111]}
df = pd.DataFrame(data)
print(df)

idx	Name	Age	City	Score
0	Joe	44	New York	92.5
1	Eva	32	San Francisco	88.0
2	Charlie	33	Boston	95.2
3	David	33	Paris	78.9
4	Eva	22	Tokyo	90.11111

Images

mocks out python plotting to allow plots to be interspersed with printing, and allows multiple to be made. :)

import matplotlib.pyplot as plt
import pandas as pd

print("look!")
df = pd.DataFrame(
    {
        "x": [0, 2, 3, 4, 5, 6, 7],
        "y": [10, 11, 12, 13, 14, 15, 16],
    }
)
print(df)
df.plot(x="x", y="y", kind="line")
plt.show()
print("tada!")

idx	x	y
0	0	10
1	2	11
2	3	12
3	4	13
4	5	14
5	6	15
6	7	16

tada!

HTML formatting

import base64
from io import BytesIO

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

# Create sample data
df = pd.DataFrame(
    {
        "x": np.linspace(0, 10, 100),
        "sin": np.sin(np.linspace(0, 10, 100)),
        "cos": np.cos(np.linspace(0, 10, 100)),
    }
)

# Create matplotlib plot
plt.figure(figsize=(8, 4))
plt.plot(df["x"][:20], df["sin"][:20], label="sin")
plt.plot(df["x"][:20], df["cos"][:20], label="cos")
plt.legend()
plt.title("Sine and Cosine Waves")

# Convert plot to base64
buf = BytesIO()
plt.savefig(buf, format="png")
plt.close()
img_base64 = base64.b64encode(buf.getvalue()).decode("utf-8")

# Create HTML with table and image
html = f"""
<h1>Data Analysis Results</h1>
<p>Here's a sample of our trigonometric functions:</p>
{df.head().to_html(classes='dataframe')}
<p><b>Visualization:</b></p>
<img src="data:image/png;base64,{img_base64}"/>
<p><i>Figure 1: First few periods of sine and cosine waves</i></p>
"""

print(html)

	x	sin	cos
0	0.00000	0.000000	1.000000
1	0.10101	0.100838	0.994903
2	0.20202	0.200649	0.979663
3	0.30303	0.298414	0.954437
4	0.40404	0.393137	0.919480

Visualization:

Figure 1: First few periods of sine and cosine waves

Also use dataframe_image to get styled dataframes from the html output as pngs.

Error handling

print(1 / 0)

x = 0
print(1 / 0)

Get more detailed errors

Last line print

#+name testing_last_line_print

x = 1
print(x)
1000 * 2 + x

Edge case handling

Last line might not be an expression. Ideally this would get the last expression, but I’m settling for just not crashing.

(Achieved by checking if the code without the last line is valid python before execing it first; otherwise exec’s the whole block. I don’t like relying on _.) #+name testing_last_line_print_not_full_expr

(
    1,
    2,
    3,
    4,
    1,
)

Need to make sure that we handle comments on the last line – in general, print(last_line) is checked to be valid python syntax.

#+name last_line_a_comment

print(1)
# a comment

Torch

import torch
x = torch.randn(3, 3)
print(x)

import torch

# Test tensor
x = torch.randn(3, 3)
print("Tensor:", x)
# Test dict
d = {"name": "Alice", "age": 30, "city": "New York", "hobbies": ["reading", "coding"]}
print("Dict:", d)

# Test list
l = [1, 2, 3, [4, 5, 6], {"nested": "dict"}]
print("List:", l)

# Test set
s = {1, 2, 3, 4, 5}
print("Set:", s)
import os
os.path
print(100)