TabPFNEmbedding Documentation and Usecase Help

[TomatoTurtlez]

Hello,

We are using 'from tabpfn_extensions.embedding import TabPFNEmbedding to get embedding data using the following:

train_embeddings = embedding_extractor.get_embeddings(X_train, y_train, X_train, data_source='train')
test_embeddings = embedding_extractor.get_embeddings(X_train, y_train, X_test, data_source='test')

I have a question on what data_source is and how it's working in the background. I could not understand its purpose and function by reading the source code/documentation.

Also, if I have n_estimators = 8, is there a recommended approach for combining the 8 different embedding datasets into a single dataset for further analysis? In your examples, you use n_estimators = 1 to bypass this problem.

[LennartPurucker]

Heyho @TomatoTurtlez,

To get a better understanding of the function and data_source, checkout https://github.com/PriorLabs/TabPFN/blob/main/src/tabpfn/base.py#L444.

data_source='train' produces the embeddings for the data passed to .fit() (so the train data) and data_source='test' for the test data.

The embedding function of TabPFN returns the embedding for each estimator in the last dimension. It is hard to say how best combine them. Depending on the use case, you could concatenate or average them, but that might not give you what you are looking for.