Access filter defaults to public#1517
Closed
sergiimk wants to merge 37 commits intosku/moleculefrom
Closed
Conversation
* GQL: split Molecule into MoleculeV1 & MoleculeV2 * GQL: MoleculeMutV2 WIP * Move account_mut.rs to ./account_mut/ * GQL: Account quotas * test_search_accounts_by_name_pattern(): fix rebase collision * GQL: MoleculeDataRoomMut * GQL: MoleculeV2: announcements & whole file tweaks * GQL: MoleculeMutV2: announcements * GQL: MoleculeV2::activity() * Fixes after self review * AccountQuotasMut::set_user_level_quotas(): add doc string * Backported changes to make clippy a bit happier
* Tests: mark the current molecule tests as v1 * Modularize Molecule queries * Molecule: extract common things into common.rs * molecule/v2: create a dir * molecule/v1: create a dir * molecule_mut/: create a dir * molecule_project_v2.rs: extract * molecule_activity_event_v2.rs: extract * molecule_data_room_dataset_v2.rs: extract * molecule/: split rest entities * molecule_mut/: split rest entities * MoleculeV2::activity(): add "filters" arguments * MoleculeAnnouncementsDatasetV2::tail(): add "filters" argument * schema.gql: regenerate
* UpdateVersionFileUseCaseHelper: extract * MoleculeDataRoomMutV2::start_upload_file(): implement * QueryService::get_changelog_projection(): implement * QueryService::get_changelog_projection(): add "hint" options * MoleculeDataRoomMutV2: polished methods expect "finish_upload_file()" * MoleculeDataRoomMutV2::finish_upload_file_new_file() * MoleculeDataRoomFinishUploadFileV2 * MoleculeDataRoomMutV2::update_file_metadata(): a try to add the second step * MoleculeDataRoomMutV2::finish_upload_file_new_file_version() * clippy fixes * schema.gql: regenerate * Fix Molecule v1 tests * Working on Molecule v2 data room api * Molecule v2 API: implement basic data room operations * Migration fix * Inlined `UpdateVersionFileUseCaseHelper` within GQL utils, as we may not use `UploadService` in the "datasets" domain. + fixed MySQL migration * Try fixing ODF code generation flow * MoleculeDataRoomMutV2::finish_upload_file_new_file_version(): use UpdateCollectionEntriesUseCase * MoleculeDataRoomMutV2::finish_upload_file_new_file(): use UpdateCollectionEntriesUseCase * MoleculeDataRoomMutV2::move_entry(): use UpdateCollectionEntriesUseCase * MoleculeDataRoomMutV2::remove_entry(): use UpdateCollectionEntriesUseCase * MoleculeDataRoomMutV2::update_file_metadata(): implement * Schema regenerated * test_molecule_v2_data_room_operations(): use pretty_assertions::assert_eq * moveEntry test * removeEntry test * updateFileMetadata test * schema.gql: regenerate --------- Co-authored-by: Sergii Mikhtoniuk <mikhtoniuk@gmail.com> Co-authored-by: Sergei Zaychenko <zaychenko.sergei@gmail.com>
…1463) * Established Molecule domain & service crates. Extracted "View Molecule Projects" use case, and plugged into v1/v2 APIs * Extracted "Find Molecule project" use case * Extracted "Create Molecule project use case". Molecule dataset snapshots, as well as generic VersionedFile/Collection dataset snapshots are now a domain, not GQL concern. * Extracted `MoleculeProjectEntity` object to use instead of untyped JSON objects at domain level. GQL objects converted to [Object], replacing [SimpleObject], and keep entity. * Improved telemetry in new Molecule service layer * Sketched MoleculeProjectMessage outbox events. For now, sending "Created" message from the corresponding use case. * Review notes fixed
* UpdateCollectionEntriesUseCaseImpl::build_data_batches(): return note * MoleculeDatasetSnapshots::data_room_v2(): create alias internally * MoleculeDatasetSnapshots::projects(): create alias internally * MoleculeDatasetSnapshots::announcements(): create alias internally * StageDataResult: update doc strings * PushIngestOpts: fix a typo * MoleculeDatasetSnapshots::global_data_room_activity() * MoleculeProjectService -> MoleculeDatasetService * MoleculeDatasetService::get_global_data_room_activity_dataset() * MoleculeProjectV2::ipnft_token_id(): use U256 * MoleculeAppendDataRoomActivityUseCaseImpl * Tests stabilized * MoleculeDatasetSnapshots::global_data_room_activity(): update comment re LIST<BYTE_ARRAY item (STRING)>) * MoleculeDataRoomMutV2::finish_upload_file_new_file_version(): write data room activity * MoleculeDataRoomMutV2::finish_upload_file_new_file(): add maintainer permissions to molecule * DatasetHandleLoader: add AccessCheckedDatasetRef-related load() method * DatasetHandleLoader: add AccessCheckedDatasetRef-related load() method + ResolvedDataset * MoleculeVersionedFile::latest(): use data loader * MoleculeViewDataRoomActivitiesUseCaseImpl * MoleculeV2::activity() * MoleculeV2::activity() * test_molecule_v2_data_room_operations(): global activity checks (part) * MoleculeProjectV2::get_data_room_activity_events(): fixed, unit-tested * clippy fixes * test_internal_error(): simplify * OperationType::deserialize(): simplify * access_level: update todo * Add todos * Consts for snapshot names * Add global prefix
#1468) * Datasets domain use cases reorganized by folders * Extracted 'ViewCollectionEntriesUseCase' use case in datasets domain * Merge corrections * Extracted `FindCollectionEntryUseCase` * FindCollectionEntryUseCase => FindCollectionEntriesUseCase * More renaming cleanups * Listing structs: use EntityPageListing template. Introduced `CollectionPath` at datasets domain level. Domain's `ExtraDataFields` applied more systematically. * Extracted `ViewVersionedFileHistoryUseCase` * Extracted `FindVersionedFileVersionUseCase` use case * Simplified structures in `UpdateCollectionEntriesUseCase` * Cleanups in GQL adapters for collections and versioned files * Minor review
* Finish global data room activity * Correct project data room activities
* Add disable/enable project api * Fix clippy * Add comments * Fix review comments - Iter 1 * Refactor changelog entry duplication * Refactore: use GraphQLQueryRequest in tests * Add chain length asserts * Make chain search with alias parameter * Fix review comments * Fix tests * Update schema
…1472) * Molecule use cases need some folder structure too. Extracted use cases `MoleculeFindProjectDataRoomEntryUseCase` and `MoleculeViewProjectDataRoomEntriesUseCase`: those indirectly request project data rooms as a collection dataset, and map structures. A direct collection adapter talking to service layer of `kamu-datasets` (not to GQL!), with the extension seam for future federation (invoking collection entries from base GQL API remotly) * Got rid of manual DataFrame at GQL level when writing or updating file versions * `MoleculeDataRoomEntry`: simplified domain structure and GQL equivalent * Some intermediate cleanups after merge * First attempt to extract data room UPSERT use case * Upsert data room entry: returning new data room record * Extracted `MoleculeRemoveProjectDataRoomEntryUseCase` * Extracted `MoleculeMoveProjectDataRoomEntryUseCase` + aligned common parts with removals * Update metadata uses data-room level upsert use case * Telemetry cleanup * Naming cleanups * First sketch of data room outbox message: sending for move and remove * Split upsert data room entry on create and update UC, as they need to produce different outbox output * Propagating source event time for collection entry operations * Propagating system time from versioned file ingest properly * Got rid of extra ReBAC check at highest data room access point
* MoleculeDatasetService::get_global_announcements_dataset() * MoleculeCreateAnnouncementUseCase * MoleculeProjectMutV2::announcements() * format-utils crate * MoleculeAnnouncementsDatasetMutV2::create() * MoleculeViewGlobalDataRoomActivitiesUseCaseImpl: respect announcements * Molecule use cases: activity/ -> activities/ * Adaptation to the latest refactoring * MoleculeV2::activity() * MoleculeAnnouncements (project) * MoleculeProjectV2::activity(): update with announcements * MoleculeCreateAnnouncementUseCaseImpl: register * Tests fix * Tests fix [2] * Tests fix [3] * MoleculeProjectAnnouncementDataRecord: add a TODO * schema.gql: regenerate
* test_molecule_v2_announcements_operations(): checkpoint -- add 2 files * test_molecule_v2_announcements_operations(): checkpoint -- create empty announcement * test_molecule_v2_announcements_operations(): checkpoint -- Create an announcement with one attachment * test_molecule_v2_announcements_operations(): checkpoint -- Create an announcement with two attachments * test_molecule_v2_announcements_operations(): checkpoint -- Create an announcement with attachment DID that does not exist * test_molecule_v2_announcements_operations(): checkpoint -- Announcements are listed as expected * test_molecule_v2_announcements_operations(): finish * MoleculeAnnouncementEntry: system_time/event_time
* MoleculeEncryptionMetadata * schema.gql: regenerate * MoleculeDatasetSnapshots::versioned_file_v2(): remove todo * MoleculeEncryptionMetadata: extract to domain
…nPathV2` scalar (#1480) * DatasetNameGenerator * MoleculeDataRoomMutV2::build_new_file_dataset_alias(): use DatasetNameGenerator * CollectionPathV2 (domain) * CollectionPathV2 (domain): updates * CollectionPathV2 (GQL) * CollectionPathV2 (GQL): tests * schema.gql: regenerate * kamu-datasets: remove unused dep * Test fixes after resent changes * RUSTSEC-2025-0134
* ViewCollectionEntriesUseCase: support extra data filters * From<GetDataRoomCollectionEntriesFilters> for Option<kamu_datasets::ExtraDataFieldsFilter> * MoleculeDataRoomCollectionService::get_data_room_collection_entries(): add filters * MoleculeViewDataRoomEntriesUseCaseImpl: filters * MoleculeDataRoomProjection::entries(): filters * MoleculeDatasetSnapshots::global_announcements(): update SetInfo * utils::DataFrameExtraDataFieldsFilterApplier: extract * MoleculeAnnouncements::tail(): filters * GetDataRoomCollectionEntriesFilters -> GetMoleculeDataRoomCollectionEntriesFilters * A clearer separation of filter entities * MoleculeAnnouncements::tail(): filters * MoleculeProjectV2::get_data_room_activity_events(): filters * MoleculeProjectV2::get_data_room_activity_events(): filters [2] * MoleculeProjectV2::activity(): filters * MoleculeViewGlobalActivitiesUseCase: filters * schema.gql: regenerate * test_molecule_v2_activity(): start unlocking * test_molecule_v2_activity(): Activities are empty
…#1489) * Versioned files: sketched and plugged create/update use cases * Specialized use case for update file metadata. Integrated read file version use case, and simplified read model. * Minor: versioned file API moved out of data room file * Minor: avoid cloning file info for serde * Minor: unifying arguments of update/upload use cases * Clarified access checking in versioned file use cases * Isolated versioned file content access behind a service * Avoiding ResolvedDataset and similar in Molecule domain interface * MoleculeVersionedFile::asOf supported. Drafted MoleculeVersionedFile::matching (not public) - takes the versioned file version that exactly matches data room entry. MoleculeVersionedFile::latest is correctly not reusing denromalized data, as it's not guaranteed the data room entry is the latest one. * Revised schema optionals * Spelling * Enabled MoleculeVersionedFile::matching endpoint. Optimized MoleculeVersionedfile::latest endpoint, when data room entry is also the latest, using denormalized data. * Guiding comments
) * test_molecule_v2_activity(): Create a few versioned files * test_molecule_v2_activity(): Upload new file versions * test_molecule_v2_activity(): Link new file into the project data room -- not relevant for v2 * test_molecule_v2_activity(): Move a file (retract + append) * test_molecule_v2_activity(): Update a file (correction from-to) -- not relevant for v2 * test_molecule_v2_activity(): Create an announcement * test_molecule_v2_activity(): Upload a new file version * test_molecule_v2_activity(): Remove a file * test_molecule_v2_activity(): Check project activity events * test_molecule_v2_activity(): Create another project * test_molecule_v2_activity(): Create an announcement for the second project * test_molecule_v2_activity(): Check global activity events * test_molecule_v2_activity(): In-between activity asserts * test_gql_custom_molecule_v2: remove misleading clone() * test_molecule_v2_activity(): Filters without values * datafusion: register array functions * DataFrameExtraDataFieldsFilterApplier:: respect array columns * test_molecule_v2_activity(): Filters by tags: tag1 * test_molecule_v2_activity(): Filters by tags: [tag2] * test_molecule_v2_activity(): // Filters by tags: [tag2, tag1] * test_molecule_v2_activity(): Filters by categories: [test-category-1] * test_molecule_v2_activity(): Filters by categories: [test-category-2] * test_molecule_v2_activity(): Filters by categories: [test-category-2, test-category-1] * test_molecule_v2_activity(): Filters by access levels: [public] * test_molecule_v2_activity(): Filters by access levels: [holders] * test_molecule_v2_activity(): Filters by access levels: [public, holders] * test_molecule_v2_activity(): Filters combination: [test-tag2] AND [test-category-1] AND [holders] * test_molecule_v2_activity(): Project filters * test_molecule_v2_announcements_operations(): announcements filters * test_molecule_v2_data_room_operations(): announcements filters
…ouncements) * Extracted 'view project announcements' use case * Extracted "find project announcement" use case * Split Molecule dataset services * Moved most services from Molecule domain to services crate, broke all dependencies from API level * Minor cleanups * Sketched new approach of dataset accessor and used it to simplify announcements use cases for now * Same accessor approach applied to activities * Same accessor approach applied to projects dataset * Better reader/writer helpers for projects * Naming simplifications * Simplified announcements data model * Further model unifications: clear separation between changelog entry, changelog insertion record, payload record, and entity. Cleaned up event time / system time propagation in all Molecule write use cases. * Outbox events for announcements * Outbox event for activities * Activities: extracted view project activity use case, related structures cleanup * Minor correction
* kamu-molecule-domain: export utils as module * MoleculeSearchUseCaseImpl * molecule_extra_data_fields_filter() -> molecule_fields_filter() * GQL: MoleculeV2::search() * GQL: MoleculeV2::get_molecule_projects_mapping(): extract * extra_data_fields_filter.rs -> molecule_fields_filter.rs * MoleculeGlobalActivitiesService -> MoleculeGlobalDataRoomActivitiesService * schema.gql: regenerate * MoleculeSearchUseCaseImpl::get_global_data_room_activities_listing(): use projection * MoleculeSearchTypeInput: use Enum * MoleculeSemanticSearchFoundItem -> MoleculeSemanticSearchHit * MoleculeSearchUseCaseImpl: correct pattern & global data room projection * GQL: MoleculeV2::search(): return data room entries instead of files * test_molecule_v2_search(): Empty prompt * test_molecule_v2_search(): Prompt: "text" (files + announcement (body)) * test_molecule_v2_search(): Prompt: "tESt" (files + announcement (headline)) * test_molecule_v2_search(): Prompt: "bLaH" (only announcements (body)) * test_molecule_v2_search(): Prompt: "lain" (only files) * MoleculeSearchTypeInput: update * test_molecule_v2_search(): Filters: byIpnftUids: [PROJECT_1_UID] * test_molecule_v2_search(): Filters: byIpnftUids: [PROJECT_2_UID] * test_molecule_v2_search(): Filters: byIpnftUids: [PROJECT_2_UID, PROJECT_1_UID] * test_molecule_v2_search(): Filters: byType * test_molecule_v2_search(): Filters: byTags * test_molecule_v2_search(): Filters: byCategories * test_molecule_v2_search(): Filters: byAccessLevels * test_molecule_v2_search(): Filters combo * MoleculeSearchFilters::by_type(): represent as collection * MoleculeSearchTypeInput -> MoleculeSearchEntityKindInput * schema.gql: regenerate
Member
Author
|
Closing as Molecule did not provide any feedback on this proposal |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changed:
["public"]for extra safetyTODO:
In global search and activity APIs I think
byAccessLevelsis not enough - we need an ability to specify something like: