Skip to content

Conversation

@JadeCara
Copy link
Contributor

@JadeCara JadeCara commented Dec 12, 2025

Ticket ENG-2185

Description Of Changes

This ticket was originally asking for consent manual tasks. We are going forward with a work around #7102 because consent task graphs and access/erasure task graphs are different and have different capabilities - especially around conditional dependencies.

There are going to be a few PRs cleaning up some un-used functionality or adjusting functionality for improved usage patterns. This PR removes the Manual Tasks Logs table as the first step in getting manual tasks and conditions into shape for the above requirements. It is a clean up based on the first few rounds of customer use and learning actual usage patterns.

This PR:

  • The logging being used by manual tasks and front end data is entirely the general audit logs so the manual task log table and utilities are now redundant.

Future PRs:

  • Update the Conditional Dependency table with a JSONB dictionary column. Allowing us to test the migration without being destructive [ENG-2185] #7133
  • Allow conditional dependencies to be assigned to a specific policy type or id
  • Allow users to be assigned to specific submissions rather than the whole task.

Code Changes

  • .fides/db_dataset.yml - removed ManualTaskLog table
  • src/fides/api/alembic/migrations/versions/ remove the manual task table.
  • src/fides/api/db/base.py - removed ManualTaskLog
  • src/fides/api/models/manual_task/manual_task.py removed the log class and updated the functions where manual task logs were created.
  • Updated all tests removing the manual task log.

Steps to Confirm

  1. There should be NO change in functionality.
  2. Manual tasks, conditional dependencies etc should continue to function as before.
  3. Please test with FidesPlus 2920
  4. Run FidesPlus branch above pointed at this branch.
  5. Create a ManualTask with several submissions required and several conditions, both dataset and privacy request based.
  6. Create privacy requests that meet and do not meet the requirements. Verify that they create/do not create as expected.
  7. Verify that you can complete the manual tasks with no errors.
  8. Verify that you can see the skipped logs and the manual task and manual task actions appear on the DSR activity log.
  9. Verify that you receive the access package with all expected manual task inclusions (attachments and text)
  10. Update the ManualTask submission types, delete one or more.
  11. Delete manual task integration

Pre-Merge Checklist

  • Issue requirements met
  • All CI pipelines succeeded
  • CHANGELOG.md updated
    • Add a db-migration This indicates that a change includes a database migration label to the entry if your change includes a DB migration
    • Add a high-risk This issue suggests changes that have a high-probability of breaking existing code label to the entry if your change includes a high-risk change (i.e. potential for performance impact or unexpected regression) that should be flagged
    • Updates unreleased work already in Changelog, no new entry necessary
  • UX feedback:
    • All UX related changes have been reviewed by a designer
    • No UX review needed
  • Followup issues:
    • Followup issues created
    • No followup issues
  • Database migrations:
    • Ensure that your downrev is up to date with the latest revision on main
    • Ensure that your downgrade() migration is correct and works
      • If a downgrade migration is not possible for this change, please call this out in the PR description!
    • No migrations
  • Documentation:
    • Documentation complete, PR opened in fidesdocs
    • Documentation issue created in fidesdocs
    • If there are any new client scopes created as part of the pull request, remember to update public-facing documentation that references our scope registry
    • No documentation updates required

@vercel
Copy link

vercel bot commented Dec 12, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

2 Skipped Deployments
Project Deployment Review Updated (UTC)
fides-plus-nightly Ignored Ignored Preview Dec 18, 2025 2:33pm
fides-privacy-center Ignored Ignored Dec 18, 2025 2:33pm

@codecov
Copy link

codecov bot commented Dec 12, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.22%. Comparing base (f6ed3ee) to head (de3205e).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #7124      +/-   ##
==========================================
- Coverage   87.26%   87.22%   -0.04%     
==========================================
  Files         533      533              
  Lines       35161    35126      -35     
  Branches     4083     4082       -1     
==========================================
- Hits        30683    30639      -44     
- Misses       3591     3597       +6     
- Partials      887      890       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@JadeCara JadeCara added the db-migration This indicates that a change includes a database migration label Dec 15, 2025
@JadeCara JadeCara marked this pull request as ready for review December 15, 2025 15:34
@JadeCara JadeCara requested a review from a team as a code owner December 15, 2025 15:34
@JadeCara JadeCara requested review from galvana and removed request for a team December 15, 2025 15:34
@greptile-apps
Copy link
Contributor

greptile-apps bot commented Dec 15, 2025

Greptile Overview

Greptile Summary

This PR removes the ManualTaskLog table and all associated functionality as part of cleaning up unused manual task features based on customer usage patterns.

Key Changes:

  • Drops the manual_task_log database table via migration (lossy - data cannot be restored on downgrade)
  • Removes the ManualTaskLog ORM model from manual_task.py
  • Deletes two entire test files (test_manual_task_log.py and test_logger_context_utils.py) totaling 569 lines
  • Removes log creation code from config creation methods
  • Updates test files to remove assertions about log creation

Migration Details:
The migration provides a complete downgrade() path that recreates the table structure, but as noted in the CHANGELOG, this is a lossy migration - existing data will not be restored.

Rationale:
According to the PR description, the frontend primarily uses general audit logs rather than the specialized manual task logs, making this table redundant. This cleanup prepares the codebase for upcoming conditional dependency improvements.

Confidence Score: 5/5

  • This PR is safe to merge - it's a clean removal of unused functionality with proper migration and test cleanup
  • The changes are straightforward and well-executed: dropping an unused table, removing the corresponding model, and cleaning up all test references. The migration provides a proper downgrade path, the CHANGELOG is properly documented with the db-migration label, and all code removal is complete with no dangling references.
  • No files require special attention

Important Files Changed

File Analysis

Filename Score Overview
src/fides/api/alembic/migrations/versions/xx_2025_12_12_1822_2c067179b727_remove_redundant_manual_task_logger.py 5/5 Drops manual_task_log table and all associated indexes; downgrade fully recreates table structure
src/fides/api/models/manual_task/manual_task.py 5/5 Removed ManualTaskLog class and all logging functionality; cleaned up unused log creation code in config creation
tests/api/models/manual_task/test_manual_task_log.py 5/5 Deleted entire test file (190 lines) that tested ManualTaskLog functionality
tests/ops/util/test_logger_context_utils.py 5/5 Deleted entire test file (379 lines) that tested logger context utilities
CHANGELOG.md 5/5 Added changelog entry documenting ManualTaskLog table removal with db-migration label and lossy migration warning

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

11 files reviewed, no comments

Edit Code Review Agent Settings | Greptile

@JadeCara JadeCara mentioned this pull request Dec 16, 2025
18 tasks
@JadeCara JadeCara added this pull request to the merge queue Dec 18, 2025
Merged via the queue into main with commit d9d6a6d Dec 18, 2025
69 checks passed
@JadeCara JadeCara deleted the ENG-2185-migration-remove-redundant-logging branch December 18, 2025 16:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

db-migration This indicates that a change includes a database migration

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants