Add personal data and confidentiality indicators (fixes #58, #59)

[deeeeeepesh]

Summary

This PR implements two feature requests:

• Fixes New attribute : contain personal data #58 - New attribute: contain personal data
• Fixes New attribute : confidential indicator #59 - New attribute: confidential indicator

Changes

Added two new optional boolean attributes to both Field and Dataset schemas:

Attribute	Purpose
contains_personal_data	Flags fields/datasets containing PII for privacy compliance (GDPR, CCPA)
is_confidential	Indicates data that should not be exposed to LLMs or external tools

Files Modified

• core-spec/osi-schema.json - JSON Schema definitions
• core-spec/spec.yaml - YAML specification
• core-spec/spec.md - Documentation with examples

Example Usage

fields:
  - name: email
    expression:
      dialects:
        - dialect: ANSI_SQL
          expression: email
    contains_personal_data: true
    is_confidential: false
    
  - name: ssn
    expression:
      dialects:
        - dialect: ANSI_SQL
          expression: social_security_number
    contains_personal_data: true
    is_confidential: true

[ananthdurai]

Should this be extended to classification_type rather than just an indicator of whether it is a personal type or not? Or the expression should be used as a classification type? Here are some of the classification type can be considered as contains_personal_data
https://github.com/ananthdurai/schemata/blob/main/src/opencontract/v1/org/schemata/protobuf/schemata.proto#L87

[khush-bhatia]

Does GDPR have an official standard for data classification ?

[khush-bhatia]

deeeeeepesh Thanks for this PR. But there is a catalog working group that is figuring out a full proposal for catalog , sensitive data classification and governance. Let's wait for the proposal from that working group to update the spec cohesively. You are welcome to join the working group as well.

Slack Join link