Skip to content

Fix Multiple Artifacts Processing Errors #1400

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
michaelgrace72 wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Fix Multiple Artifacts Processing Errors #1400

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
c4d2db7
fix: fix duplicate column name error in chromeLoginData artifact
michaelgrace72 Dec 8, 2025
773b441
fix: fix TypeError in cloudkitParticipants artifact with proper type …
michaelgrace72 Dec 8, 2025
1781010
fix: fix TypeError in cloudkitSharing artifact
michaelgrace72 Dec 8, 2025
11dc9e6
fix: fix OperationalError in FitnessWorkoutsLocationData artifact
michaelgrace72 Dec 8, 2025
c694143
fix: fix IndexError in health_workouts artifact
michaelgrace72 Dec 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion scripts/artifacts/FitnessWorkoutsLocationData.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ def get_Health(files_found, report_folder, seeker, wrap_text, timezone_offset):

cursor = db.cursor()

if does_table_exist_in_db(healthdb_secure, 'location_series_data') == True:
if does_table_exist_in_db(healthdb_secure, 'location_series_data') == True and does_table_exist_in_db(healthdb_secure, 'associations') == True:
Copy link

Copilot AI Dec 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The boolean comparison == True is redundant. In Python, boolean values can be used directly in conditional statements.

Change:

if does_table_exist_in_db(healthdb_secure, 'location_series_data') == True and does_table_exist_in_db(healthdb_secure, 'associations') == True:

To:

if does_table_exist_in_db(healthdb_secure, 'location_series_data') and does_table_exist_in_db(healthdb_secure, 'associations'):
Suggested change
if does_table_exist_in_db(healthdb_secure, 'location_series_data') == True and does_table_exist_in_db(healthdb_secure, 'associations') == True:
if does_table_exist_in_db(healthdb_secure, 'location_series_data') and does_table_exist_in_db(healthdb_secure, 'associations'):

Copilot uses AI. Check for mistakes.

# Fitness Workouts Location Data Analysis
Copy link

Copilot AI Dec 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment on line 33 is not properly indented. It should be indented to match the block it belongs to (inside the if statement).

Current:

if does_table_exist_in_db(...):

# Fitness Workouts Location Data Analysis

Should be:

if does_table_exist_in_db(...):
    # Fitness Workouts Location Data Analysis
Suggested change
# Fitness Workouts Location Data Analysis
# Fitness Workouts Location Data Analysis

Copilot uses AI. Check for mistakes.

Expand Down
2 changes: 1 addition & 1 deletion scripts/artifacts/chrome.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1231,7 +1231,7 @@ def chromeLoginData(files_found, report_folder, seeker, wrap_text, timezone_offs

lava_data_headers[0] = (lava_data_headers[0], 'datetime')

all_data_headers = lava_data_headers + ['Browser Name']
all_data_headers = lava_data_headers

report_file = 'Unknown'
for file_found in files_found:
Expand Down
50 changes: 39 additions & 11 deletions scripts/artifacts/cloudkitParticipants.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,17 +41,45 @@ def get_cloudkitParticipants(files_found, report_folder, seeker, wrap_text, time
for item in deserialized_plist:
if 'Participants' in item:
for participant in item['Participants']:
record_id = participant['UserIdentity']['UserRecordID']['RecordName']
email_address = participant['UserIdentity']['LookupInfo']['EmailAddress']
phone_number = participant['UserIdentity']['LookupInfo']['PhoneNumber']
first_name = participant['UserIdentity']['NameComponents']['NS.nameComponentsPrivate']['NS.givenName']
middle_name = participant['UserIdentity']['NameComponents']['NS.nameComponentsPrivate']['NS.middleName']
last_name = participant['UserIdentity']['NameComponents']['NS.nameComponentsPrivate']['NS.familyName']
name_prefix = participant['UserIdentity']['NameComponents']['NS.nameComponentsPrivate']['NS.namePrefix']
name_suffix = participant['UserIdentity']['NameComponents']['NS.nameComponentsPrivate']['NS.nameSuffix']
nickname = participant['UserIdentity']['NameComponents']['NS.nameComponentsPrivate']['NS.nickname']

user_dictionary[record_id] = [record_id, email_address, phone_number, name_prefix, first_name, middle_name, last_name, name_suffix, nickname]
try:
if not isinstance(participant, dict) or 'UserIdentity' not in participant:
continue

# must be dict and have UserIdentity
user_identity = participant.get('UserIdentity')
if not isinstance(user_identity, dict):
continue

user_record_id = user_identity.get('UserRecordID')
if not isinstance(user_record_id, dict):
continue

record_id = user_record_id.get('RecordName', '')
lookup_info = user_identity.get('LookupInfo', {})
email_address = lookup_info.get('EmailAddress', '') if isinstance(lookup_info, dict) else ''
phone_number = lookup_info.get('PhoneNumber', '') if isinstance(lookup_info, dict) else ''

name_components = user_identity.get('NameComponents', {})
if isinstance(name_components, dict):
name_private = name_components.get('NS.nameComponentsPrivate', {})
if isinstance(name_private, dict):
first_name = name_private.get('NS.givenName', '')
middle_name = name_private.get('NS.middleName', '')
last_name = name_private.get('NS.familyName', '')
name_prefix = name_private.get('NS.namePrefix', '')
name_suffix = name_private.get('NS.nameSuffix', '')
nickname = name_private.get('NS.nickname', '')
else:
first_name = middle_name = last_name = name_prefix = name_suffix = nickname = ''
else:
first_name = middle_name = last_name = name_prefix = name_suffix = nickname = ''

# only add valid entries
if record_id:
user_dictionary[record_id] = [record_id, email_address, phone_number, name_prefix, first_name, middle_name, last_name, name_suffix, nickname]
except (KeyError, TypeError, AttributeError) as e:
logfunc(f'Error processing participant data: {str(e)}')
Copy link

Copilot AI Dec 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error logging uses an f-string with str(e) which is redundant. When an exception is formatted in an f-string, it's automatically converted to a string.

Change:

logfunc(f'Error processing participant data: {str(e)}')

To:

logfunc(f'Error processing participant data: {e}')
Suggested change
logfunc(f'Error processing participant data: {str(e)}')
logfunc(f'Error processing participant data: {e}')

Copilot uses AI. Check for mistakes.
continue
db.close()

# Build the array after dealing with all the files
Expand Down
44 changes: 32 additions & 12 deletions scripts/artifacts/cloudkitSharing.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,18 +61,38 @@ def get_cloudkitServerSharedData(file_found, report_folder, seeker, wrap_text):
for item in deserialized_plist:
if 'Participants' in item:
for participant in item['Participants']:
record_id = participant['UserIdentity']['UserRecordID']['RecordName']
email_address = participant['UserIdentity']['LookupInfo']['EmailAddress']
phone_number = participant['UserIdentity']['LookupInfo']['PhoneNumber']
first_name = participant['UserIdentity']['NameComponents']['NS.nameComponentsPrivate']['NS.givenName']
middle_name = participant['UserIdentity']['NameComponents']['NS.nameComponentsPrivate']['NS.middleName']
last_name = participant['UserIdentity']['NameComponents']['NS.nameComponentsPrivate']['NS.familyName']
name_prefix = participant['UserIdentity']['NameComponents']['NS.nameComponentsPrivate']['NS.namePrefix']
name_suffix = participant['UserIdentity']['NameComponents']['NS.nameComponentsPrivate']['NS.nameSuffix']
nickname = participant['UserIdentity']['NameComponents']['NS.nameComponentsPrivate']['NS.nickname']

user_dictionary[record_id] = [record_id, email_address, phone_number, name_prefix, first_name,
middle_name, last_name, name_suffix, nickname]
try:
if not isinstance(participant, dict) or 'UserIdentity' not in participant:
continue

# must be dict and have UserIdentity
user_identity = participant.get('UserIdentity', {})
if not isinstance(user_identity, dict):
continue

user_record_id = user_identity.get('UserRecordID', {})
if not isinstance(user_record_id, dict):
continue

record_id = user_record_id.get('RecordName', '')
lookup_info = user_identity.get('LookupInfo', {})
email_address = lookup_info.get('EmailAddress', '') if isinstance(lookup_info, dict) else ''
phone_number = lookup_info.get('PhoneNumber', '') if isinstance(lookup_info, dict) else ''

name_components = user_identity.get('NameComponents', {})
name_private = name_components.get('NS.nameComponentsPrivate', {}) if isinstance(name_components, dict) else {}
first_name = name_private.get('NS.givenName', '') if isinstance(name_private, dict) else ''
middle_name = name_private.get('NS.middleName', '') if isinstance(name_private, dict) else ''
last_name = name_private.get('NS.familyName', '') if isinstance(name_private, dict) else ''
name_prefix = name_private.get('NS.namePrefix', '') if isinstance(name_private, dict) else ''
name_suffix = name_private.get('NS.nameSuffix', '') if isinstance(name_private, dict) else ''
nickname = name_private.get('NS.nickname', '') if isinstance(name_private, dict) else ''
Comment on lines +65 to +89
Copy link

Copilot AI Dec 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] The nested type checks are repetitive and could be simplified. Each .get() call is followed by an isinstance() check with early return, which adds unnecessary verbosity.

Consider using a more Pythonic approach with try-except or simplifying the nested checks:

try:
    user_identity = participant.get('UserIdentity', {})
    record_id = user_identity.get('UserRecordID', {}).get('RecordName', '')
    lookup_info = user_identity.get('LookupInfo', {})
    # ... continue with safe gets
except (AttributeError, TypeError):
    continue

This is more maintainable and follows Python's "easier to ask for forgiveness than permission" (EAFP) principle.

Suggested change
if not isinstance(participant, dict) or 'UserIdentity' not in participant:
continue
# must be dict and have UserIdentity
user_identity = participant.get('UserIdentity', {})
if not isinstance(user_identity, dict):
continue
user_record_id = user_identity.get('UserRecordID', {})
if not isinstance(user_record_id, dict):
continue
record_id = user_record_id.get('RecordName', '')
lookup_info = user_identity.get('LookupInfo', {})
email_address = lookup_info.get('EmailAddress', '') if isinstance(lookup_info, dict) else ''
phone_number = lookup_info.get('PhoneNumber', '') if isinstance(lookup_info, dict) else ''
name_components = user_identity.get('NameComponents', {})
name_private = name_components.get('NS.nameComponentsPrivate', {}) if isinstance(name_components, dict) else {}
first_name = name_private.get('NS.givenName', '') if isinstance(name_private, dict) else ''
middle_name = name_private.get('NS.middleName', '') if isinstance(name_private, dict) else ''
last_name = name_private.get('NS.familyName', '') if isinstance(name_private, dict) else ''
name_prefix = name_private.get('NS.namePrefix', '') if isinstance(name_private, dict) else ''
name_suffix = name_private.get('NS.nameSuffix', '') if isinstance(name_private, dict) else ''
nickname = name_private.get('NS.nickname', '') if isinstance(name_private, dict) else ''
user_identity = participant.get('UserIdentity', {})
user_record_id = user_identity.get('UserRecordID', {})
record_id = user_record_id.get('RecordName', '')
lookup_info = user_identity.get('LookupInfo', {})
email_address = lookup_info.get('EmailAddress', '')
phone_number = lookup_info.get('PhoneNumber', '')
name_components = user_identity.get('NameComponents', {})
name_private = name_components.get('NS.nameComponentsPrivate', {})
first_name = name_private.get('NS.givenName', '')
middle_name = name_private.get('NS.middleName', '')
last_name = name_private.get('NS.familyName', '')
name_prefix = name_private.get('NS.namePrefix', '')
name_suffix = name_private.get('NS.nameSuffix', '')
nickname = name_private.get('NS.nickname', '')

Copilot uses AI. Check for mistakes.

if record_id: # Only add valid entries
user_dictionary[record_id] = [record_id, email_address, phone_number, name_prefix, first_name, middle_name, last_name, name_suffix, nickname]
except (KeyError, TypeError, AttributeError) as e:
logfunc(f'Error processing participant data: {str(e)}')
Copy link

Copilot AI Dec 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error logging uses an f-string with str(e) which is redundant. When an exception is formatted in an f-string, it's automatically converted to a string.

Change:

logfunc(f'Error processing participant data: {str(e)}')

To:

logfunc(f'Error processing participant data: {e}')
Suggested change
logfunc(f'Error processing participant data: {str(e)}')
logfunc(f'Error processing participant data: {e}')

Copilot uses AI. Check for mistakes.
continue
db.close()

# Build the array after dealing with all the files
Expand Down
30 changes: 21 additions & 9 deletions scripts/artifacts/health.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -533,19 +533,31 @@ def health_workouts(context):
for record in db_records:
start_timestamp = convert_cocoa_core_data_ts_to_utc(record[0])
end_timestamp = convert_cocoa_core_data_ts_to_utc(record[1])
added_timestamp = convert_cocoa_core_data_ts_to_utc(record[26])
device_model = context.get_device_model(record[22])

# Check if record has enough elements to avoid IndexError
added_timestamp = convert_cocoa_core_data_ts_to_utc(record[26]) if len(record) > 26 else ''
device_model = context.get_device_model(record[22]) if len(record) > 22 else ''

if record[16]:
celcius_temp = ''
if len(record) > 16 and record[16]:
celcius_temp = round(((record[16] - 32) * (5 / 9)), 2)

# safe access
data_list.append(
(start_timestamp, end_timestamp, str(record[2]).title(), record[3],
record[4], record[5], record[6], record[7], record[8], record[9],
record[10], record[11], record[12], record[13], record[14],
record[15], celcius_temp, record[16], record[17], record[18],
record[19], record[20], record[21], record[22], device_model,
record[23], record[24], record[25], added_timestamp)
(start_timestamp, end_timestamp, str(record[2]).title() if len(record) > 2 else '',
record[3] if len(record) > 3 else '', record[4] if len(record) > 4 else '',
record[5] if len(record) > 5 else '', record[6] if len(record) > 6 else '',
record[7] if len(record) > 7 else '', record[8] if len(record) > 8 else '',
record[9] if len(record) > 9 else '', record[10] if len(record) > 10 else '',
record[11] if len(record) > 11 else '', record[12] if len(record) > 12 else '',
record[13] if len(record) > 13 else '', record[14] if len(record) > 14 else '',
record[15] if len(record) > 15 else '', celcius_temp,
record[16] if len(record) > 16 else '', record[17] if len(record) > 17 else '',
record[18] if len(record) > 18 else '', record[19] if len(record) > 19 else '',
record[20] if len(record) > 20 else '', record[21] if len(record) > 21 else '',
record[22] if len(record) > 22 else '', device_model,
record[23] if len(record) > 23 else '', record[24] if len(record) > 24 else '',
record[25] if len(record) > 25 else '', added_timestamp)
Comment on lines +547 to +560
Copy link

Copilot AI Dec 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The bounds checking logic is inconsistent with DRY (Don't Repeat Yourself) principle. Each index is checked individually with len(record) > N, resulting in significant code duplication across 26+ checks. This approach is error-prone and makes the code harder to maintain.

Consider refactoring to use a helper function or list comprehension:

def safe_get(record, index, default=''):
    return record[index] if len(record) > index else default

# Then use: safe_get(record, 3), safe_get(record, 4), etc.

Or pad the record to expected length:

# Pad record to expected length
padded_record = list(record) + [''] * (28 - len(record))

Copilot uses AI. Check for mistakes.
)
Comment on lines 546 to 561
Copy link

Copilot AI Dec 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] The tuple construction spans multiple lines without proper formatting. The opening parenthesis on line 546 starts the tuple, but line 547 begins the first element without proper indentation alignment. This makes the code harder to read and maintain.

Consider formatting it as:

data_list.append((
    start_timestamp, end_timestamp, str(record[2]).title() if len(record) > 2 else '',
    record[3] if len(record) > 3 else '', record[4] if len(record) > 4 else '',
    # ... rest of elements
    record[25] if len(record) > 25 else '', added_timestamp
))

Copilot uses AI. Check for mistakes.

return data_headers, data_list, data_source
Expand Down