Add journal info extractor CLI tool for DOCX to XLSX conversion

[Rossi-Luciano]

Descrição do PR:

O que esse PR faz?

Este PR adiciona uma nova ferramenta CLI ao packtools para extrair informações estruturadas de arquivos DOCX de periódicos científicos e gerar relatórios em formato XLSX. A ferramenta processa documentos multilíngues (português, espanhol e inglês) contendo informações sobre periódicos científicos e extrai duas categorias principais de dados:

1. Seções do documento (About the Journal, Editorial Policy, Bibliographic Record, etc.) com normalização para inglês
2. Corpo editorial completo com metadados dos membros (nomes, afiliações, cidades, estados, países, roles, ORCID, Lattes, emails)

Onde a revisão poderia começar?

Iniciar a revisão pelo arquivo principal:

packtools/journal_info_extractor.py

Como este poderia ser testado manualmente?

1. Instalar dependências:

pip install python-docx>=0.8.11 openpyxl>=3.0.10

2. Reinstalar o packtools:

cd packtools
pip install -e .

3. Preparar arquivos de teste (formato esperado: YYYYMMDD_ACRONYM_*_LANGUAGE_ok.docx)

4. Executar o extrator:

journal-extractor /path/to/docx_files --output /path/to/output --loglevel DEBUG

5. Verificar os arquivos gerados:

   • TIMESTAMP-sections.xlsx: deve conter todas as seções normalizadas
   • TIMESTAMP-editorial_board.xlsx: deve conter membros com roles corretos

6. Validar que membros do corpo editorial têm roles específicos (ex: "Associate Editors: Theoretical Physics...") e não roles genéricos incorretos

7. Verificar campos obrigatórios na planilha: title_journal, issn_scielo, affiliation, given_names, last_name, country_code, state_name, city_name, std_role

Algum cenário de contexto que queira dar?

N.A.

Screenshots

N.A.

Quais são tickets relevantes?

N.A.

Referências

1. Estrutura do projeto packtools: packtools/data_checker.py, packtools/htmlgenerator.py
2. Biblioteca python-docx: https://python-docx.readthedocs.io/
3. Biblioteca openpyxl: https://openpyxl.readthedocs.io/
4. Padrões de metadados SciELO para periódicos científicos
5. Arquivos de exemplo: 20251010_RBEF_Total_Página_Informativa_inglês_ok.docx, 20251010_RBEF_Total_Página_Informativa_português_ok.docx

Nota: Este desenvolvimento utilizou Claude (Anthropic) como ferramenta auxiliar para análise, implementação e debug. Todo código foi revisado, testado e validado pelo desenvolvedor.

[Copilot]

Pull request overview

This PR introduces a comprehensive CLI tool for extracting structured journal information from multilingual DOCX files and generating XLSX reports. The tool processes scientific journal documents in Portuguese, Spanish, and English, normalizing section names to English and extracting both document sections and editorial board member details.

Key Changes

• New journal-extractor CLI command for DOCX to XLSX conversion
• Multi-stage extraction pipeline: metadata parsing, section extraction, editorial board parsing, and XLSX report generation
• Support for complex data structures including multilingual section mapping and detailed member metadata (names, affiliations, ORCID, Lattes, emails)

Reviewed changes

Copilot reviewed 2 out of 3 changed files in this pull request and generated 21 comments.

File	Description
setup.py	Adds new CLI entry point journal-extractor to console_scripts
requirements.txt	Adds duplicate python-docx dependency (already specified on line 16)
packtools/journal_info_extractor.py	Complete implementation of journal info extraction tool with section/editorial board extractors, XLSX generators, and CLI interface (1238 lines)

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The column letter calculation chr(64 + col_idx) will fail when col_idx > 26 (i.e., more than 25 files processed, since col_idx starts at 2). Excel columns beyond 'Z' require two letters (e.g., 'AA', 'AB'). This will cause a bug if many DOCX files are processed together.

Consider using openpyxl's built-in get_column_letter() function:

from openpyxl.utils import get_column_letter
col_letter = get_column_letter(col_idx)
ws.column_dimensions[col_letter].width = 60

[Copilot]

Overly broad exception handling. The except Exception as e: blocks catch all exceptions including system errors like KeyboardInterrupt and SystemExit. These should catch more specific exceptions to avoid masking serious issues. Consider catching specific exceptions like KeyError, IndexError, or ValueError depending on the expected failure modes.

[Copilot]

No test coverage for the new journal_info_extractor module. The repository has comprehensive test coverage for other modules (e.g., test_htmlgenerator.py, test_utils.py, test_checks.py), but there are no tests for the new functionality. Consider adding tests for:

• extract_metadata_from_filename() with various filename patterns
• Section extraction logic (SectionExtractor class)
• Editorial board member parsing (EditorialBoardExtractor class)
• XLSX report generation (XLSXReportGenerator class)
• Edge cases like malformed DOCX files, missing sections, invalid data formats

[Copilot]

Hardcoded default values for country ('Brasil') and country_code ('BR') create an inappropriate assumption. Not all editorial board members will be from Brazil. These fields should remain empty strings and be properly extracted from the document data, or the code should handle missing country information gracefully without assuming a default.

Suggested change

	'country': 'Brasil', # Default
	'country_code': 'BR', # Default
	'country': '',
	'country_code': '',

[Copilot]

Fragile email extraction logic with potential for incorrect parsing. The loop at lines 824-831 attempts to extract email by iterating character-by-character and stopping at the first '.' before '@', but this will fail for common email patterns like 'john.doe@example.com' where the '.' before '@' is part of the valid email address. Consider using a regex pattern or the standard library's email.utils for more robust email extraction.

Suggested change

	email_part = text.split("e-mail:")[-1].strip()
	email = email_part.replace("_", "").strip()
	if "." in email:
	email_clean = ""
	for char in email:
	if char == ".":
	if "@" in email_clean:
	email_clean += char
	else:
	break
	else:
	email_clean += char
	if "@" in email_clean:
	member['email'] = email_clean
	email_part = text.split("e-mail:")[-1].replace("_", "").strip()
	# Use regex to extract email address
	match = re.search(r'([a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+)', email_part)
	if match:
	member['email'] = match.group(1).strip()

[Copilot]

[nitpick] Inconsistent docstring style. The project appears to use triple-quoted docstrings for functions (based on packtools/htmlgenerator.py and other files), but this module uses a mix of formats. The module docstring spans lines 3-16 with proper formatting, but it would be more consistent to follow PEP 257 conventions more closely with a summary line, blank line, then detailed description.

[Copilot]

Missing handling for case when location_parts has exactly 2 elements. The code handles cases with 3, 4, and 5+ elements (lines 713-762), but there's no explicit handling for when len(location_parts) == 2. This would fall through to the catch-all on line 763, which only sets institution. Consider adding explicit handling for the 2-element case or documenting that this is intentional.

Suggested change

	elif len(location_parts) >= 1:
	# Só tem instituição e país
	member['institution'] = location_parts[0].strip() if len(location_parts) > 1 else ''
	elif len(location_parts) == 2:
	# Instituição e país
	member['institution'] = location_parts[0].strip()
	member['country'] = location_parts[1].strip()
	elif len(location_parts) >= 1:
	# Só tem instituição
	member['institution'] = location_parts[0].strip()

[Copilot]

Duplicate detection may not work correctly for members with missing data. The tuple on lines 942-948 includes member.get('date', '') and member.get('acronym', ''), but if these fields are missing from some members, duplicates could be incorrectly identified or missed. Additionally, if a person has the same name and role but in different documents (different dates/acronyms), they will be treated as separate entries. Consider if this is the intended behavior or if the deduplication logic needs refinement.

Suggested change

	member.get('role', ''),
	member.get('date', ''),
	member.get('acronym', '')
	member.get('role', '')

[Copilot]

Magic number 64 makes the code harder to understand. The expression chr(64 + col_idx) uses 64 because it's the ASCII code for '@' (and 65 is 'A'), but this is not immediately clear. Consider adding a comment explaining the calculation or using a more explicit constant like ord('A') - 1 to make the intent clearer.

Suggested change

	col_letter = chr(64 + col_idx)
	# Convert column index to Excel column letter (A=1, B=2, ...)
	col_letter = chr(ord('A') - 1 + col_idx)

[Copilot]

Variable in_editorial_board is not used.