Skip to content

テストの修正: PostgreSQL対応とsort_noカラム追加 #17

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
nobuhiko merged 24 commits into from
Jan 13, 2026
Merged

テストの修正: PostgreSQL対応とsort_noカラム追加 #17

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
a2b6677
テストの修正: PostgreSQL対応とsort_noカラム追加
nobuhiko Jan 12, 2026
8e53389
テストの修正: creator_id外部キー制約エラーの修正
nobuhiko Jan 12, 2026
0ccc8d4
テストの修正: rankカラム追加と不要なサービス取得を削除
nobuhiko Jan 12, 2026
a850d08
テストの修正: dtb_member.csvのidをmember_idに変更
nobuhiko Jan 12, 2026
66d4edb
テストの修正: del_flgカラム追加とカラム名を旧バージョン形式に変更
nobuhiko Jan 12, 2026
3e5d2cb
テストの修正: logoutTo()メソッドを削除し、シンプルなインポート確認に変更
nobuhiko Jan 12, 2026
9dbbbc3
テストの修正: dtb_member.csvにrankカラムを追加
nobuhiko Jan 12, 2026
6ea670d
テストの修正: 元のテスト構造に統合してメンバーインポートをテスト
nobuhiko Jan 12, 2026
8e5af35
テストの修正: クラスの閉じ波括弧を追加
nobuhiko Jan 12, 2026
c89dc2c
全バージョンのテストを有効化
nobuhiko Jan 12, 2026
35863a0
4.0→4.2/4.3マイグレーション対応: two_factor_auth_enabledカラムのデフォルト値設定
nobuhiko Jan 12, 2026
38ec760
fix4x()にもtwo_factor_auth_enabled/keyの処理を追加
nobuhiko Jan 12, 2026
cfb30c6
fix4x関数で4.0/4.1からの移行時のカラムマッピングを修正
nobuhiko Jan 12, 2026
c66b1ad
4.0/4.1移行時のタイムスタンプとカラムマッピング修正
nobuhiko Jan 12, 2026
76a0097
4.0/4.1移行時のtwo_factor_auth_enabled NOT NULL制約違反を修正
nobuhiko Jan 12, 2026
60aa45e
isset()バグ修正: array_key_exists()に変更
nobuhiko Jan 12, 2026
fd9522d
PostgreSQLで外部キー制約を遅延させる対応
nobuhiko Jan 12, 2026
ab3a608
PostgreSQLのresetTable()でTRUNCATE CASCADEを使用
nobuhiko Jan 12, 2026
50ec6ef
PostgreSQLのresetTable()でセーブポイントを使用
nobuhiko Jan 12, 2026
568348e
PostgreSQL 4.0/4.1移行時に全テーブルを一括TRUNCATE
nobuhiko Jan 13, 2026
e25fee3
PostgreSQL TRUNCATE前にテーブル存在確認を追加
nobuhiko Jan 13, 2026
46af4ca
PostgreSQL TRUNCATE時にマスタテーブル(mtb_*)を除外
nobuhiko Jan 13, 2026
72205d8
PostgreSQL fix4x()をUPSERT方式に変更
nobuhiko Jan 13, 2026
b0c0773
DataMigrationServiceのコメントを更新
nobuhiko Jan 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
113 changes: 105 additions & 8 deletions Controller/Admin/ConfigController.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,7 @@ public function index(Request $request, Connection $em)
]);

// $csvDir 内のファイルをすべて読み込む
// PostgreSQLはUPSERT方式を使うため、TRUNCATE不要
$files = scandir($csvDir);
foreach ($files as $file) {
// csvファイルのみ処理
Expand Down Expand Up @@ -343,6 +344,12 @@ private function saveToC($em, $tmpDir, $csvName, $tableName = null, $allow_zero
$value[$column] = !empty($data[$column]) ? $data[$column] : null;
} elseif ($column == 'point') {
$value[$column] = empty($data[$column]) ? 0 : (int) $data[$column];
} elseif ($column == 'two_factor_auth_enabled' && ($tableName == 'dtb_member' || $tableName == 'dtb_customer')) {
// 4.0系には存在しないカラム。デフォルト値として0（無効）を設定
$value[$column] = isset($data[$column]) ? $data[$column] : 0;
} elseif ($column == 'two_factor_auth_key' && ($tableName == 'dtb_member' || $tableName == 'dtb_customer')) {
// 4.0系には存在しないカラム。NULLを設定
$value[$column] = isset($data[$column]) ? $data[$column] : null;
} elseif ($allow_zero) {
$value[$column] = isset($data[$column]) ? $data[$column] : null;
} else {
Expand Down Expand Up @@ -399,6 +406,9 @@ private function saveToC($em, $tmpDir, $csvName, $tableName = null, $allow_zero
$value[$column] = !empty($data[$column]) ? $data[$column] : null;
} elseif ($column == 'creator_id') {
$value[$column] = !empty($data[$column]) ? $data[$column] : 1;
} elseif ($column == 'two_factor_auth_enabled' && $tableName == 'dtb_member') {
// 4.0系には存在しないカラム。デフォルト値として0（無効）を設定
$value[$column] = isset($data[$column]) ? $data[$column] : 0;
} elseif ($column == 'plg_mailmagazine_flg') {
$value[$column] = (!empty($data['mailmaga_flg']) && $data['mailmaga_flg'] != 3) ? 1 : 0;
} elseif ($column == 'id' && $tableName == 'dtb_member') {
Expand Down Expand Up @@ -1050,6 +1060,15 @@ protected function upsertAuthorityAndMember($em, $dir)
$insertValues[$col] = $data[$col] ?? 'member';
continue;
}
// 4.0系のカラム名マッピング
if ($col === 'work' && !array_key_exists($col, $data) && array_key_exists('work_id', $data)) {
$insertValues[$col] = $data['work_id'];
continue;
}
if ($col === 'authority' && !array_key_exists($col, $data) && array_key_exists('authority_id', $data)) {
$insertValues[$col] = $data['authority_id'];
continue;
}
if (array_key_exists($col, $data)) {
$insertValues[$col] = $data[$col];
} else {
Expand All @@ -1064,6 +1083,19 @@ protected function upsertAuthorityAndMember($em, $dir)
$insertValues[$dcol] = $now;
}
}
// login_dateなどのNULL許可のタイムスタンプカラムは、空文字列をnullに変換
foreach (['login_date', 'first_buy_date', 'last_buy_date', 'payment_date'] as $dcol) {
if (isset($insertValues[$dcol]) && (empty($insertValues[$dcol]) || strpos($insertValues[$dcol], '0000') === 0)) {
$insertValues[$dcol] = null;
}
}
// 4.0系には存在しないカラムのデフォルト値を設定
if (array_key_exists('two_factor_auth_enabled', $insertValues) && $insertValues['two_factor_auth_enabled'] === null) {
$insertValues['two_factor_auth_enabled'] = 0;
}
if (array_key_exists('two_factor_auth_key', $insertValues) && $insertValues['two_factor_auth_key'] === null) {
$insertValues['two_factor_auth_key'] = null; // NULL許可（この行は冗長だが明示的に残す）
}
$colsSql = implode(',', array_map(fn($c) => '"' . $c . '"', array_keys($insertValues)));
$placeholders = implode(',', array_fill(0, count($insertValues), '?'));
$updateSql = implode(', ', array_map(fn($c) => '"' . $c . '" = EXCLUDED."' . $c . '"', $updateCols));
Expand Down Expand Up @@ -2153,7 +2185,22 @@ private function fix4x($em, $tmpDir, $csvName)
}

$platform = $this->dataMigrationService->begin($em);
$this->dataMigrationService->resetTable($em, $tableName);

// PostgreSQLではUPSERTを使うため、resetTableは不要
// MySQLは従来通りresetTable()を使用
if ($platform !== 'postgresql') {
$this->dataMigrationService->resetTable($em, $tableName);
}

// PostgreSQLの場合、UPSERT用のプライマリキーを取得
$primaryKeys = [];
if ($platform === 'postgresql') {
$schemaManager = $em->getSchemaManager();
$table = $schemaManager->introspectTable($tableName);
if ($table->hasPrimaryKey()) {
$primaryKeys = $table->getPrimaryKey()->getColumns();
}
}

$builder = new BulkInsertQuery($em, $tableName);
$builder->setColumns($listTableColumns);
Expand All @@ -2176,30 +2223,80 @@ private function fix4x($em, $tmpDir, $csvName)
foreach ($columns as $column) {

$columnName = $column->getName();
if ($column->getNotNull()) {

// 特定カラムの処理
if ($columnName == 'two_factor_auth_enabled' && ($tableName == 'dtb_member' || $tableName == 'dtb_customer')) {
// 4.0系には存在しないカラム。デフォルト値として0（無効）を設定
$value[$columnName] = isset($data[$columnName]) && $data[$columnName] !== '' ? $data[$columnName] : 0;
} elseif ($columnName == 'two_factor_auth_key' && ($tableName == 'dtb_member' || $tableName == 'dtb_customer')) {
// 4.0系には存在しないカラム。NULLを設定
$value[$columnName] = isset($data[$columnName]) && $data[$columnName] !== '' ? $data[$columnName] : null;
} elseif ($columnName == 'work' && $tableName == 'dtb_member') {
// 4.0系ではwork_idというカラム名
$value[$columnName] = isset($data['work_id']) && $data['work_id'] !== '' ? $data['work_id'] : null;
} elseif ($columnName == 'authority' && $tableName == 'dtb_member') {
// 4.0系ではauthority_idというカラム名
$value[$columnName] = isset($data['authority_id']) && $data['authority_id'] !== '' ? $data['authority_id'] : null;
} elseif ($columnName == 'create_date' || $columnName == 'update_date') {
// create_date/update_dateは、空または'0000-00-00 00:00:00'の場合は現在時刻を設定
$value[$columnName] = (isset($data[$columnName]) && $data[$columnName] !== '' && $data[$columnName] != '0000-00-00 00:00:00') ? $data[$columnName] : date('Y-m-d H:i:s');
} elseif ($columnName == 'login_date' || $columnName == 'first_buy_date' || $columnName == 'last_buy_date' || $columnName == 'payment_date') {
// タイムスタンプ型カラムで、NULL許可の場合は、空または'0000-00-00 00:00:00'の場合はnullを設定
$value[$columnName] = (isset($data[$columnName]) && $data[$columnName] !== '' && $data[$columnName] != '0000-00-00 00:00:00') ? $data[$columnName] : null;
} elseif ($columnName == 'sex_id' || $columnName == 'job_id' || $columnName == 'country_id' || $columnName == 'pref_id') {
// 外部キー制約があるカラムは、空の場合nullを設定（0を設定すると外部キー違反になる）
$value[$columnName] = isset($data[$columnName]) && $data[$columnName] !== '' ? $data[$columnName] : null;
} elseif ($columnName == 'discriminator_type') {
// discriminator_typeは、テーブル名から生成
$search = ['dtb_', 'mtb_', '_'];
$value[$columnName] = str_replace($search, '', $tableName);
} elseif ($column->getNotNull()) {
$value[$columnName] = isset($data[$columnName]) && $data[$columnName] !== '' ? $data[$columnName] : 0;
} else {
$value[$columnName] = isset($data[$columnName]) && $data[$columnName] !== '' ? $data[$columnName] : null;
}
}

$builder->setValues($value);

if (($i % $batchSize) === 0) {
if ($platform === 'postgresql' && !empty($primaryKeys)) {
// PostgreSQLはUPSERTで行ごとに処理
try {
$builder->execute();
$this->addSuccess($tableName, 'admin');
$cols = array_map(fn($c) => '"' . $c . '"', array_keys($value));
$placeholders = array_fill(0, count($value), '?');
$updateCols = array_filter(array_keys($value), fn($c) => !in_array($c, $primaryKeys));
$updateSet = array_map(fn($c) => '"' . $c . '" = EXCLUDED."' . $c . '"', $updateCols);
$conflictCols = array_map(fn($c) => '"' . $c . '"', $primaryKeys);

$sql = 'INSERT INTO "' . $tableName . '" (' . implode(', ', $cols) . ') ' .
'VALUES (' . implode(', ', $placeholders) . ') ' .
'ON CONFLICT (' . implode(', ', $conflictCols) . ') ' .
'DO UPDATE SET ' . implode(', ', $updateSet);

$em->executeStatement($sql, array_values($value));
} catch (\Exception $e) {
$this->addDanger($e->getMessage(), 'admin');
$em->rollback();
return;
}
} else {
// MySQLはバッチINSERT
$builder->setValues($value);

if (($i % $batchSize) === 0) {
try {
$builder->execute();
$this->addSuccess($tableName, 'admin');
} catch (\Exception $e) {
$this->addDanger($e->getMessage(), 'admin');
$em->rollback();
return;
}
}
}

$i++;
}

if (count($builder->getValues()) > 0) {
if ($platform !== 'postgresql' && count($builder->getValues()) > 0) {
try {
$builder->execute();
$this->addSuccess($tableName, 'admin');
Expand Down
12 changes: 11 additions & 1 deletion Service/DataMigrationService.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,10 @@ public function resetTable(Connection $em, $tableName)

if ($platform == 'mysql') {
$em->exec('DELETE FROM ' . $tableName);
} elseif ($platform == 'postgresql') {
// PostgreSQLでは fix4x() はUPSERTを使うため、このメソッドは呼ばれない
// saveToC() などから呼ばれる場合はDELETEを実行
$em->exec('DELETE FROM "' . $tableName . '"');
} else {
$em->exec('DELETE FROM ' . $tableName);
}
Expand Down Expand Up @@ -271,7 +275,13 @@ public function begin($em, $context = NULL)
if ($platform == 'mysql') {
$em->exec('SET FOREIGN_KEY_CHECKS = 0;');
$em->exec("SET SESSION sql_mode = 'NO_AUTO_VALUE_ON_ZERO'"); // STRICT_TRANS_TABLESを無効にする。
} else {
} elseif ($platform == 'postgresql') {
// PostgreSQLでは外部キー制約チェックをトランザクション終了時まで遅延
// fix4x()ではUPSERTを使うため不要だが、他の処理（saveToC等）のために残す
$em->exec('SET CONSTRAINTS ALL DEFERRED;');
}

if ($platform != 'mysql') {
try {
switch ($context) {
case "Customer":
Expand Down
Binary file added Tests/Fixtures/member_test.tar.gz
View file
Open in desktop
Binary file not shown.
3 changes: 0 additions & 3 deletions Tests/Fixtures/member_test/dtb_member.csv
View file
Open in desktop

This file was deleted.

3 changes: 0 additions & 3 deletions Tests/Fixtures/member_test/mtb_authority.csv
View file
Open in desktop

This file was deleted.

Loading