Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
98169fd
Move tests into adapter folders
andrewmarkle Oct 29, 2025
ea0e14d
Add mysql and a devcontainer to easily run it
andrewmarkle Oct 29, 2025
8466298
Add mysql adapter with tests
andrewmarkle Oct 29, 2025
d975bd4
Simplify devcontainer setup
andrewmarkle Oct 29, 2025
bbd4e53
Clean up the mysql adapter
andrewmarkle Oct 29, 2025
4445329
Refactor
andrewmarkle Oct 30, 2025
1b87d51
Update integration script
andrewmarkle Oct 30, 2025
1c6eda2
Add adapter folder and get sqlite integration tests to pass
andrewmarkle Oct 30, 2025
94e6d86
Add chrome to devcontainer so that we can run integration tests in it
andrewmarkle Oct 30, 2025
132e9c6
A couple fixes. Switch to mysql
andrewmarkle Oct 30, 2025
9da4474
Fix database tasks
andrewmarkle Nov 3, 2025
ef0819d
Fix tests for sqlite
andrewmarkle Nov 3, 2025
5929062
Make fake database creation work for all adapters
andrewmarkle Nov 3, 2025
e980a15
shorten the db name so it's shorter than the mysql max (64)
andrewmarkle Nov 3, 2025
6c9121b
Working(ish) integration tests with mysql
andrewmarkle Nov 3, 2025
1b147f0
Specify RAILS_ENV
andrewmarkle Nov 4, 2025
ce9f288
Refactor and try to make test_workerize work
andrewmarkle Nov 4, 2025
7b28b4b
Isolate the dbs for integration tests
andrewmarkle Nov 4, 2025
f69a5c1
Make sure dbs start from a clean state everytime
andrewmarkle Nov 4, 2025
bda859a
Enable integration tests for mysql and sqlite
andrewmarkle Nov 4, 2025
7152457
Add mysql schema_cache and make sure the unit tests use it
andrewmarkle Nov 5, 2025
d47aa8c
Refactor cleaning up databases in unit tests
andrewmarkle Nov 5, 2025
8418c24
Rename methods
andrewmarkle Nov 5, 2025
31fe774
Simplify SystemTest setup
andrewmarkle Nov 5, 2025
e004cd9
Revert un-needed change
andrewmarkle Nov 5, 2025
b22fa30
More cleanup
andrewmarkle Nov 5, 2025
6f46023
Simplify
andrewmarkle Nov 5, 2025
798c40d
Add test to ensure shared dbs don't get passed to adapter
andrewmarkle Nov 5, 2025
84fd830
Add tests to ensure we get a valid mysql database name
andrewmarkle Nov 5, 2025
bc76b0e
Add tests for when a host option is provided and more cleanup
andrewmarkle Nov 5, 2025
1027b04
Move comment
andrewmarkle Nov 5, 2025
8ce81ff
Refactor test-integration setup for mysql
andrewmarkle Nov 6, 2025
bb2b311
Simplify migration logic
andrewmarkle Nov 6, 2025
6431651
Improve the test coverage of database_tasks
andrewmarkle Nov 6, 2025
554f378
Refactor and clean up
andrewmarkle Nov 6, 2025
c9c421a
Run both types of adapters in the integration tests
andrewmarkle Nov 6, 2025
c47579c
Reject the connection if the tenant is an empty string
andrewmarkle Nov 6, 2025
18ca09c
Configure CI to add mysql
andrewmarkle Nov 6, 2025
cb16d87
Align testing adapters more closely and add more test scenarios to mysql
andrewmarkle Nov 6, 2025
2e8b21c
Fix test and ensure temp connection is more obviously set (and exclude
andrewmarkle Nov 7, 2025
3c055d5
Ensure unit test only runs for sqlite
andrewmarkle Nov 7, 2025
eabf8c8
Fix mismatch in system tests for action cable setup
andrewmarkle Nov 7, 2025
ed0539c
Try explicitly busting the cache
andrewmarkle Nov 7, 2025
76c2acf
Updo the cache busting and add some debug info
andrewmarkle Nov 7, 2025
41e1f23
Try this
andrewmarkle Nov 7, 2025
745df5d
Try to fix the debug methods
andrewmarkle Nov 7, 2025
d238e6b
Cleanup fix for test
andrewmarkle Nov 7, 2025
d8d75c9
Refactor temp_connection name
andrewmarkle Nov 14, 2025
1f8804b
Remove dev containers to prefer a bin/setup script
andrewmarkle Nov 21, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,20 +28,52 @@ jobs:

unit:
runs-on: ubuntu-latest
services:
mysql:
image: mysql:8.0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it worth setting up a matrix to test against 8 and 9? And maybe even patch releases like 8.4?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could! I'm not that worried that we're doing anything too novel that would break b/w versions of mysql though. Probably testing against the latest version is good enough 🤷‍♂️. But I'm very open to doing that if you think it's a good idea.

We're really only doing "SHOW DATABASES LIKE '#{database_path}' and DROP DATABASE IF EXISTS which will work across all versions. The rest of the stuff we're just calling the built-in Rails methods which are already well tested. Just not too worried about it I guess!

env:
MYSQL_ROOT_PASSWORD: devcontainer
MYSQL_DATABASE: test
ports:
- 3306:3306
options: >-
--health-cmd="mysqladmin ping --silent"
--health-interval=10s
--health-timeout=5s
--health-retries=3
steps:
- uses: actions/checkout@v5
- uses: ruby/setup-ruby@d5126b9b3579e429dd52e51e68624dda2e05be25 # v1.267.0
with:
ruby-version: "3.4"
bundler-cache: true
- run: bin/test-unit
env:
MYSQL_HOST: 127.0.0.1
MYSQL_PORT: "3306"

integration:
runs-on: ubuntu-latest
services:
mysql:
image: mysql:8.0
env:
MYSQL_ROOT_PASSWORD: devcontainer
MYSQL_DATABASE: test
ports:
- 3306:3306
options: >-
--health-cmd="mysqladmin ping --silent"
--health-interval=10s
--health-timeout=5s
--health-retries=3
steps:
- uses: actions/checkout@v5
- uses: ruby/setup-ruby@d5126b9b3579e429dd52e51e68624dda2e05be25 # v1.267.0
with:
ruby-version: "3.4"
bundler-cache: true
- run: bin/test-integration
env:
MYSQL_HOST: 127.0.0.1
MYSQL_PORT: "3306"
1 change: 1 addition & 0 deletions Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ group :development, :test do
gem "sqlite3", "2.7.4"
gem "debug", "1.11.0"
gem "minitest-parallel_fork", "2.1.0", require: false
gem "mysql2", "~> 0.5", require: false
end

group :rubocop do
Expand Down
3 changes: 3 additions & 0 deletions Gemfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,8 @@ GEM
minitest (5.26.0)
minitest-parallel_fork (2.1.0)
minitest (>= 5.15.0)
mysql2 (0.5.7)
bigdecimal
net-imap (0.5.12)
date
net-protocol
Expand Down Expand Up @@ -349,6 +351,7 @@ DEPENDENCIES
importmap-rails
jbuilder
minitest-parallel_fork (= 2.1.0)
mysql2 (~> 0.5)
propshaft
puma (>= 5.0)
rails!
Expand Down
63 changes: 63 additions & 0 deletions bin/setup
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
#!/usr/bin/env bash
set -e

CONTAINER_NAME="${CONTAINER_NAME:-activerecord-tenanted-mysql}"
MYSQL_IMAGE="mysql:latest"
MYSQL_ROOT_PASSWORD="devcontainer"
MYSQL_PORT="${MYSQL_PORT:-3307}"

echo "==> Checking Docker installation..."
if ! command -v docker &> /dev/null; then
echo "ERROR: Docker is not installed. Please install Docker and try again."
exit 1
fi

if ! docker info &> /dev/null; then
echo "ERROR: Docker daemon is not running. Please start Docker and try again."
exit 1
fi

echo "==> Setting up MySQL container..."
if docker ps -a --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
echo " Stopping and removing existing container..."
docker stop "$CONTAINER_NAME" > /dev/null 2>&1 || true
docker rm "$CONTAINER_NAME" > /dev/null 2>&1 || true
fi

echo "==> Creating and starting MySQL container..."
docker run -d \
--name="$CONTAINER_NAME" \
--restart unless-stopped \
-p "127.0.0.1:${MYSQL_PORT}:3306" \
-e MYSQL_ROOT_PASSWORD="$MYSQL_ROOT_PASSWORD" \
"$MYSQL_IMAGE"

echo "==> Waiting for MySQL to be ready..."
MAX_TRIES=30
TRIES=0
while [ $TRIES -lt $MAX_TRIES ]; do
if docker exec "$CONTAINER_NAME" mysqladmin ping -h localhost --silent &> /dev/null; then
echo " MySQL is ready!"
break
fi
TRIES=$((TRIES + 1))
if [ $TRIES -eq $MAX_TRIES ]; then
echo "ERROR: MySQL failed to start within expected time"
exit 1
fi
echo -n "."
sleep 2
done

echo "==> Verifying database connection..."
docker exec "$CONTAINER_NAME" mysql -uroot -p"$MYSQL_ROOT_PASSWORD" -e "SELECT 1;" > /dev/null 2>&1
echo " Database ready!"

echo "==> Running bundle install..."
bundle check > /dev/null 2>&1 || bundle install

echo ""
echo "==> Setup complete!"
echo ""
echo "MySQL is running on 127.0.0.1:${MYSQL_PORT}"
echo ""
41 changes: 25 additions & 16 deletions bin/test-integration
Original file line number Diff line number Diff line change
Expand Up @@ -17,12 +17,14 @@ require "fileutils"
require "tmpdir"
require "yaml"
require "open3"
require "erb"
require "debug"
require "active_support"
require "active_support/core_ext/object" # deep_dup

scenarios = Dir.glob("test/scenarios/*db/*record.rb").map do |path|
database, models = path.scan(%r{test/scenarios/(.+db)/(.+record).rb}).flatten
{ database: database, models: models }
scenarios = Dir.glob("test/scenarios/*/*db/*record.rb").map do |path|
adapter, database, models = path.scan(%r{test/scenarios/(.+)/(.+db)/(.+record).rb}).flatten
{ adapter: adapter, database: database, models: models }
end

COLOR_FG_BLUE = "\e[0;34m"
Expand All @@ -49,28 +51,34 @@ def run_scenario(scenario)
at_exit { FileUtils.remove_entry app_path } unless OPT_KEEP_DIR
puts "Creating integration app for #{COLOR_FG_BLUE}#{scenario}#{COLOR_RESET} at #{app_path}"

# Generate a unique database prefix for this scenario to avoid conflicts
db_prefix = "test_#{scenario[:database]}_#{scenario[:models]}_#{Process.pid}".gsub(/[^a-zA-Z0-9_]/, "_")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious what conflicts you're avoiding with this that DatabaseConfiguration#test_worker_id= isn't sufficient to prevent. Can you explain a bit more?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was having a bunch of contention issues before this when running integration tests in parallel.

In SQLite each test run uses a unique temporary directory which means that multiple test runs never touched the same database file.

But with MySQL it's different b/c we have the same shared server. What this means is that all parallel test runs originally used the same database name which caused collisions and contention. Different tests were trying to drop, create, or migrate using the same schema.

There are different ways we could set this up but I just went with an env variable that gets evaluated at runtime (<%= ENV.fetch('MYSQL_DB_PREFIX', 'primary') %>) from the database.yml. This happens when we db:drop, db:prepare, etc. and just ensures that each test database is unique from any others.

I actually didn't realize you could run the unit tests in parallel and I didn't implement something similar for them. But the problem you're seeing running the unit tests is the same problem that this was supposed to address.


# make a copy of the smarty app
FileUtils.copy_entry(SMARTY_PATH, app_path)

# generate database file
database_file = File.join(GEM_PATH, "test/scenarios/#{scenario[:database]}/database.yml")
database_file_contents = sprintf(File.read(database_file), storage: "storage", db_path: "db")
database_file_hash = YAML.load(database_file_contents)
database_file = File.join(GEM_PATH, "test/scenarios/#{scenario[:adapter]}/#{scenario[:database]}/database.yml")
erb_content = ERB.new(File.read(database_file)).result
storage_path = File.join(app_path, "storage")
db_path = File.join(app_path, "db")
database_file_contents = sprintf(erb_content, storage: storage_path, db_path: db_path)
database_file_hash = YAML.unsafe_load(database_file_contents)
database_file_hash["development"] = database_file_hash["test"].deep_dup
database_file_hash["development"].each_value do |hash|
hash["database"] = hash["database"].sub("/test/", "/development/")
end
File.write(File.join(app_path, "config/database.yml"), database_file_hash.to_yaml)

# generate models using ApplicationRecord
models_file = File.join(GEM_PATH, "test/scenarios/#{scenario[:database]}/#{scenario[:models]}.rb")
models_file = File.join(GEM_PATH, "test/scenarios/#{scenario[:adapter]}/#{scenario[:database]}/#{scenario[:models]}.rb")
models_file_contents = File.read(models_file)
.gsub("TenantedApplicationRecord", "ApplicationRecord")
File.write(File.join(app_path, "app/models/application_record.rb"), models_file_contents)

# copy migrations from scenario and smarty app
FileUtils.mkdir_p(File.join(app_path, "db"))
FileUtils.cp_r(Dir.glob(File.join(GEM_PATH, "test/scenarios/#{scenario[:database]}/*migrations")),
FileUtils.cp_r(Dir.glob(File.join(GEM_PATH, "test/scenarios/#{scenario[:adapter]}/#{scenario[:database]}/*migrations")),
File.join(app_path, "db"))
FileUtils.cp_r(Dir.glob(File.join(app_path, "db/migrate/*rb")),
File.join(app_path, "db/tenanted_migrations"))
Expand All @@ -82,20 +90,21 @@ def run_scenario(scenario)
Bundler.with_original_env do
run_cmd(scenario, "bundle check || bundle install")

# set up the database and schema files
run_cmd(scenario, "bin/rails db:prepare")
adapter_config = {"RAILS_ENV" => "test","ARTENANT_SCHEMA_DUMP" => "1"}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On main, the integration test suite runs rails db:prepare in the development environment, and then the tests get run in the test environment (implicitly setting up the databases).

Can you say a bit more about this change?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added this in b/c I kept getting this error when trying to run any database tasks.

bin/rails db:drop (1.89s)
bin/rails aborted!
ActiveRecord::EnvironmentMismatchError: You are attempting to modify a database that was last run in `test` environment. (ActiveRecord::EnvironmentMismatchError)
You are running in `development` environment. If you are sure you want to continue, first set the environment using:

        bin/rails db:environment:set RAILS_ENV=development


Tasks: TOP => db:drop => db:check_protected_environments
(See full trace by running task with --trace)
ERROR: Command failed

But I could easily be doing something wrong!

if scenario[:adapter] == "mysql"
adapter_config.merge!({ "MYSQL_UNIQUE_PREFIX" => db_prefix })
end

# Drop existing databases to ensure fresh setup
run_cmd(scenario, "bin/rails db:drop", env: { **adapter_config }) rescue nil
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe only run this command when testing non-sqlite databases?


run_cmd(scenario, "bin/rails db:prepare", env: { **adapter_config })

# validate-ish the setup
prefix = scenario[:database].match?(/\Asecondary/) ? "tenanted_" : ""
File.exist?("db/#{prefix}schema.rb") || abort("Schema dump not generated")
File.exist?("db/#{prefix}schema_cache.yml") || abort("Schema cache dump not generated")

# create a fake tenant database to validate that it is deleted in the test suite
dev_db = Dir.glob("storage/development/*/development-tenant/main.sqlite3").first
test_db = dev_db.gsub("/development/", "/test/").gsub("development-tenant", "delete-me")
FileUtils.mkdir_p(File.dirname(test_db))
FileUtils.touch(test_db)

Comment on lines -93 to -98
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this being removed? This test database needs to be here to make this test meaningful:

https://github.com/basecamp/activerecord-tenanted/blob/main/test/integration/test/testcase_test.rb#L12-L15

And I think the mysql implementation also needs to delete existing tenant databases at the start of the test suite, though you may need to change this setup to something that will work for both databases, maybe:

run_cmd(scenario, "bin/rails db:migrate", env: { ARTENANT: "delete-me" })

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I may have misunderstood what this does. I just added in a db:drop at the start of the test run which I thought was a bit simpler / handles all database types. But the test you mentioned would be redundant then: https://github.com/basecamp/activerecord-tenanted/blob/main/test/integration/test/testcase_test.rb#L12-L15.

We can keep this in if you want! But we still need to db:drop with mysql just to ensure that we're starting from a clean slate every time.

# run in parallel half the time
env = if rand(2).zero?
{ "PARALLEL_WORKERS" => "2" }
Expand Down
1 change: 1 addition & 0 deletions lib/active_record/tenanted.rb
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
loader = Zeitwerk::Loader.for_gem_extension(ActiveRecord)
loader.inflector.inflect(
"sqlite" => "SQLite",
"mysql" => "MySQL",
)
loader.setup

Expand Down
5 changes: 3 additions & 2 deletions lib/active_record/tenanted/cable_connection.rb
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,10 @@ def connect

private
def set_current_tenant
return unless tenant = tenant_resolver.call(request)
tenant = tenant_resolver.call(request)
return if tenant.nil?

if connection_class.tenant_exist?(tenant)
if tenant.present? && connection_class.tenant_exist?(tenant)
Comment on lines -22 to +25
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting! Can you say a bit more about the distinction between nil and blank tenant names here in your application? Under what conditions does your tenant resolver return each of those?

More specifically, why couldn't the method be rewritten like this to handle untenanted connections?

          def set_current_tenant
            tenant = tenant_resolver.call(request)

            if tenant.present?
              if connection_class.tenant_exist?(tenant)
                self.current_tenant = tenant
              else
                reject_unauthorized_connection
              end
            end
          end

self.current_tenant = tenant
else
reject_unauthorized_connection
Expand Down
2 changes: 2 additions & 0 deletions lib/active_record/tenanted/database_adapter.rb
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ def new(db_config)
end

register "sqlite3", "ActiveRecord::Tenanted::DatabaseAdapters::SQLite"
register "trilogy", "ActiveRecord::Tenanted::DatabaseAdapters::MySQL"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eventually I'd like to run the mysql tests against trilogy. Doesn't need to be in this PR, though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree! Right now we're using mysql2 in our app but we should be switching over to trilogy this year / early next. So I'll also be able to test it out IRL too.

register "mysql2", "ActiveRecord::Tenanted::DatabaseAdapters::MySQL"
end
end
end
Loading
Loading