extend_queries with support_unencrypted_data breaks when combined with normalizes #56684
Description
Steps to reproduce
# frozen_string_literal: true
require "bundler/inline"
gemfile(true) do
source "https://rubygems.org"
gem "rails", "~> 8.0.0"
gem "sqlite3", "~> 2.0"
end
require "active_record/railtie"
require "minitest/autorun"
ENV["DATABASE_URL"] = "sqlite3::memory:"
class TestApp < Rails::Application
config.load_defaults Rails::VERSION::STRING.to_f
config.eager_load = false
config.logger = Logger.new($stdout)
config.secret_key_base = "secret"
config.active_record.encryption.primary_key = "test" * 8
config.active_record.encryption.deterministic_key = "test" * 8
config.active_record.encryption.key_derivation_salt = "test" * 8
config.active_record.encryption.support_unencrypted_data = true
config.active_record.encryption.extend_queries = true
end
Rails.application.initialize!
ActiveRecord::Schema.define do
create_table :users, force: true do |t|
t.text :email
end
end
class User < ActiveRecord::Base
encrypts :email, deterministic: true
normalizes :email, with: ->(email) { email.downcase.strip }
end
class BugTest < Minitest::Test
def test_query_should_include_plaintext
sql = User.where(email: "test@example.com").to_sql
assert sql.include?("test@example.com"),
"Expected plaintext in query, got: #{sql}"
end
def test_should_find_plaintext_record
# Insert plaintext directly (simulating pre-encryption data)
ActiveRecord::Base.connection.execute(
"INSERT INTO users (email) VALUES ('test@example.com')"
)
user = User.find_by(email: "test@example.com")
refute_nil user, "Should find plaintext record"
end
endExpected behavior
When using encrypts with support_unencrypted_data: true and extend_queries: true, queries should include both the encrypted value AND the plaintext value, allowing records with unencrypted data to be found.
From the Rails Guide:
"With
support_unencrypted_data, the system will also include the clean version of the content in the list of values used to query the database."
Expected SQL:
WHERE email IN ('{"p":"encrypted..."}', 'test@example.com')This works correctly when using encrypts alone (without normalizes) or when using downcase: true.
Actual behavior
When normalizes is combined with encrypts on the same attribute, the plaintext value is not included in the query. Instead, the Ruby object inspection string of an internal AdditionalValue object gets encrypted:
Actual SQL:
WHERE email IN ('{"p":"encrypted..."}', '{"p":"GARBAGE..."}')The second encrypted value is the encrypted form of:
"#<ActiveRecord::Encryption::ExtendedDeterministicQueries::AdditionalValue:0x...>"
Root cause: NormalizedValueType#serialize calls normalize(value) on the AdditionalValue object. The normalization lambda (e.g., email.downcase.strip) triggers to_s on the object, producing the garbage string which then gets encrypted.
System configuration
Rails version: 8.0.0 (affects Rails 7.1+ where normalizes was introduced)
Ruby version: 3.4.8
Possible fix
Two changes in activerecord/lib/active_record/encryption/extended_deterministic_queries.rb:
1. Prepend ExtendedEncryptableType to NormalizedValueType in install_support:
def self.install_support
ActiveRecord::Relation.prepend(RelationQueries)
ActiveRecord::Base.include(CoreQueries)
ActiveRecord::Encryption::EncryptedAttributeType.prepend(ExtendedEncryptableType)
ActiveModel::Attributes::Normalization::NormalizedValueType.prepend(ExtendedEncryptableType)
end2. Use type.cast(value) in additional_values_for to normalize the value:
def additional_values_for(value, type)
type.previous_types.collect do |additional_type|
AdditionalValue.new(type.cast(value), additional_type)
end
endUsing type.cast(value) lets the type chain handle normalization automatically, regardless of declaration order.
Note: This fix prepends to an ActiveModel class (NormalizedValueType) from ActiveRecord, which crosses module boundaries. There may be a cleaner approach that keeps the fix entirely within ActiveRecord or ActiveModel.