Merge branch 'release/v3.1.1'

.gitignore

@@ -6,7 +6,8 @@ Gemfile.lock
 pkg/*
 spec/internal/config/test.sphinx.conf
 spec/internal/db/sphinx
+spec/internal/log/*.log
 !spec/internal/tmp/.gitkeep
 spec/internal/tmp/*
-!spec/internal/tmp/.gitkeep
+tmp
 _site

.travis.yml

@@ -6,19 +6,16 @@ rvm:
   - jruby-19mode
 before_install:
   - gem update --system
-  - sudo apt-get install postgresql-server-dev-9.1 libmysql++-dev -y
-  - curl -O http://fs-packages.s3.amazonaws.com/fs-sphinx-2.0.9_x86_64_12.04.deb
-  - sudo dpkg -i fs-sphinx-2.0.9_x86_64_12.04.deb
-  - curl -O http://fs-packages.s3.amazonaws.com/fs-sphinx-2.1.4_x86_64_12.04.deb
-  - sudo dpkg -i fs-sphinx-2.1.4_x86_64_12.04.deb
 before_script:
   - "mysql -e 'create database thinking_sphinx;' > /dev/null"
   - "psql -c 'create database thinking_sphinx;' -U postgres >/dev/null"
 env:
   - DATABASE=mysql2 SPHINX_BIN=/usr/local/sphinx-2.0.9/bin/ SPHINX_VERSION=2.0.9
   - DATABASE=postgresql SPHINX_BIN=/usr/local/sphinx-2.0.9/bin/ SPHINX_VERSION=2.0.9
-  - DATABASE=mysql2 SPHINX_BIN=/usr/local/sphinx-2.1.4/bin/ SPHINX_VERSION=2.1.4
-  - DATABASE=postgresql SPHINX_BIN=/usr/local/sphinx-2.1.4/bin/ SPHINX_VERSION=2.1.4
+  - DATABASE=mysql2 SPHINX_BIN=/usr/local/sphinx-2.1.3/bin/ SPHINX_VERSION=2.1.3
+  - DATABASE=postgresql SPHINX_BIN=/usr/local/sphinx-2.1.3/bin/ SPHINX_VERSION=2.1.3
+  - DATABASE=mysql2 SPHINX_BIN=/usr/local/sphinx-2.2.1/bin/ SPHINX_VERSION=2.2.1
+  - DATABASE=postgresql SPHINX_BIN=/usr/local/sphinx-2.2.1/bin/ SPHINX_VERSION=2.2.1
 gemfile:
   - gemfiles/rails_3_2.gemfile
   - gemfiles/rails_4_0.gemfile

HISTORY

@@ -1,3 +1,30 @@
+2014-04-22: 3.1.1
+* [CHANGE] Include full statements when query execution errors are raised (uglier, but more useful when debugging).
+* [FEATURE] Allow for common section in generated Sphinx configuration files for Sphinx 2.2.x (disabled by default, though) (Trevor Smith).
+* [FEATURE] Basic support for HABTM associations and MVAs with query/ranged-query sources.
+* [CHANGE] Connection error messages now mention Sphinx, instead of just MySQL.
+* [FIX] Don't apply attribute-only updates to real-time indices.
+* [FIX] Don't instantiate blank strings (via inheritance type columns) as constants.
+* [FIX] Don't presume all indices for a model have delta pairs, even if one does.
+* [CHANGE] Raise an exception when a referenced column does not exist.
+* [CHANGE] Capistrano tasks use thinking_sphinx_rails_env (defaults to standard environment) (Robert Coleman).
+* [FIX] Always use connection options for connection information.
+* [FIX] respond_to? works reliably with masks (Konstantin Burnaev).
+* [FEATURE] Real-time indices callbacks can be disabled (useful for unit tests).
+* [FEATURE] ThinkingSphinx::Test has a clear method and no-index option for starting for real-time setups.
+* [FIX] Avoid null values in MVA query/ranged-query sources.
+* [CHANGE] Alias group and count columns for easier referencing in other clauses.
+* [FEATURE] Allow disabling of distributed indices.
+* [FIX] Don't send unicode null characters to real-time Sphinx indices.
+* [FIX] :populate option is now respected for single-model searches.
+* [FIX] :thinking_sphinx_roles is now used consistently in Capistrano v3 tasks.
+* [CHANGE] Log real-time index updates (Demian Ferreiro).
+* [FIX] Only expand log directory if it exists.
+* [FIX] Handle JDBC connection errors appropriately (Adam Hutchison).
+* [FIX] Fixing wildcarding of Unicode strings.
+* [CHANGE] All indices now respond to a public attributes method.
+* [FIX] Improved handling of association searches with real-time indices, including via has_many :though associations (Rob Anderton).
+
 2014-01-11: 3.1.0
 * [CHANGE] Updating Riddle requirement to >= 1.5.10.
 * [CHANGE] Extracting join generation into its own gem: Joiner.

README.textile

@@ -1,11 +1,12 @@
 h1. Thinking Sphinx
 
-Thinking Sphinx is a library for connecting ActiveRecord to the Sphinx full-text search tool, and integrates closely with Rails (but also works with other Ruby web frameworks). The current release is v3.1.0.
+Thinking Sphinx is a library for connecting ActiveRecord to the Sphinx full-text search tool, and integrates closely with Rails (but also works with other Ruby web frameworks). The current release is v3.1.1.
 
 h2. Upgrading
 
 Please refer to the release notes for any changes you need to make when upgrading:
 
+* "v3.1.1":https://github.com/pat/thinking-sphinx/releases/tag/v3.1.1
 * "v3.1.0":https://github.com/pat/thinking-sphinx/releases/tag/v3.1.0
 * "v3.0.6":https://github.com/pat/thinking-sphinx/releases/tag/v3.0.6
 
@@ -15,9 +16,9 @@ h2. Installation
 
 It's a gem, so install it like you would any other gem. You will also need to specify the mysql2 gem if you're using MRI, or jdbc-mysql if you're using JRuby:
 
-<pre><code>gem 'mysql2',          '0.3.13', :platform => :ruby
-gem 'jdbc-mysql',      '5.1.28', :platform => :jruby
-gem 'thinking-sphinx', '3.1.0'</code></pre>
+<pre><code>gem 'mysql2',          '~> 0.3.13', :platform => :ruby
+gem 'jdbc-mysql',      '~> 5.1.28', :platform => :jruby
+gem 'thinking-sphinx', '~> 3.1.1'</code></pre>
 
 The MySQL gems mentioned are required for connecting to Sphinx, so please include it even when you're using PostgreSQL for your database.
 
@@ -31,7 +32,7 @@ h3. Extending with Middleware, Glazes and Panes
 
 These are covered in "a blog post":http://freelancing-gods.com/posts/rewriting_thinking_sphinx_middleware_glazes_and_panes.
 
-h2. Requiements
+h2. Requirements
 
 h3. Sphinx
 
@@ -45,7 +46,7 @@ If you want ActiveRecord 3.1 support, then refer to the 3.0.x releases of Thinki
 
 h3. Ruby
 
-You'll need either the standard Ruby (v1.9.3 or newer) or JRuby. I'm open to patches to improve Rubinius support (if required - it may work with it right now).
+You'll need either the standard Ruby (v1.9.3 or newer) or JRuby (1.7.9 or newer). I'm open to patches to improve Rubinius support (if required - it may work with it right now).
 
 JRuby is only supported as of Thinking Sphinx v3.1.0, and requires Sphinx 2.1.2 or newer.
 
@@ -68,7 +69,7 @@ psql
 # In that console, create the database:
 CREATE DATABASE thinking_sphinx;</code></pre>
 
-You can then run the unit tests with @rake spec:unit@, the acceptance tests with @rake spec:acceptance@, or all of the tests with just @rake@. To run these with PostgreSQL, you'll need to set the `DATABASE` environment variable accordingly:
+You can then run the unit tests with @rake spec:unit@, the acceptance tests with @rake spec:acceptance@, or all of the tests with just @rake@. To run these with PostgreSQL, you'll need to set the @DATABASE@ environment variable accordingly:
 
 <pre><code>DATABASE=postgresql rake</code></pre>
 

gemfiles/rails_3_2.gemfile

@@ -4,9 +4,8 @@ source "https://rubygems.org"
 
 gem "mysql2", "~> 0.3.12b4", :platform=>:ruby
 gem "pg", "~> 0.16.0", :platform=>:ruby
-gem "activerecord-jdbcmysql-adapter", "~> 1.1.3", :platform=>:jruby
-gem "activerecord-jdbcpostgresql-adapter", "~> 1.1.3", :platform=>:jruby
-gem "riddle", :git=>"git://github.com/pat/riddle.git", :branch=>"develop"
+gem "activerecord-jdbcmysql-adapter", "~> 1.3.4", :platform=>:jruby
+gem "activerecord-jdbcpostgresql-adapter", "~> 1.3.4", :platform=>:jruby
 gem "rails", "~> 3.2.0"
 
 gemspec :path=>"../"

gemfiles/rails_4_0.gemfile

@@ -4,9 +4,8 @@ source "https://rubygems.org"
 
 gem "mysql2", "~> 0.3.12b4", :platform=>:ruby
 gem "pg", "~> 0.16.0", :platform=>:ruby
-gem "activerecord-jdbcmysql-adapter", "~> 1.1.3", :platform=>:jruby
-gem "activerecord-jdbcpostgresql-adapter", "~> 1.1.3", :platform=>:jruby
-gem "riddle", :git=>"git://github.com/pat/riddle.git", :branch=>"develop"
+gem "activerecord-jdbcmysql-adapter", "~> 1.3.4", :platform=>:jruby
+gem "activerecord-jdbcpostgresql-adapter", "~> 1.3.4", :platform=>:jruby
 gem "rails", "~> 4.0.2"
 
 gemspec :path=>"../"

gemfiles/rails_4_1.gemfile

@@ -4,9 +4,8 @@ source "https://rubygems.org"
 
 gem "mysql2", "~> 0.3.12b4", :platform=>:ruby
 gem "pg", "~> 0.16.0", :platform=>:ruby
-gem "activerecord-jdbcmysql-adapter", "~> 1.1.3", :platform=>:jruby
-gem "activerecord-jdbcpostgresql-adapter", "~> 1.1.3", :platform=>:jruby
-gem "riddle", :git=>"git://github.com/pat/riddle.git", :branch=>"develop"
+gem "activerecord-jdbcmysql-adapter", "~> 1.3.4", :platform=>:jruby
+gem "activerecord-jdbcpostgresql-adapter", "~> 1.3.4", :platform=>:jruby
 gem "rails", "~> 4.1.0.beta1"
 
 gemspec :path=>"../"

lib/thinking_sphinx.rb

@@ -73,6 +73,7 @@ module Subscribers; end
 require 'thinking_sphinx/active_record'
 require 'thinking_sphinx/deltas'
 require 'thinking_sphinx/distributed'
+require 'thinking_sphinx/logger'
 require 'thinking_sphinx/real_time'
 
 require 'thinking_sphinx/railtie' if defined?(Rails)

lib/thinking_sphinx/active_record.rb

@@ -21,6 +21,7 @@ module Callbacks; end
 require 'thinking_sphinx/active_record/polymorpher'
 require 'thinking_sphinx/active_record/property_query'
 require 'thinking_sphinx/active_record/property_sql_presenter'
+require 'thinking_sphinx/active_record/simple_many_query'
 require 'thinking_sphinx/active_record/sql_builder'
 require 'thinking_sphinx/active_record/sql_source'
 

lib/thinking_sphinx/active_record/association_proxy.rb

@@ -30,3 +30,4 @@ def association_filter
 end
 
 require 'thinking_sphinx/active_record/association_proxy/attribute_finder'
+require 'thinking_sphinx/active_record/association_proxy/attribute_matcher'

lib/thinking_sphinx/active_record/association_proxy/attribute_finder.rb

@@ -5,17 +5,16 @@ def initialize(association)
 
   def attribute
     attributes.detect { |attribute|
-      # Don't bother with attributes built from multiple columns
-      next if attribute.columns.many?
-
-      attribute.columns.first.__name == foreign_key.to_sym ||
-      attribute.name == foreign_key.to_s
+      ThinkingSphinx::ActiveRecord::AssociationProxy::AttributeMatcher.new(
+        attribute, foreign_key
+      ).matches?
     } or raise "Missing Attribute for Foreign Key #{foreign_key}"
   end
 
   private
+
   def attributes
-    sources.collect(&:attributes).flatten
+    indices.collect(&:attributes).flatten
   end
 
   def configuration
@@ -40,8 +39,4 @@ def reflection_target
     target = target.through_reflection if target.through_reflection
     target
   end
-
-  def sources
-    indices.collect(&:sources).flatten
-  end
 end

lib/thinking_sphinx/active_record/association_proxy/attribute_matcher.rb

@@ -0,0 +1,38 @@
+class ThinkingSphinx::ActiveRecord::AssociationProxy::AttributeMatcher
+    def initialize(attribute, foreign_key)
+      @attribute, @foreign_key = attribute, foreign_key.to_s
+    end
+
+    def matches?
+      return false if many?
+
+      column_name_matches? || attribute_name_matches? || multi_singular_match?
+    end
+
+    private
+
+    attr_reader :attribute, :foreign_key
+
+    delegate :name, :multi?, :to => :attribute
+
+    def attribute_name_matches?
+      name == foreign_key
+    end
+
+    def column_name_matches?
+      column.__name.to_s == foreign_key
+    end
+
+    def column
+      attribute.respond_to?(:columns) ? attribute.columns.first :
+        attribute.column
+    end
+
+    def many?
+      attribute.respond_to?(:columns) && attribute.columns.many?
+    end
+
+    def multi_singular_match?
+      multi? && name.singularize == foreign_key
+    end
+  end

lib/thinking_sphinx/active_record/attribute/type.rb

@@ -40,6 +40,20 @@ def associations
     end
   end
 
+  def big_integer?
+    database_column.type == :integer && database_column.sql_type[/bigint/i]
+  end
+
+  def column_name
+    attribute.columns.first.__name.to_s
+  end
+
+  def database_column
+    @database_column ||= klass.columns.detect { |db_column|
+      db_column.name == column_name
+    }
+  end
+
   def klass
     @klass ||= associations.any? ? associations.last.klass : model
   end
@@ -57,23 +71,20 @@ def single_column_reference?
   end
 
   def type_from_database
-    db_column = klass.columns.detect { |db_column|
-      db_column.name == attribute.columns.first.__name.to_s
-    }
+    raise ThinkingSphinx::MissingColumnError,
+      "column #{column_name} does not exist" if database_column.nil?
 
-    if db_column.type == :integer && db_column.sql_type[/bigint/i]
-      return :bigint
-    end
+    return :bigint if big_integer?
 
-    case db_column.type
+    case database_column.type
     when :datetime, :date
       :timestamp
     when :text
       :string
     when :decimal
       :float
     else
-      db_column.type
+      database_column.type
     end
   end
 end

lib/thinking_sphinx/active_record/base.rb

@@ -50,7 +50,9 @@ def merge_search(search, query, options)
           'You cannot search with Sphinx through ActiveRecord scopes'
       end
 
-      merger.merge! nil, :classes => [self]
+      result = merger.merge! nil, :classes => [self]
+      result.populate if result.options[:populate]
+      result
     end
   end
 end

lib/thinking_sphinx/active_record/callbacks/delta_callbacks.rb

@@ -30,7 +30,7 @@ def config
   end
 
   def core_indices
-    @core_indices ||= indices.reject &:delta?
+    @core_indices ||= indices.select(&:delta_processor).reject(&:delta?)
   end
 
   def delta_indices

lib/thinking_sphinx/active_record/callbacks/update_callbacks.rb

@@ -28,7 +28,10 @@ def configuration
   end
 
   def indices
-    @indices ||= configuration.indices_for_references reference
+    @indices ||= begin
+      all = configuration.indices_for_references(reference)
+      all.reject { |index| index.type == 'rt' }
+    end
   end
 
   def reference

lib/thinking_sphinx/active_record/index.rb

@@ -12,6 +12,10 @@ def append_source
     end
   end
 
+  def attributes
+    sources.collect(&:attributes).flatten
+  end
+
   def delta?
     @options[:delta?]
   end
@@ -40,10 +44,6 @@ def adapter
       adapter_for(model)
   end
 
-  def attributes
-    sources.collect(&:attributes).flatten
-  end
-
   def fields
     sources.collect(&:fields).flatten
   end