diff --git a/doc/build/changelog/changelog_01.rst b/doc/build/changelog/changelog_01.rst deleted file mode 100644 index 2122c36f2db..00000000000 --- a/doc/build/changelog/changelog_01.rst +++ /dev/null @@ -1,976 +0,0 @@ - -============= -0.1 Changelog -============= - - -.. changelog:: - :version: 0.1.7 - :released: Fri May 05 2006 - - .. change:: - :tags: - :tickets: - - some fixes to topological sort algorithm - - .. change:: - :tags: - :tickets: - - added DISTINCT ON support to Postgres (just supply distinct=[col1,col2..]) - - .. change:: - :tags: - :tickets: - - added __mod__ (% operator) to sql expressions - - .. change:: - :tags: - :tickets: - - "order_by" mapper property inherited from inheriting mapper - - .. change:: - :tags: - :tickets: - - fix to column type used when mapper UPDATES/DELETEs - - .. change:: - :tags: - :tickets: - - with convert_unicode=True, reflection was failing, has been fixed - - .. change:: - :tags: - :tickets: - - types types types! still weren't working....have to use TypeDecorator again :( - - .. change:: - :tags: - :tickets: - - mysql binary type converts array output to buffer, fixes PickleType - - .. change:: - :tags: - :tickets: - - fixed the attributes.py memory leak once and for all - - .. change:: - :tags: - :tickets: - - unittests are qualified based on the databases that support each one - - .. change:: - :tags: - :tickets: - - fixed bug where column defaults would clobber VALUES clause of insert objects - - .. change:: - :tags: - :tickets: - - fixed bug where table def w/ schema name would force engine connection - - .. change:: - :tags: - :tickets: - - fix for parenthesis to work correctly with subqueries in INSERT/UPDATE - - .. change:: - :tags: - :tickets: - - HistoryArraySet gets extend() method - - .. change:: - :tags: - :tickets: - - fixed lazyload support for other comparison operators besides = - - .. change:: - :tags: - :tickets: - - lazyload fix where two comparisons in the join condition point to the - samem column - - .. change:: - :tags: - :tickets: - - added "construct_new" flag to mapper, will use __new__ to create instances - instead of __init__ (standard in 0.2) - - .. change:: - :tags: - :tickets: - - added selectresults.py to SVN, missed it last time - - .. change:: - :tags: - :tickets: - - tweak to allow a many-to-many relationship from a table to itself via - an association table - - .. change:: - :tags: - :tickets: - - small fix to "translate_row" function used by polymorphic example - - .. change:: - :tags: - :tickets: - - create_engine uses cgi.parse_qsl to read query string (out the window in 0.2) - - .. change:: - :tags: - :tickets: - - tweaks to CAST operator - - .. change:: - :tags: - :tickets: - - fixed function names LOCAL_TIME/LOCAL_TIMESTAMP -> LOCALTIME/LOCALTIMESTAMP - - .. change:: - :tags: - :tickets: - - fixed order of ORDER BY/HAVING in compile - -.. changelog:: - :version: 0.1.6 - :released: Wed Apr 12 2006 - - .. change:: - :tags: - :tickets: - - support for MS-SQL added courtesy Rick Morrison, Runar Petursson - - .. change:: - :tags: - :tickets: - - the latest SQLSoup from J. Ellis - - .. change:: - :tags: - :tickets: - - ActiveMapper has preliminary support for inheritance (Jeff Watkins) - - .. change:: - :tags: - :tickets: - - added a "mods" system which allows pluggable modules that modify/augment - core functionality, using the function "install_mods(\*modnames)". - - .. change:: - :tags: - :tickets: - - added the first "mod", SelectResults, which modifies mapper selects to - return generators that turn ranges into LIMIT/OFFSET queries - (Jonas Borgstr? - - .. change:: - :tags: - :tickets: - - factored out querying capabilities of Mapper into a separate Query object - which is Session-centric. this improves the performance of mapper.using(session) - and makes other things possible. - - .. change:: - :tags: - :tickets: - - objectstore/Session refactored, the official way to save objects is now - via the flush() method. The begin/commit functionality of Session is factored - into LegacySession which is still established as the default behavior, until - the 0.2 series. - - .. change:: - :tags: - :tickets: - - types system is bound to an engine at query compile time, not schema - construction time. this simplifies the types system as well as the ProxyEngine. - - .. change:: - :tags: - :tickets: - - added 'version_id' keyword argument to mapper. this keyword should reference a - Column object with type Integer, preferably non-nullable, which will be used on - the mapped table to track version numbers. this number is incremented on each - save operation and is specified in the UPDATE/DELETE conditions so that it - factors into the returned row count, which results in a ConcurrencyError if the - value received is not the expected count. - - .. change:: - :tags: - :tickets: - - added 'entity_name' keyword argument to mapper. a mapper is now associated - with a class via the class object as well as an optional entity_name parameter, - which is a string defaulting to None. any number of primary mappers can be - created for a class, qualified by the entity name. instances of those classes - will issue all of their load and save operations through their - entity_name-qualified mapper, and maintain separate a identity in the identity - map for an otherwise equivalent object. - - .. change:: - :tags: - :tickets: - - overhaul to the attributes system. code has been clarified, and also fixed to - support proper polymorphic behavior on object attributes. - - .. change:: - :tags: - :tickets: - - added "for_update" flag to Select objects - - .. change:: - :tags: - :tickets: - - some fixes for backrefs - - .. change:: - :tags: - :tickets: - - fix for postgres1 DateTime type - - .. change:: - :tags: - :tickets: - - documentation pages mostly switched over to Markdown syntax - -.. changelog:: - :version: 0.1.5 - :released: Mon Mar 27 2006 - - .. change:: - :tags: - :tickets: - - added SQLSession concept to SQLEngine. this object keeps track of retrieving a - connection from the connection pool as well as an in-progress transaction. - methods push_session() and pop_session() added to SQLEngine which push/pop a new - SQLSession onto the engine, allowing operation upon a second connection "nested" - within the previous one, allowing nested transactions. Other tricks are sure to - come later regarding SQLSession. - - .. change:: - :tags: - :tickets: - - added nest_on argument to objectstore.Session. This is a single SQLEngine or - list of engines for which push_session()/pop_session() will be called each time - this Session becomes the active session (via objectstore.push_session() or - equivalent). This allows a unit of work Session to take advantage of the nested - transaction feature without explicitly calling push_session/pop_session on the - engine. - - .. change:: - :tags: - :tickets: - - factored apart objectstore/unitofwork to separate "Session scoping" from - "uow commit heavy lifting" - - .. change:: - :tags: - :tickets: - - added populate_instance() method to MapperExtension. allows an extension to - modify the population of object attributes. this method can call the - populate_instance() method on another mapper to proxy the attribute population - from one mapper to another; some row translation logic is also built in to help - with this. - - .. change:: - :tags: - :tickets: - - fixed Oracle8-compatibility "use_ansi" flag which converts JOINs to - comparisons with the = and (+) operators, passes basic unittests - - .. change:: - :tags: - :tickets: - - tweaks to Oracle LIMIT/OFFSET support - - .. change:: - :tags: - :tickets: - - Oracle reflection uses ALL_** views instead of USER_** to get larger - list of stuff to reflect from - - .. change:: - :tags: - :tickets: 105 - - fixes to Oracle foreign key reflection - - .. change:: - :tags: - :tickets: - - objectstore.commit(obj1, obj2,...) adds an extra step to seek out private - relations on properties and delete child objects, even though its not a global - commit - - .. change:: - :tags: - :tickets: - - lots and lots of fixes to mappers which use inheritance, strengthened the - concept of relations on a mapper being made towards the "local" table for that - mapper, not the tables it inherits. allows more complex compositional patterns - to work with lazy/eager loading. - - .. change:: - :tags: - :tickets: - - added support for mappers to inherit from others based on the same table, - just specify the same table as that of both parent/child mapper. - - .. change:: - :tags: - :tickets: - - some minor speed improvements to the attributes system with regards to - instantiating and populating new objects. - - .. change:: - :tags: - :tickets: - - fixed MySQL binary unit test - - .. change:: - :tags: - :tickets: - - INSERTs can receive clause elements as VALUES arguments, not just literal - values - - .. change:: - :tags: - :tickets: - - support for calling multi-tokened functions, i.e. schema.mypkg.func() - - .. change:: - :tags: - :tickets: - - added J. Ellis' SQLSoup module to extensions package - - .. change:: - :tags: - :tickets: - - added "polymorphic" examples illustrating methods to load multiple object types - from one mapper, the second of which uses the new populate_instance() method. - small improvements to mapper, UNION construct to help the examples along - - .. change:: - :tags: - :tickets: - - improvements/fixes to session.refresh()/session.expire() (which may have - been called "invalidate" earlier..) - - .. change:: - :tags: - :tickets: - - added session.expunge() which totally removes an object from the current - session - - .. change:: - :tags: - :tickets: - - added \*args, \**kwargs pass-through to engine.transaction(func) allowing easier - creation of transactionalizing decorator functions - - .. change:: - :tags: - :tickets: - - added iterator interface to ResultProxy: "for row in result:..." - - .. change:: - :tags: - :tickets: - - added assertion to tx = session.begin(); tx.rollback(); tx.begin(), i.e. can't - use it after a rollback() - - .. change:: - :tags: - :tickets: - - added date conversion on bind parameter fix to SQLite enabling dates to - work with pysqlite1 - - .. change:: - :tags: - :tickets: 116 - - improvements to subqueries to more intelligently construct their FROM - clauses - - .. change:: - :tags: - :tickets: - - added PickleType to types. - - .. change:: - :tags: - :tickets: - - fixed two bugs with column labels with regards to bind parameters: bind param - keynames they are now generated from a column "label" in all relevant cases to - take advantage of excess-name-length rules, and checks for a peculiar collision - against a column named the same as "tablename_colname" added - - .. change:: - :tags: - :tickets: - - major overhaul to unit of work documentation, other documentation sections. - - .. change:: - :tags: - :tickets: - - fixed attributes bug where if an object is committed, its lazy-loaded list got - blown away if it hadn't been loaded - - .. change:: - :tags: - :tickets: - - added unique_connection() method to engine, connection pool to return a - connection that is not part of the thread-local context or any current - transaction - - .. change:: - :tags: - :tickets: - - added invalidate() function to pooled connection. will remove the connection - from the pool. still need work for engines to auto-reconnect to a stale DB - though. - - .. change:: - :tags: - :tickets: - - added distinct() function to column elements so you can do - func.count(mycol.distinct()) - - .. change:: - :tags: - :tickets: - - added "always_refresh" flag to Mapper, creates a mapper that will always - refresh the attributes of objects it gets/selects from the DB, overwriting any - changes made. - -.. changelog:: - :version: 0.1.4 - :released: Mon Mar 13 2006 - - .. change:: - :tags: - :tickets: - - create_engine() now uses genericized parameters; host/hostname, - db/dbname/database, password/passwd, etc. for all engine connections. makes - engine URIs much more "universal" - - .. change:: - :tags: - :tickets: - - added support for SELECT statements embedded into a column clause, using the - flag "scalar=True" - - .. change:: - :tags: - :tickets: - - another overhaul to EagerLoading when used in conjunction with mappers that - inherit; improvements to eager loads figuring out their aliased queries - correctly, also relations set up against a mapper with inherited mappers will - create joins against the table that is specific to the mapper itself (i.e. and - not any tables that are inherited/are further down the inheritance chain), - this can be overridden by using custom primary/secondary joins. - - .. change:: - :tags: - :tickets: - - added J.Ellis patch to mapper.py so that selectone() throws an exception - if query returns more than one object row, selectfirst() to not throw the - exception. also adds selectfirst_by (synonymous with get_by) and selectone_by - - .. change:: - :tags: - :tickets: - - added onupdate parameter to Column, will exec SQL/python upon an update - statement.Also adds "for_update=True" to all DefaultGenerator subclasses - - .. change:: - :tags: - :tickets: - - added support for Oracle table reflection contributed by Andrija Zaric; - still some bugs to work out regarding composite primary keys/dictionary selection - - .. change:: - :tags: - :tickets: - - checked in an initial Firebird module, awaiting testing. - - .. change:: - :tags: - :tickets: - - added sql.ClauseParameters dictionary object as the result for - compiled.get_params(), does late-typeprocessing of bind parameters so - that the original values are easier to access - - .. change:: - :tags: - :tickets: - - more docs for indexes, column defaults, connection pooling, engine construction - - .. change:: - :tags: - :tickets: - - overhaul to the construction of the types system. uses a simpler inheritance - pattern so that any of the generic types can be easily subclassed, with no need - for TypeDecorator. - - .. change:: - :tags: - :tickets: - - added "convert_unicode=False" parameter to SQLEngine, will cause all String - types to perform unicode encoding/decoding (makes Strings act like Unicodes) - - .. change:: - :tags: - :tickets: - - added 'encoding="utf8"' parameter to engine. the given encoding will be - used for all encode/decode calls within Unicode types as well as Strings - when convert_unicode=True. - - .. change:: - :tags: - :tickets: - - improved support for mapping against UNIONs, added polymorph.py example - to illustrate multi-class mapping against a UNION - - .. change:: - :tags: - :tickets: - - fix to SQLite LIMIT/OFFSET syntax - - .. change:: - :tags: - :tickets: - - fix to Oracle LIMIT syntax - - .. change:: - :tags: - :tickets: - - added backref() function, allows backreferences to have keyword arguments - that will be passed to the backref. - - .. change:: - :tags: - :tickets: - - Sequences and ColumnDefault objects can do execute()/scalar() standalone - - .. change:: - :tags: - :tickets: - - SQL functions (i.e. func.foo()) can do execute()/scalar() standalone - - .. change:: - :tags: - :tickets: - - fix to SQL functions so that the ANSI-standard functions, i.e. current_timestamp - etc., do not specify parenthesis. all other functions do. - - .. change:: - :tags: - :tickets: - - added settattr_clean and append_clean to SmartProperty, which set - attributes without triggering a "dirty" event or any history. used as: - myclass.prop1.setattr_clean(myobject, 'hi') - - .. change:: - :tags: - :tickets: - - improved support to column defaults when used by mappers; mappers will pull - pre-executed defaults from statement's executed bind parameters - (pre-conversion) to populate them into a saved object's attributes; if any - PassiveDefaults have fired off, will instead post-fetch the row from the DB to - populate the object. - - .. change:: - :tags: - :tickets: - - added 'get_session().invalidate(\*obj)' method to objectstore, instances will - refresh() themselves upon the next attribute access. - - .. change:: - :tags: - :tickets: - - improvements to SQL func calls including an "engine" keyword argument so - they can be execute()d or scalar()ed standalone, also added func accessor to - SQLEngine - - .. change:: - :tags: - :tickets: - - fix to MySQL4 custom table engines, i.e. TYPE instead of ENGINE - - .. change:: - :tags: - :tickets: - - slightly enhanced logging, includes timestamps and a somewhat configurable - formatting system, in lieu of a full-blown logging system - - .. change:: - :tags: - :tickets: - - improvements to the ActiveMapper class from the TG gang, including - many-to-many relationships - - .. change:: - :tags: - :tickets: - - added Double and TinyInt support to mysql - -.. changelog:: - :version: 0.1.3 - :released: Thu Mar 02 2006 - - .. change:: - :tags: - :tickets: - - completed "post_update" feature, will add a second update statement before - inserts and after deletes in order to reconcile a relationship without any - dependencies being created; used when persisting two rows that are dependent - on each other - - .. change:: - :tags: - :tickets: - - completed mapper.using(session) function, localized per-object Session - functionality; objects can be declared and manipulated as local to any - user-defined Session - - .. change:: - :tags: - :tickets: - - fix to Oracle "row_number over" clause with multiple tables - - .. change:: - :tags: - :tickets: - - mapper.get() was not selecting multiple-keyed objects if the mapper's table was a join, - such as in an inheritance relationship, this is fixed. - - .. change:: - :tags: - :tickets: - - overhaul to sql/schema packages so that the sql package can run all on its own, - producing selects, inserts, etc. without any engine dependencies. builds upon - new TableClause/ColumnClause lexical objects. Schema's Table/Column objects - are the "physical" subclasses of them. simplifies schema/sql relationship, - extensions (like proxyengine), and speeds overall performance by a large margin. - removes the entire getattr() behavior that plagued 0.1.1. - - .. change:: - :tags: - :tickets: - - refactoring of how the mapper "synchronizes" data between two objects into a - separate module, works better with properties attached to a mapper that has an - additional inheritance relationship to one of the related tables, also the same - methodology used to synchronize parent/child objects now used by mapper to - synchronize between inherited and inheriting mappers. - - .. change:: - :tags: - :tickets: - - made objectstore "check for out-of-identitymap" more aggressive, will perform the - check when object attributes are modified or the object is deleted - - .. change:: - :tags: - :tickets: - - Index object fully implemented, can be constructed standalone, or via - "index" and "unique" arguments on Columns. - - .. change:: - :tags: - :tickets: - - added "convert_unicode" flag to SQLEngine, will treat all String/CHAR types - as Unicode types, with raw-byte/utf-8 translation on the bind parameter and - result set side. - - .. change:: - :tags: - :tickets: - - postgres maintains a list of ANSI functions that must have no parenthesis so - function calls with no arguments work consistently - - .. change:: - :tags: - :tickets: - - tables can be created with no engine specified. this will default their engine - to a module-scoped "default engine" which is a ProxyEngine. this engine can - be connected via the function "global_connect". - - .. change:: - :tags: - :tickets: - - added "refresh(\*obj)" method to objectstore / Session to reload the attributes of - any set of objects from the database unconditionally - -.. changelog:: - :version: 0.1.2 - :released: Fri Feb 24 2006 - - .. change:: - :tags: - :tickets: - - fixed a recursive call in schema that was somehow running 994 times then returning - normally. broke nothing, slowed down everything. thanks to jpellerin for finding this. - -.. changelog:: - :version: 0.1.1 - :released: Thu Feb 23 2006 - - .. change:: - :tags: - :tickets: - - small fix to Function class so that expressions with a func.foo() use the type of the - Function object (i.e. the left side) as the type of the boolean expression, not the - other side which is more of a moving target (changeset 1020). - - .. change:: - :tags: - :tickets: - - creating self-referring mappers with backrefs slightly easier (but still not that easy - - changeset 1019) - - .. change:: - :tags: - :tickets: - - fixes to one-to-one mappings (changeset 1015) - - .. change:: - :tags: - :tickets: - - psycopg1 date/time issue with None fixed (changeset 1005) - - .. change:: - :tags: - :tickets: - - two issues related to postgres, which doesn't want to give you the "lastrowid" - since oids are deprecated: - - * postgres database-side defaults that are on primary key cols *do* execute - explicitly beforehand, even though that's not the idea of a PassiveDefault. this is - because sequences on columns get reflected as PassiveDefaults, but need to be explicitly - executed on a primary key col so we know what we just inserted. - * if you did add a row that has a bunch of database-side defaults on it, - and the PassiveDefault thing was working the old way, i.e. they just execute on - the DB side, the "can't get the row back without an OID" exception that occurred - also will not happen unless someone (usually the ORM) explicitly asks for it. - - .. change:: - :tags: - :tickets: - - fixed a glitch with engine.execute_compiled where it was making a second - ResultProxy that just got thrown away. - - .. change:: - :tags: - :tickets: - - began to implement newer logic in object properties. you can now say - myclass.attr.property, which will give you the PropertyLoader corresponding to that - attribute, i.e. myclass.mapper.props['attr'] - - .. change:: - :tags: - :tickets: - - eager loading has been internally overhauled to use aliases at all times. more - complicated chains of eager loads can now be created without any need for explicit - "use aliases"-type instructions. EagerLoader code is also much simpler now. - - .. change:: - :tags: - :tickets: - - a new somewhat experimental flag "use_update" added to relations, indicates that - this relationship should be handled by a second UPDATE statement, either after a - primary INSERT or before a primary DELETE. handles circular row dependencies. - - .. change:: - :tags: - :tickets: - - added exceptions module, all raised exceptions (except for some - KeyError/AttributeError exceptions) descend from these classes. - - .. change:: - :tags: - :tickets: - - fix to date types with MySQL, returned timedelta converted to datetime.time - - .. change:: - :tags: - :tickets: - - two-phase objectstore.commit operations (i.e. begin/commit) now return a - transactional object (SessionTrans), to more clearly indicate transaction boundaries. - - .. change:: - :tags: - :tickets: - - Index object with create/drop support added to schema - - .. change:: - :tags: - :tickets: - - fix to postgres, where it will explicitly pre-execute a PassiveDefault on a table - if it is a primary key column, pursuant to the ongoing "we can't get inserted rows - back from postgres" issue - - .. change:: - :tags: - :tickets: - - change to information_schema query that gets back postgres table defs, now - uses explicit JOIN keyword, since one user had faster performance with 8.1 - - .. change:: - :tags: - :tickets: - - fix to engine.process_defaults so it works correctly with a table that has - different column name/column keys (changeset 982) - - .. change:: - :tags: - :tickets: - - a column can only be attached to one table - this is now asserted - - .. change:: - :tags: - :tickets: - - postgres time types descend from Time type - - .. change:: - :tags: - :tickets: - - fix to alltests so that it runs types test (now named testtypes) - - .. change:: - :tags: - :tickets: - - fix to Join object so that it correctly exports its foreign keys (cs 973) - - .. change:: - :tags: - :tickets: - - creating relationships against mappers that use inheritance fixed (cs 973) diff --git a/doc/build/changelog/changelog_02.rst b/doc/build/changelog/changelog_02.rst deleted file mode 100644 index 3d40a79a32a..00000000000 --- a/doc/build/changelog/changelog_02.rst +++ /dev/null @@ -1,1191 +0,0 @@ - -============= -0.2 Changelog -============= - - -.. changelog:: - :version: 0.2.8 - :released: Tue Sep 05 2006 - - .. change:: - :tags: - :tickets: - - cleanup on connection methods + documentation. custom DBAPI - arguments specified in query string, 'connect_args' argument - to 'create_engine', or custom creation function via 'creator' - function to 'create_engine'. - - .. change:: - :tags: - :tickets: 274 - - added "recycle" argument to Pool, is "pool_recycle" on create_engine, - defaults to 3600 seconds; connections after this age will be closed and - replaced with a new one, to handle db's that automatically close - stale connections - - .. change:: - :tags: - :tickets: 121 - - changed "invalidate" semantics with pooled connection; will - instruct the underlying connection record to reconnect the next - time its called. "invalidate" will also automatically be called - if any error is thrown in the underlying call to connection.cursor(). - this will hopefully allow the connection pool to reconnect to a - database that had been stopped and started without restarting - the connecting application - - .. change:: - :tags: - :tickets: - - eesh ! the tutorial doctest was broken for quite some time. - - .. change:: - :tags: - :tickets: - - add_property() method on mapper does a "compile all mappers" - step in case the given property references a non-compiled mapper - (as it did in the case of the tutorial !) - - .. change:: - :tags: - :tickets: 277 - - check for pg sequence already existing before create - - .. change:: - :tags: - :tickets: - - if a contextual session is established via MapperExtension.get_session - (as it is using the sessioncontext plugin, etc), a lazy load operation - will use that session by default if the parent object is not - persistent with a session already. - - .. change:: - :tags: - :tickets: - - lazy loads will not fire off for an object that does not have a - database identity (why? - see https://www.sqlalchemy.org/trac/wiki/WhyDontForeignKeysLoadData) - - .. change:: - :tags: - :tickets: - - unit-of-work does a better check for "orphaned" objects that are - part of a "delete-orphan" cascade, for certain conditions where the - parent isn't available to cascade from. - - .. change:: - :tags: - :tickets: - - mappers can tell if one of their objects is an "orphan" based - on interactions with the attribute package. this check is based - on a status flag maintained for each relationship - when objects are attached and detached from each other. - - .. change:: - :tags: - :tickets: - - it is now invalid to declare a self-referential relationship with - "delete-orphan" (as the abovementioned check would make them impossible - to save) - - .. change:: - :tags: - :tickets: - - improved the check for objects being part of a session when the - unit of work seeks to flush() them as part of a relationship.. - - .. change:: - :tags: - :tickets: 280 - - statement execution supports using the same BindParam - object more than once in an expression; simplified handling of positional - parameters. nice job by Bill Noon figuring out the basic idea. - - .. change:: - :tags: - :tickets: 60, 71 - - postgres reflection moved to use pg_schema tables, can be overridden - with use_information_schema=True argument to create_engine. - - .. change:: - :tags: - :tickets: 155 - - added case_sensitive argument to MetaData, Table, Column, determines - itself automatically based on if a parent schemaitem has a non-None - setting for the flag, or if not, then whether the identifier name is all lower - case or not. when set to True, quoting is applied to identifiers with mixed or - uppercase identifiers. quoting is also applied automatically in all cases to - identifiers that are known to be reserved words or contain other non-standard - characters. various database dialects can override all of this behavior, but - currently they are all using the default behavior. tested with postgres, mysql, - sqlite, oracle. needs more testing with firebird, ms-sql. part of the ongoing - work with - - .. change:: - :tags: - :tickets: - - unit tests updated to run without any pysqlite installed; pool - test uses a mock DBAPI - - .. change:: - :tags: - :tickets: 281 - - urls support escaped characters in passwords - - .. change:: - :tags: - :tickets: - - added limit/offset to UNION queries (though not yet in oracle) - - .. change:: - :tags: - :tickets: - - added "timezone=True" flag to DateTime and Time types. postgres - so far will convert this to "TIME[STAMP] (WITH|WITHOUT) TIME ZONE", - so that control over timezone presence is more controllable (psycopg2 - returns datetimes with tzinfo's if available, which can create confusion - against datetimes that don't). - - .. change:: - :tags: - :tickets: 287 - - fix to using query.count() with distinct, \**kwargs with SelectResults - count() - - .. change:: - :tags: - :tickets: 289 - - deregister Table from MetaData when autoload fails; - - .. change:: - :tags: - :tickets: 293 - - import of py2.5s sqlite3 - - .. change:: - :tags: - :tickets: 296 - - unicode fix for startswith()/endswith() - -.. changelog:: - :version: 0.2.7 - :released: Sat Aug 12 2006 - - .. change:: - :tags: - :tickets: - - quoting facilities set up so that database-specific quoting can be - turned on for individual table, schema, and column identifiers when - used in all queries/creates/drops. Enabled via "quote=True" in - Table or Column, as well as "quote_schema=True" in Table. Thanks to - Aaron Spike for the excellent efforts. - - .. change:: - :tags: - :tickets: - - assignmapper was setting is_primary=True, causing all sorts of mayhem - by not raising an error when redundant mappers were set up, fixed - - .. change:: - :tags: - :tickets: - - added allow_null_pks option to Mapper, allows rows where some - primary key columns are null (i.e. when mapping to outer joins etc) - - .. change:: - :tags: - :tickets: - - modification to unitofwork to not maintain ordering within the - "new" list or within the UOWTask "objects" list; instead, new objects - are tagged with an ordering identifier as they are registered as new - with the session, and the INSERT statements are then sorted within the - mapper save_obj. the INSERT ordering has basically been pushed all - the way to the end of the flush cycle. that way the various sorts and - organizations occurring within UOWTask (particularly the circular task - sort) don't have to worry about maintaining order (which they weren't anyway) - - .. change:: - :tags: - :tickets: - - fixed reflection of foreign keys to autoload the referenced table - if it was not loaded already - - .. change:: - :tags: - :tickets: 256 - - - pass URL query string arguments to connect() function - - .. change:: - :tags: - :tickets: 257 - - - oracle boolean type - - .. change:: - :tags: - :tickets: - - custom primary/secondary join conditions in a relation *will* be propagated - to backrefs by default. specifying a backref() will override this behavior. - - .. change:: - :tags: - :tickets: - - better check for ambiguous join conditions in sql.Join; propagates to a - better error message in PropertyLoader (i.e. relation()/backref()) for when - the join condition can't be reasonably determined. - - .. change:: - :tags: - :tickets: - - sqlite creates ForeignKeyConstraint objects properly upon table - reflection. - - .. change:: - :tags: - :tickets: 224 - - adjustments to pool stemming from changes made for. - overflow counter should only be decremented if the connection actually - succeeded. added a test script to attempt testing this. - - .. change:: - :tags: - :tickets: - - fixed mysql reflection of default values to be PassiveDefault - - .. change:: - :tags: - :tickets: 263, 264 - - added reflected 'tinyint', 'mediumint' type to MS-SQL. - - .. change:: - :tags: - :tickets: - - SingletonThreadPool has a size and does a cleanup pass, so that - only a given number of thread-local connections stay around (needed - for sqlite applications that dispose of threads en masse) - - .. change:: - :tags: - :tickets: 267, 265 - - fixed small pickle bug(s) with lazy loaders - - .. change:: - :tags: - :tickets: - - fixed possible error in mysql reflection where certain versions - return an array instead of string for SHOW CREATE TABLE call - - .. change:: - :tags: - :tickets: 1770 - - fix to lazy loads when mapping to joins - - .. change:: - :tags: - :tickets: - - all create()/drop() calls have a keyword argument of "connectable". - "engine" is deprecated. - - .. change:: - :tags: - :tickets: - - fixed ms-sql connect() to work with adodbapi - - .. change:: - :tags: - :tickets: - - added "nowait" flag to Select() - - .. change:: - :tags: - :tickets: 271 - - inheritance check uses issubclass() instead of direct __mro__ check - to make sure class A inherits from B, allowing mapper inheritance to more - flexibly correspond to class inheritance - - .. change:: - :tags: - :tickets: 252 - - SelectResults will use a subselect, when calling an aggregate (i.e. - max, min, etc.) on a SelectResults that has an ORDER BY clause - - .. change:: - :tags: - :tickets: 269 - - fixes to types so that database-specific types more easily used; - fixes to mysql text types to work with this methodology - - .. change:: - :tags: - :tickets: - - some fixes to sqlite date type organization - - .. change:: - :tags: - :tickets: 263 - - added MSTinyInteger to MS-SQL - -.. changelog:: - :version: 0.2.6 - :released: Thu Jul 20 2006 - - .. change:: - :tags: - :tickets: 76 - - big overhaul to schema to allow truly composite primary and foreign - key constraints, via new ForeignKeyConstraint and PrimaryKeyConstraint - objects. - Existing methods of primary/foreign key creation have not been changed - but use these new objects behind the scenes. table creation - and reflection is now more table oriented rather than column oriented. - - .. change:: - :tags: - :tickets: - - overhaul to MapperExtension calling scheme, wasn't working very well - previously - - .. change:: - :tags: - :tickets: - - tweaks to ActiveMapper, supports self-referential relationships - - .. change:: - :tags: - :tickets: - - slight rearrangement to objectstore (in activemapper/threadlocal) - so that the SessionContext is referenced by '.context' instead - of subclassed directly. - - .. change:: - :tags: - :tickets: - - activemapper will use threadlocal's objectstore if the mod is - activated when activemapper is imported - - .. change:: - :tags: - :tickets: - - small fix to URL regexp to allow filenames with '@' in them - - .. change:: - :tags: - :tickets: - - fixes to Session expunge/update/etc...needs more cleanup. - - .. change:: - :tags: - :tickets: - - select_table mappers *still* weren't always compiling - - .. change:: - :tags: - :tickets: - - fixed up Boolean datatype - - .. change:: - :tags: - :tickets: - - added count()/count_by() to list of methods proxied by assignmapper; - this also adds them to activemapper - - .. change:: - :tags: - :tickets: - - connection exceptions wrapped in DBAPIError - - .. change:: - :tags: - :tickets: - - ActiveMapper now supports autoloading column definitions from the - database if you supply a __autoload__ = True attribute in your - mapping inner-class. Currently this does not support reflecting - any relationships. - - .. change:: - :tags: - :tickets: - - deferred column load could screw up the connection status in - a flush() under some circumstances, this was fixed - - .. change:: - :tags: - :tickets: - - expunge() was not working with cascade, fixed. - - .. change:: - :tags: - :tickets: - - potential endless loop in cascading operations fixed. - - .. change:: - :tags: - :tickets: - - added "synonym()" function, applied to properties to have a - propname the same as another, for the purposes of overriding props - and allowing the original propname to be accessible in select_by(). - - .. change:: - :tags: - :tickets: - - fix to typing in clause construction which specifically helps - type issues with polymorphic_union (CAST/ColumnClause propagates - its type to proxy columns) - - .. change:: - :tags: - :tickets: - - mapper compilation work ongoing, someday it'll work....moved - around the initialization of MapperProperty objects to be after - all mappers are created to better handle circular compilations. - do_init() method is called on all properties now which are more - aware of their "inherited" status if so. - - .. change:: - :tags: - :tickets: - - eager loads explicitly disallowed on self-referential relationships, or - relationships to an inheriting mapper (which is also self-referential) - - .. change:: - :tags: - :tickets: 244 - - reduced bind param size in query._get to appease the picky oracle - - .. change:: - :tags: - :tickets: 234 - - added 'checkfirst' argument to table.create()/table.drop(), as - well as table.exists() - - .. change:: - :tags: - :tickets: 245 - - some other ongoing fixes to inheritance - - .. change:: - :tags: - :tickets: - - attribute/backref/orphan/history-tracking tweaks as usual... - -.. changelog:: - :version: 0.2.5 - :released: Sat Jul 08 2006 - - .. change:: - :tags: - :tickets: - - fixed endless loop bug in select_by(), if the traversal hit - two mappers that referenced each other - - .. change:: - :tags: - :tickets: - - upgraded all unittests to insert './lib/' into sys.path, - working around new setuptools PYTHONPATH-killing behavior - - .. change:: - :tags: - :tickets: - - further fixes with attributes/dependencies/etc.... - - .. change:: - :tags: - :tickets: - - improved error handling for when DynamicMetaData is not connected - - .. change:: - :tags: - :tickets: - - MS-SQL support largely working (tested with pymssql) - - .. change:: - :tags: - :tickets: - - ordering of UPDATE and DELETE statements within groups is now - in order of primary key values, for more deterministic ordering - - .. change:: - :tags: - :tickets: - - after_insert/delete/update mapper extensions now called per object, - not per-object-per-table - - .. change:: - :tags: - :tickets: - - further fixes/refactorings to mapper compilation - -.. changelog:: - :version: 0.2.4 - :released: Tue Jun 27 2006 - - .. change:: - :tags: - :tickets: - - try/except when the mapper sets init.__name__ on a mapped class, - supports python 2.3 - - .. change:: - :tags: - :tickets: - - fixed bug where threadlocal engine would still autocommit - despite a transaction in progress - - .. change:: - :tags: - :tickets: - - lazy load and deferred load operations require the parent object - to be in a Session to do the operation; whereas before the operation - would just return a blank list or None, it now raises an exception. - - .. change:: - :tags: - :tickets: - - Session.update() is slightly more lenient if the session to which - the given object was formerly attached to was garbage collected; - otherwise still requires you explicitly remove the instance from - the previous Session. - - .. change:: - :tags: - :tickets: - - fixes to mapper compilation, checking for more error conditions - - .. change:: - :tags: - :tickets: - - small fix to eager loading combined with ordering/limit/offset - - .. change:: - :tags: - :tickets: 206 - - utterly remarkable: added a single space between 'CREATE TABLE' - and '(' since *that's how MySQL indicates a non- - reserved word tablename.....* - - .. change:: - :tags: - :tickets: - - more fixes to inheritance, related to many-to-many relations - properly saving - - .. change:: - :tags: - :tickets: - - fixed bug when specifying explicit module to mysql dialect - - .. change:: - :tags: - :tickets: - - when QueuePool times out it raises a TimeoutError instead of - erroneously making another connection - - .. change:: - :tags: - :tickets: - - Queue.Queue usage in pool has been replaced with a locally - modified version (works in py2.3/2.4!) that uses a threading.RLock - for a mutex. this is to fix a reported case where a ConnectionFairy's - __del__() method got called within the Queue's get() method, which - then returns its connection to the Queue via the put() method, - causing a reentrant hang unless threading.RLock is used. - - .. change:: - :tags: - :tickets: - - postgres will not place SERIAL keyword on a primary key column - if it has a foreign key constraint - - .. change:: - :tags: - :tickets: 221 - - cursor() method on ConnectionFairy allows db-specific extension - arguments to be propagated - - .. change:: - :tags: - :tickets: 225 - - lazy load bind params properly propagate column type - - .. change:: - :tags: - :tickets: - - new MySQL types: MSEnum, MSTinyText, MSMediumText, MSLongText, etc. - more support for MS-specific length/precision params in numeric types - patch courtesy Mike Bernson - - .. change:: - :tags: - :tickets: 224 - - some fixes to connection pool invalidate() - -.. changelog:: - :version: 0.2.3 - :released: Sat Jun 17 2006 - - .. change:: - :tags: - :tickets: - - overhaul to mapper compilation to be deferred. this allows mappers - to be constructed in any order, and their relationships to each - other are compiled when the mappers are first used. - - .. change:: - :tags: - :tickets: - - fixed a pretty big speed bottleneck in cascading behavior particularly - when backrefs were in use - - .. change:: - :tags: - :tickets: - - the attribute instrumentation module has been completely rewritten; its - now a large degree simpler and clearer, slightly faster. the "history" - of an attribute is no longer micromanaged with each change and is - instead part of a "CommittedState" object created when the - instance is first loaded. HistoryArraySet is gone, the behavior of - list attributes is now more open ended (i.e. they're not sets anymore). - - .. change:: - :tags: - :tickets: - - py2.4 "set" construct used internally, falls back to sets.Set when - "set" not available/ordering is needed. - - .. change:: - :tags: - :tickets: - - fix to transaction control, so that repeated rollback() calls - don't fail (was failing pretty badly when flush() would raise - an exception in a larger try/except transaction block) - - .. change:: - :tags: - :tickets: 151 - - "foreignkey" argument to relation() can also be a list. fixed - auto-foreignkey detection - - .. change:: - :tags: - :tickets: - - fixed bug where tables with schema names weren't getting indexed in - the MetaData object properly - - .. change:: - :tags: - :tickets: 207 - - fixed bug where Column with redefined "key" property wasn't getting - type conversion happening in the ResultProxy - - .. change:: - :tags: - :tickets: - - fixed 'port' attribute of URL to be an integer if present - - .. change:: - :tags: - :tickets: - - fixed old bug where if a many-to-many table mapped as "secondary" - had extra columns, delete operations didn't work - - .. change:: - :tags: - :tickets: - - bugfixes for mapping against UNION queries - - .. change:: - :tags: - :tickets: - - fixed incorrect exception class thrown when no DB driver present - - .. change:: - :tags: - :tickets: 138 - - added NonExistentTable exception thrown when reflecting a table - that doesn't exist - - .. change:: - :tags: - :tickets: - - small fix to ActiveMapper regarding one-to-one backrefs, other - refactorings - - .. change:: - :tags: - :tickets: - - overridden constructor in mapped classes gets __name__ and - __doc__ from the original class - - .. change:: - :tags: - :tickets: 200 - - fixed small bug in selectresult.py regarding mapper extension - - .. change:: - :tags: - :tickets: - - small tweak to cascade_mappers, not very strongly supported - function at the moment - - .. change:: - :tags: - :tickets: 202 - - some fixes to between(), column.between() to propagate typing - information better - - .. change:: - :tags: - :tickets: 203 - - if an object fails to be constructed, is not added to the - session - - .. change:: - :tags: - :tickets: - - CAST function has been made into its own clause object with - its own compilation function in ansicompiler; allows MySQL - to silently ignore most CAST calls since MySQL - seems to only support the standard CAST syntax with Date types. - MySQL-compatible CAST support for strings, ints, etc. a TODO - -.. changelog:: - :version: 0.2.2 - :released: Mon Jun 05 2006 - - .. change:: - :tags: - :tickets: 190 - - big improvements to polymorphic inheritance behavior, enabling it - to work with adjacency list table structures - - .. change:: - :tags: - :tickets: - - major fixes and refactorings to inheritance relationships overall, - more unit tests - - .. change:: - :tags: - :tickets: - - fixed "echo_pool" flag on create_engine() - - .. change:: - :tags: - :tickets: - - fix to docs, removed incorrect info that close() is unsafe to use - with threadlocal strategy (its totally safe !) - - .. change:: - :tags: - :tickets: 188 - - create_engine() can take URLs as string or unicode - - .. change:: - :tags: - :tickets: - - firebird support partially completed; - thanks to James Ralston and Brad Clements for their efforts. - - .. change:: - :tags: - :tickets: - - Oracle url translation was broken, fixed, will feed host/port/sid - into cx_oracle makedsn() if 'database' field is present, else uses - straight TNS name from the 'host' field - - .. change:: - :tags: - :tickets: - - fix to using unicode criterion for query.get()/query.load() - - .. change:: - :tags: - :tickets: - - count() function on selectables now uses table primary key or - first column instead of "1" for criterion, also uses label "rowcount" - instead of "count". - - .. change:: - :tags: - :tickets: - - got rudimental "mapping to multiple tables" functionality cleaned up, - more correctly documented - - .. change:: - :tags: - :tickets: - - restored global_connect() function, attaches to a DynamicMetaData - instance called "default_metadata". leaving MetaData arg to Table - out will use the default metadata. - - .. change:: - :tags: - :tickets: - - fixes to session cascade behavior, entity_name propagation - - .. change:: - :tags: - :tickets: - - reorganized unittests into subdirectories - - .. change:: - :tags: - :tickets: - - more fixes to threadlocal connection nesting patterns - -.. changelog:: - :version: 0.2.1 - :released: Mon May 29 2006 - - .. change:: - :tags: - :tickets: - - "pool" argument to create_engine() properly propagates - - .. change:: - :tags: - :tickets: - - fixes to URL, raises exception if not parsed, does not pass blank - fields along to the DB connect string (a string such as - user:host@/db was breaking on postgres) - - .. change:: - :tags: - :tickets: - - small fixes to Mapper when it inserts and tries to get - new primary key values back - - .. change:: - :tags: - :tickets: - - rewrote half of TLEngine, the ComposedSQLEngine used with - 'strategy="threadlocal"'. it now properly implements engine.begin()/ - engine.commit(), which nest fully with connection.begin()/trans.commit(). - added about six unittests. - - .. change:: - :tags: - :tickets: - - major "duh" in pool.Pool, forgot to put back the WeakValueDictionary. - unittest which was supposed to check for this was also silently missing - it. fixed unittest to ensure that ConnectionFairy properly falls out - of scope. - - .. change:: - :tags: - :tickets: - - placeholder dispose() method added to SingletonThreadPool, doesn't - do anything yet - - .. change:: - :tags: - :tickets: - - rollback() is automatically called when an exception is raised, - but only if there's no transaction in process (i.e. works more like - autocommit). - - .. change:: - :tags: - :tickets: - - fixed exception raise in sqlite if no sqlite module present - - .. change:: - :tags: - :tickets: - - added extra example detail for association object doc - - .. change:: - :tags: - :tickets: - - Connection adds checks for already being closed - -.. changelog:: - :version: 0.2.0 - :released: Sat May 27 2006 - - .. change:: - :tags: - :tickets: - - overhaul to Engine system so that what was formerly the SQLEngine - is now a ComposedSQLEngine which consists of a variety of components, - including a Dialect, ConnectionProvider, etc. This impacted all the - db modules as well as Session and Mapper. - - .. change:: - :tags: - :tickets: - - create_engine now takes only RFC-1738-style strings: - ``driver://user:password@host:port/database`` - - **update** this format is generally but not exactly RFC-1738, - including that underscores, not dashes or periods, are accepted in the - "scheme" portion. - - .. change:: - :tags: - :tickets: 152 - - total rewrite of connection-scoping methodology, Connection objects - can now execute clause elements directly, added explicit "close" as - well as support throughout Engine/ORM to handle closing properly, - no longer relying upon __del__ internally to return connections - to the pool. - - .. change:: - :tags: - :tickets: - - overhaul to Session interface and scoping. uses hibernate-style - methods, including query(class), save(), save_or_update(), etc. - no threadlocal scope is installed by default. Provides a binding - interface to specific Engines and/or Connections so that underlying - Schema objects do not need to be bound to an Engine. Added a basic - SessionTransaction object that can simplistically aggregate transactions - across multiple engines. - - .. change:: - :tags: - :tickets: - - overhaul to mapper's dependency and "cascade" behavior; dependency logic - factored out of properties.py into a separate module "dependency.py". - "cascade" behavior is now explicitly controllable, proper implementation - of "delete", "delete-orphan", etc. dependency system can now determine at - flush time if a child object has a parent or not so that it makes better - decisions on how that child should be updated in the DB with regards to deletes. - - .. change:: - :tags: - :tickets: - - overhaul to Schema to build upon MetaData object instead of an Engine. - Entire SQL/Schema system can be used with no Engines whatsoever, executed - solely by an explicit Connection object. the "bound" methodology exists via the - BoundMetaData for schema objects. ProxyEngine is generally not needed - anymore and is replaced by DynamicMetaData. - - .. change:: - :tags: - :tickets: 167 - - true polymorphic behavior implemented, fixes - - .. change:: - :tags: - :tickets: 147 - - "oid" system has been totally moved into compile-time behavior; - if they are used in an order_by where they are not available, the order_by - doesn't get compiled, fixes - - .. change:: - :tags: - :tickets: - - overhaul to packaging; "mapping" is now "orm", "objectstore" is now - "session", the old "objectstore" namespace gets loaded in via the - "threadlocal" mod if used - - .. change:: - :tags: - :tickets: - - mods now called in via "import ". extensions favored over - mods as mods are globally-monkeypatching - - .. change:: - :tags: - :tickets: 154 - - fix to add_property so that it propagates properties to inheriting - mappers - - .. change:: - :tags: - :tickets: - - backrefs create themselves against primary mapper of its originating - property, primary/secondary join arguments can be specified to override. - helps their usage with polymorphic mappers - - .. change:: - :tags: - :tickets: 31 - - "table exists" function has been implemented - - .. change:: - :tags: - :tickets: 98 - - "create_all/drop_all" added to MetaData object - - .. change:: - :tags: - :tickets: - - improvements and fixes to topological sort algorithm, as well as more - unit tests - - .. change:: - :tags: - :tickets: - - tutorial page added to docs which also can be run with a custom doctest - runner to ensure its properly working. docs generally overhauled to - deal with new code patterns - - .. change:: - :tags: - :tickets: - - many more fixes, refactorings. - - .. change:: - :tags: - :tickets: - - migration guide is available on the Wiki at - https://www.sqlalchemy.org/trac/wiki/02Migration diff --git a/doc/build/changelog/changelog_03.rst b/doc/build/changelog/changelog_03.rst deleted file mode 100644 index f2ffb81e3d2..00000000000 --- a/doc/build/changelog/changelog_03.rst +++ /dev/null @@ -1,2927 +0,0 @@ - -============= -0.3 Changelog -============= - - -.. changelog:: - :version: 0.3.11 - :released: Sun Oct 14 2007 - - .. change:: - :tags: sql - :tickets: - - tweak DISTINCT precedence for clauses like - `func.count(t.c.col.distinct())` - - .. change:: - :tags: sql - :tickets: 719 - - Fixed detection of internal '$' characters in :bind$params - - .. change:: - :tags: sql - :tickets: 768 - - don't assume join criterion consists only of column objects - - .. change:: - :tags: sql - :tickets: 764 - - adjusted operator precedence of NOT to match '==' and others, so that - ~(x==y) produces NOT (x=y), which is compatible with MySQL < 5.0 - (doesn't like "NOT x=y") - - .. change:: - :tags: orm - :tickets: 687 - - added a check for joining from A->B using join(), along two - different m2m tables. this raises an error in 0.3 but is - possible in 0.4 when aliases are used. - - .. change:: - :tags: orm - :tickets: - - fixed small exception throw bug in Session.merge() - - .. change:: - :tags: orm - :tickets: - - fixed bug where mapper, being linked to a join where one table had - no PK columns, would not detect that the joined table had no PK. - - .. change:: - :tags: orm - :tickets: 769 - - fixed bugs in determining proper sync clauses from custom inherit - conditions - - .. change:: - :tags: orm - :tickets: 813 - - backref remove object operation doesn't fail if the other-side - collection doesn't contain the item, supports noload collections - - .. change:: - :tags: engine - :tickets: - - fixed another occasional race condition which could occur - when using pool with threadlocal setting - - .. change:: - :tags: mysql - :tickets: - - fixed specification of YEAR columns when generating schema - - .. change:: - :tags: mssql - :tickets: 679 - - added support for TIME columns (simulated using DATETIME) - - .. change:: - :tags: mssql - :tickets: 721 - - added support for BIGINT, MONEY, SMALLMONEY, UNIQUEIDENTIFIER and - SQL_VARIANT - - .. change:: - :tags: mssql - :tickets: 684 - - index names are now quoted when dropping from reflected tables - - .. change:: - :tags: mssql - :tickets: - - can now specify a DSN for PyODBC, using a URI like mssql:///?dsn=bob - - .. change:: - :tags: postgres - :tickets: - - when reflecting tables from alternate schemas, the "default" placed upon - the primary key, i.e. usually a sequence name, has the "schema" name - unconditionally quoted, so that schema names which need quoting are fine. - its slightly unnecessary for schema names which don't need quoting - but not harmful. - - .. change:: - :tags: sqlite - :tickets: - - passthrough for stringified dates - - .. change:: - :tags: firebird - :tickets: - - supports_sane_rowcount() set to False due to ticket #370 (right way). - - .. change:: - :tags: firebird - :tickets: - - fixed reflection of Column's nullable property. - - .. change:: - :tags: oracle - :tickets: 622, 751 - - removed LONG_STRING, LONG_BINARY from "binary" types, so type objects - don't try to read their values as LOB. - -.. changelog:: - :version: 0.3.10 - :released: Fri Jul 20 2007 - - .. change:: - :tags: general - :tickets: - - a new mutex that was added in 0.3.9 causes the pool_timeout - feature to fail during a race condition; threads would - raise TimeoutError immediately with no delay if many threads - push the pool into overflow at the same time. this issue has been - fixed. - - .. change:: - :tags: sql - :tickets: - - got connection-bound metadata to work with implicit execution - - .. change:: - :tags: sql - :tickets: 667 - - foreign key specs can have any character in their identifiers - - .. change:: - :tags: sql - :tickets: 664 - - added commutativity-awareness to binary clause comparisons to - each other, improves ORM lazy load optimization - - .. change:: - :tags: orm - :tickets: - - cleanup to connection-bound sessions, SessionTransaction - - .. change:: - :tags: postgres - :tickets: 571 - - fixed max identifier length (63) - -.. changelog:: - :version: 0.3.9 - :released: Sun Jul 15 2007 - - .. change:: - :tags: general - :tickets: 607 - - better error message for NoSuchColumnError - - .. change:: - :tags: general - :tickets: 428 - - finally figured out how to get setuptools version in, available - as sqlalchemy.__version__ - - .. change:: - :tags: general - :tickets: - - the various "engine" arguments, such as "engine", "connectable", - "engine_or_url", "bind_to", etc. are all present, but deprecated. - they all get replaced by the single term "bind". you also - set the "bind" of MetaData using - metadata.bind = - - .. change:: - :tags: ext - :tickets: - - iteration over dict association proxies is now dict-like, not - InstrumentedList-like (e.g. over keys instead of values) - - .. change:: - :tags: ext - :tickets: 597 - - association proxies no longer bind tightly to source collections, and are constructed with a thunk instead - - .. change:: - :tags: ext - :tickets: - - added selectone_by() to assignmapper - - .. change:: - :tags: orm - :tickets: - - forwards-compatibility with 0.4: added one(), first(), and - all() to Query. almost all Query functionality from 0.4 is - present in 0.3.9 for forwards-compat purposes. - - .. change:: - :tags: orm - :tickets: - - reset_joinpoint() really really works this time, promise ! lets - you re-join from the root: - query.join(['a', 'b']).filter().reset_joinpoint().\ - join(['a', 'c']).filter().all() - in 0.4 all join() calls start from the "root" - - .. change:: - :tags: orm - :tickets: 613 - - added synchronization to the mapper() construction step, to avoid - thread collisions when pre-existing mappers are compiling in a - different thread - - .. change:: - :tags: orm - :tickets: - - a warning is issued by Mapper when two primary key columns of the - same name are munged into a single attribute. this happens frequently - when mapping to joins (or inheritance). - - .. change:: - :tags: orm - :tickets: 598 - - synonym() properties are fully supported by all Query joining/ - with_parent operations - - .. change:: - :tags: orm - :tickets: - - fixed very stupid bug when deleting items with many-to-many - uselist=False relations - - .. change:: - :tags: orm - :tickets: - - remember all that stuff about polymorphic_union ? for - joined table inheritance ? Funny thing... - You sort of don't need it for joined table inheritance, you - can just string all the tables together via outerjoin(). - The UNION still applies if concrete tables are involved, - though (since nothing to join them on). - - .. change:: - :tags: orm - :tickets: - - small fix to eager loading to better work with eager loads - to polymorphic mappers that are using a straight "outerjoin" - clause - - .. change:: - :tags: sql - :tickets: - - ForeignKey to a table in a schema that's not the default schema - requires the schema to be explicit; i.e. ForeignKey('alt_schema.users.id') - - .. change:: - :tags: sql - :tickets: - - MetaData can now be constructed with an engine or url as the first - argument, just like BoundMetaData - - .. change:: - :tags: sql - :tickets: - - BoundMetaData is now deprecated, and MetaData is a direct substitute. - - .. change:: - :tags: sql - :tickets: - - DynamicMetaData has been renamed to ThreadLocalMetaData. the - DynamicMetaData name is deprecated and is an alias for ThreadLocalMetaData - or a regular MetaData if threadlocal=False - - .. change:: - :tags: sql - :tickets: - - composite primary key is represented as a non-keyed set to allow for - composite keys consisting of cols with the same name; occurs within a - Join. helps inheritance scenarios formulate correct PK. - - .. change:: - :tags: sql - :tickets: 185 - - improved ability to get the "correct" and most minimal set of primary key - columns from a join, equating foreign keys and otherwise equated columns. - this is also mostly to help inheritance scenarios formulate the best - choice of primary key columns. - - .. change:: - :tags: sql - :tickets: - - added 'bind' argument to Sequence.create()/drop(), ColumnDefault.execute() - - .. change:: - :tags: sql - :tickets: 650 - - columns can be overridden in a reflected table with a "key" - attribute different than the column's name, including for primary key - columns - - .. change:: - :tags: sql - :tickets: 657 - - fixed "ambiguous column" result detection, when dupe col names exist - in a result - - .. change:: - :tags: sql - :tickets: - - some enhancements to "column targeting", the ability to match a column - to a "corresponding" column in another selectable. this affects mostly - ORM ability to map to complex joins - - .. change:: - :tags: sql - :tickets: 619 - - MetaData and all SchemaItems are safe to use with pickle. slow - table reflections can be dumped into a pickled file to be reused later. - Just reconnect the engine to the metadata after unpickling. - - .. change:: - :tags: sql - :tickets: - - added a mutex to QueuePool's "overflow" calculation to prevent a race - condition that can bypass max_overflow - - .. change:: - :tags: sql - :tickets: 623 - - fixed grouping of compound selects to give correct results. will break - on sqlite in some cases, but those cases were producing incorrect - results anyway, sqlite doesn't support grouped compound selects - - .. change:: - :tags: sql - :tickets: 620 - - fixed precedence of operators so that parenthesis are correctly applied - - .. change:: - :tags: sql - :tickets: 545 - - calling .in_() (i.e. with no arguments) will return - "CASE WHEN ( IS NULL) THEN NULL ELSE 0 END = 1)", so that - NULL or False is returned in all cases, rather than throwing an error - - .. change:: - :tags: sql - :tickets: - - fixed "where"/"from" criterion of select() to accept a unicode string - in addition to regular string - both convert to text() - - .. change:: - :tags: sql - :tickets: 558 - - added standalone distinct() function in addition to column.distinct() - - .. change:: - :tags: sql - :tickets: - - result.last_inserted_ids() should return a list that is identically - sized to the primary key constraint of the table. values that were - "passively" created and not available via cursor.lastrowid will be None. - - .. change:: - :tags: sql - :tickets: 589 - - long-identifier detection fixed to use > rather than >= for - max ident length - - .. change:: - :tags: sql - :tickets: 593 - - fixed bug where selectable.corresponding_column(selectable.c.col) - would not return selectable.c.col, if the selectable is a join - of a table and another join involving the same table. messed - up ORM decision making - - .. change:: - :tags: sql - :tickets: 595 - - added Interval type to types.py - - .. change:: - :tags: mysql - :tickets: 625 - - fixed catching of some errors that imply a dropped connection - - .. change:: - :tags: mysql - :tickets: 624 - - fixed escaping of the modulo operator - - .. change:: - :tags: mysql - :tickets: 590 - - added 'fields' to reserved words - - .. change:: - :tags: mysql - :tickets: - - various reflection enhancement/fixes - - .. change:: - :tags: oracle - :tickets: 604 - - datetime fixes: got subsecond TIMESTAMP to work, - added OracleDate which supports types.Date with only year/month/day - - .. change:: - :tags: oracle - :tickets: - - added dialect flag "auto_convert_lobs", defaults to True; will cause any - LOB objects detected in a result set to be forced into OracleBinary - so that the LOB is read() automatically, if no typemap was present - (i.e., if a textual execute() was issued). - - .. change:: - :tags: oracle - :tickets: 624 - - mod operator '%' produces MOD - - .. change:: - :tags: oracle - :tickets: 542 - - converts cx_oracle datetime objects to Python datetime.datetime when - Python 2.3 used - - .. change:: - :tags: oracle - :tickets: - - fixed unicode conversion in Oracle TEXT type - - .. change:: - :tags: postgres - :tickets: 624 - - fixed escaping of the modulo operator - - .. change:: - :tags: postgres - :tickets: 570 - - added support for reflection of domains - - .. change:: - :tags: postgres - :tickets: - - types which are missing during reflection resolve to Null type - instead of raising an error - - .. change:: - :tags: postgres - :tickets: - - the fix in "schema" above fixes reflection of foreign keys from an - alt-schema table to a public schema table - - .. change:: - :tags: sqlite - :tickets: - - rearranged dialect initialization so it has time to warn about pysqlite1 - being too old. - - .. change:: - :tags: sqlite - :tickets: - - sqlite better handles datetime/date/time objects mixed and matched - with various Date/Time/DateTime columns - - .. change:: - :tags: sqlite - :tickets: 603 - - string PK column inserts don't get overwritten with OID - - .. change:: - :tags: mssql - :tickets: 634 - - fix port option handling for pyodbc - - .. change:: - :tags: mssql - :tickets: - - now able to reflect start and increment values for identity columns - - .. change:: - :tags: mssql - :tickets: - - preliminary support for using scope_identity() with pyodbc - -.. changelog:: - :version: 0.3.8 - :released: Sat Jun 02 2007 - - .. change:: - :tags: engines - :tickets: - - added detach() to Connection, allows underlying DBAPI connection - to be detached from its pool, closing on dereference/close() - instead of being reused by the pool. - - .. change:: - :tags: engines - :tickets: - - added invalidate() to Connection, immediately invalidates the - Connection and its underlying DBAPI connection. - - .. change:: - :tags: sql - :tickets: - - _Label class overrides compare_self to return its ultimate - object. meaning, if you say someexpr.label('foo') == 5, it - produces the correct "someexpr == 5". - - .. change:: - :tags: sql - :tickets: - - _Label propagates "_hide_froms()" so that scalar selects - behave more properly with regards to FROM clause #574 - - .. change:: - :tags: sql - :tickets: - - fix to long name generation when using oid_column as an order by - (oids used heavily in mapper queries) - - .. change:: - :tags: sql - :tickets: - - significant speed improvement to ResultProxy, pre-caches - TypeEngine dialect implementations and saves on function calls - per column - - .. change:: - :tags: sql - :tickets: - - parenthesis are applied to clauses via a new _Grouping - construct. uses operator precedence to more intelligently apply - parenthesis to clauses, provides cleaner nesting of clauses - (doesn't mutate clauses placed in other clauses, i.e. no 'parens' - flag) - - .. change:: - :tags: sql - :tickets: - - added 'modifier' keyword, works like func. except does not - add parenthesis. e.g. select([modifier.DISTINCT(...)]) etc. - - .. change:: - :tags: sql - :tickets: 578 - - removed "no group by's in a select that's part of a UNION" - restriction - - .. change:: - :tags: orm - :tickets: - - added reset_joinpoint() method to Query, moves the "join point" - back to the starting mapper. 0.4 will change the behavior of - join() to reset the "join point" in all cases so this is an - interim method. for forwards compatibility, ensure joins across - multiple relations are specified using a single join(), i.e. - join(['a', 'b', 'c']). - - .. change:: - :tags: orm - :tickets: - - fixed bug in query.instances() that wouldn't handle more than - on additional mapper or one additional column. - - .. change:: - :tags: orm - :tickets: - - "delete-orphan" no longer implies "delete". ongoing effort to - separate the behavior of these two operations. - - .. change:: - :tags: orm - :tickets: - - many-to-many relationships properly set the type of bind params - for delete operations on the association table - - .. change:: - :tags: orm - :tickets: - - many-to-many relationships check that the number of rows deleted - from the association table by a delete operation matches the - expected results - - .. change:: - :tags: orm - :tickets: - - session.get() and session.load() propagate \**kwargs through to - query - - .. change:: - :tags: orm - :tickets: 577 - - fix to polymorphic query which allows the original - polymorphic_union to be embedded into a correlated subquery - - .. change:: - :tags: orm - :tickets: - - fix to select_by(=) -style joins in - conjunction with many-to-many relationships, bug introduced in - r2556 - - .. change:: - :tags: orm - :tickets: - - the "primary_key" argument to mapper() is propagated to the - "polymorphic" mapper. primary key columns in this list get - normalized to that of the mapper's local table. - - .. change:: - :tags: orm - :tickets: - - restored logging of "lazy loading clause" under - sa.orm.strategies logger, got removed in 0.3.7 - - .. change:: - :tags: orm - :tickets: - - improved support for eagerloading of properties off of mappers - that are mapped to select() statements; i.e. eagerloader is - better at locating the correct selectable with which to attach - its LEFT OUTER JOIN. - - .. change:: - :tags: mysql - :tickets: - - Nearly all MySQL column types are now supported for declaration - and reflection. Added NCHAR, NVARCHAR, VARBINARY, TINYBLOB, - LONGBLOB, YEAR - - .. change:: - :tags: mysql - :tickets: - - The sqltypes.Binary passthrough now always builds a BLOB, - avoiding problems with very old database versions - - .. change:: - :tags: mysql - :tickets: - - support for column-level CHARACTER SET and COLLATE declarations, - as well as ASCII, UNICODE, NATIONAL and BINARY shorthand. - - .. change:: - :tags: firebird - :tickets: - - set max identifier length to 31 - - .. change:: - :tags: firebird - :tickets: - - supports_sane_rowcount() set to False due to ticket #370. - versioned_id_col feature won't work in FB. - - .. change:: - :tags: firebird - :tickets: - - some execution fixes - - .. change:: - :tags: firebird - :tickets: - - new association proxy implementation, implementing complete - proxies to list, dict and set-based relation collections - - .. change:: - :tags: firebird - :tickets: - - added orderinglist, a custom list class that synchronizes an - object attribute with that object's position in the list - - .. change:: - :tags: firebird - :tickets: - - small fix to SelectResultsExt to not bypass itself during - select(). - - .. change:: - :tags: firebird - :tickets: - - added filter(), filter_by() to assignmapper - -.. changelog:: - :version: 0.3.7 - :released: Sun Apr 29 2007 - - .. change:: - :tags: engines - :tickets: - - warnings module used for issuing warnings (instead of logging) - - .. change:: - :tags: engines - :tickets: 480 - - cleanup of DBAPI import strategies across all engines - - .. change:: - :tags: engines - :tickets: - - refactoring of engine internals which reduces complexity, - number of codepaths; places more state inside of ExecutionContext - to allow more dialect control of cursor handling, result sets. - ResultProxy totally refactored and also has two versions of - "buffered" result sets used for different purposes. - - .. change:: - :tags: engines - :tickets: 514 - - server side cursor support fully functional in postgres. - - .. change:: - :tags: engines - :tickets: - - improved framework for auto-invalidation of connections that have - lost their underlying database, via dialect-specific detection - of exceptions corresponding to that database's disconnect - related error messages. Additionally, when a "connection no - longer open" condition is detected, the entire connection pool - is discarded and replaced with a new instance. #516 - - .. change:: - :tags: engines - :tickets: 521 - - the dialects within sqlalchemy.databases become a setuptools - entry points. loading the built-in database dialects works the - same as always, but if none found will fall back to trying - pkg_resources to load an external module - - .. change:: - :tags: engines - :tickets: - - Engine contains a "url" attribute referencing the url.URL object - used by create_engine(). - - .. change:: - :tags: sql - :tickets: - - keys() of result set columns are not lowercased, come back - exactly as they're expressed in cursor.description. note this - causes colnames to be all caps in oracle. - - .. change:: - :tags: sql - :tickets: - - preliminary support for unicode table names, column names and - SQL statements added, for databases which can support them. - Works with sqlite and postgres so far. MySQL *mostly* works - except the has_table() function does not work. Reflection - works too. - - .. change:: - :tags: sql - :tickets: 522 - - the Unicode type is now a direct subclass of String, which now - contains all the "convert_unicode" logic. This helps the variety - of unicode situations that occur in db's such as MS-SQL to be - better handled and allows subclassing of the Unicode datatype. - - .. change:: - :tags: sql - :tickets: - - ClauseElements can be used in in_() clauses now, such as bind - parameters, etc. #476 - - .. change:: - :tags: sql - :tickets: - - reverse operators implemented for `CompareMixin` elements, - allows expressions like "5 + somecolumn" etc. #474 - - .. change:: - :tags: sql - :tickets: - - the "where" criterion of an update() and delete() now correlates - embedded select() statements against the table being updated or - deleted. this works the same as nested select() statement - correlation, and can be disabled via the correlate=False flag on - the embedded select(). - - .. change:: - :tags: sql - :tickets: 512 - - column labels are now generated in the compilation phase, which - means their lengths are dialect-dependent. So on oracle a label - that gets truncated to 30 chars will go out to 63 characters - on postgres. Also, the true labelname is always attached as the - accessor on the parent Selectable so there's no need to be aware - of the "truncated" label names. - - .. change:: - :tags: sql - :tickets: - - column label and bind param "truncation" also generate - deterministic names now, based on their ordering within the - full statement being compiled. this means the same statement - will produce the same string across application restarts and - allowing DB query plan caching to work better. - - .. change:: - :tags: sql - :tickets: 513 - - the "mini" column labels generated when using subqueries, which - are to work around glitchy SQLite behavior that doesn't understand - "foo.id" as equivalent to "id", are now only generated in the case - that those named columns are selected from (part of) - - .. change:: - :tags: sql - :tickets: - - the label() method on ColumnElement will properly propagate the - TypeEngine of the base element out to the label, including a label() - created from a scalar=True select() statement. - - .. change:: - :tags: sql - :tickets: 513 - - MS-SQL better detects when a query is a subquery and knows not to - generate ORDER BY phrases for those - - .. change:: - :tags: sql - :tickets: 505 - - fix for fetchmany() "size" argument being positional in most - dbapis - - .. change:: - :tags: sql - :tickets: - - sending None as an argument to func. will produce - an argument of NULL - - .. change:: - :tags: sql - :tickets: - - query strings in unicode URLs get keys encoded to ascii - for \**kwargs compat - - .. change:: - :tags: sql - :tickets: 523 - - slight tweak to raw execute() change to also support tuples - for positional parameters, not just lists - - .. change:: - :tags: sql - :tickets: - - fix to case() construct to propagate the type of the first - WHEN condition as the return type of the case statement - - .. change:: - :tags: orm - :tickets: - - fixed critical issue when, after options(eagerload()) is used, - the mapper would then always apply query "wrapping" behavior - for all subsequent LIMIT/OFFSET/DISTINCT queries, even if no - eager loading was applied on those subsequent queries. - - .. change:: - :tags: orm - :tickets: 541 - - added query.with_parent(someinstance) method. searches for - target instance using lazy join criterion from parent instance. - takes optional string "property" to isolate the desired relation. - also adds static Query.query_from_parent(instance, property) - version. - - .. change:: - :tags: orm - :tickets: 554 - - improved query.XXX_by(someprop=someinstance) querying to use - similar methodology to with_parent, i.e. using the "lazy" clause - which prevents adding the remote instance's table to the SQL, - thereby making more complex conditions possible - - .. change:: - :tags: orm - :tickets: - - added generative versions of aggregates, i.e. sum(), avg(), etc. - to query. used via query.apply_max(), apply_sum(), etc. - #552 - - .. change:: - :tags: orm - :tickets: - - fix to using distinct() or distinct=True in combination with - join() and similar - - .. change:: - :tags: orm - :tickets: - - corresponding to label/bindparam name generation, eager loaders - generate deterministic names for the aliases they create using - md5 hashes. - - .. change:: - :tags: orm - :tickets: - - improved/fixed custom collection classes when giving it "set"/ - "sets.Set" classes or subclasses (was still looking for append() - methods on them during lazy loads) - - .. change:: - :tags: orm - :tickets: - - restored old "column_property()" ORM function (used to be called - "column()") to force any column expression to be added as a property - on a mapper, particularly those that aren't present in the mapped - selectable. this allows "scalar expressions" of any kind to be - added as relations (though they have issues with eager loads). - - .. change:: - :tags: orm - :tickets: 533 - - fix to many-to-many relationships targeting polymorphic mappers - - .. change:: - :tags: orm - :tickets: 543 - - making progress with session.merge() as well as combining its - usage with entity_name - - .. change:: - :tags: orm - :tickets: - - the usual adjustments to relationships between inheriting mappers, - in this case establishing relation()s to subclass mappers where - the join conditions come from the superclass' table - - .. change:: - :tags: informix - :tickets: - - informix support added ! courtesy James Zhang, who put a ton - of effort in. - - .. change:: - :tags: sqlite - :tickets: - - removed silly behavior where sqlite would reflect UNIQUE indexes - as part of the primary key (?!) - - .. change:: - :tags: oracle - :tickets: - - small fix to allow successive compiles of the same SELECT object - which features LIMIT/OFFSET. oracle dialect needs to modify - the object to have ROW_NUMBER OVER and wasn't performing - the full series of steps on successive compiles. - - .. change:: - :tags: mysql - :tickets: - - support for SSL arguments given as inline within URL query string, - prefixed with "ssl\_", courtesy terjeros@gmail.com. - - .. change:: - :tags: , mysql - :tickets: - - mysql uses "DESCRIBE.", catching exceptions - if table doesn't exist, in order to determine if a table exists. - this supports unicode table names as well as schema names. tested - with MySQL5 but should work with 4.1 series as well. (#557) - - .. change:: - :tags: extensions - :tickets: - - big fix to AssociationProxy so that multiple AssociationProxy - objects can be associated with a single association collection. - - .. change:: - :tags: extensions - :tickets: - - assign_mapper names methods according to their keys (i.e. __name__) - #551 - - .. change:: - :tags: mssql - :tickets: - - pyodbc is now the preferred DB-API for MSSQL, and if no module is - specifically requested, will be loaded first on a module probe. - - .. change:: - :tags: mssql - :tickets: - - The @@SCOPE_IDENTITY is now used instead of @@IDENTITY. This - behavior may be overridden with the engine_connect - "use_scope_identity" keyword parameter, which may also be specified - in the dburi. - -.. changelog:: - :version: 0.3.6 - :released: Fri Mar 23 2007 - - .. change:: - :tags: sql - :tickets: - - bindparam() names are now repeatable! specify two - distinct bindparam()s with the same name in a single statement, - and the key will be shared. proper positional/named args translate - at compile time. for the old behavior of "aliasing" bind parameters - with conflicting names, specify "unique=True" - this option is - still used internally for all the auto-generated (value-based) - bind parameters. - - .. change:: - :tags: sql - :tickets: - - slightly better support for bind params as column clauses, either - via bindparam() or via literal(), i.e. select([literal('foo')]) - - .. change:: - :tags: sql - :tickets: - - MetaData can bind to an engine either via "url" or "engine" kwargs - to constructor, or by using connect() method. BoundMetaData is - identical to MetaData except engine_or_url param is required. - DynamicMetaData is the same and provides thread-local connections be - default. - - .. change:: - :tags: sql - :tickets: - - exists() becomes usable as a standalone selectable, not just in a - WHERE clause, i.e. exists([columns], criterion).select() - - .. change:: - :tags: sql - :tickets: - - correlated subqueries work inside of ORDER BY, GROUP BY - - .. change:: - :tags: sql - :tickets: - - fixed function execution with explicit connections, i.e. - conn.execute(func.dosomething()) - - .. change:: - :tags: sql - :tickets: - - use_labels flag on select() won't auto-create labels for literal text - column elements, since we can make no assumptions about the text. to - create labels for literal columns, you can say "somecol AS - somelabel", or use literal_column("somecol").label("somelabel") - - .. change:: - :tags: sql - :tickets: - - quoting won't occur for literal columns when they are "proxied" into - the column collection for their selectable (is_literal flag is - propagated). literal columns are specified via - literal_column("somestring"). - - .. change:: - :tags: sql - :tickets: - - added "fold_equivalents" boolean argument to Join.select(), which - removes 'duplicate' columns from the resulting column clause that - are known to be equivalent based on the join condition. this is of - great usage when constructing subqueries of joins which Postgres - complains about if duplicate column names are present. - - .. change:: - :tags: sql - :tickets: 503 - - fixed use_alter flag on ForeignKeyConstraint - - .. change:: - :tags: sql - :tickets: 506 - - fixed usage of 2.4-only "reversed" in topological.py - - .. change:: - :tags: sql - :tickets: 501 - - for hackers, refactored the "visitor" system of ClauseElement and - SchemaItem so that the traversal of items is controlled by the - ClauseVisitor itself, using the method visitor.traverse(item). - accept_visitor() methods can still be called directly but will not - do any traversal of child items. ClauseElement/SchemaItem now have a - configurable get_children() method to return the collection of child - elements for each parent object. This allows the full traversal of - items to be clear and unambiguous (as well as loggable), with an - easy method of limiting a traversal (just pass flags which are - picked up by appropriate get_children() methods). - - .. change:: - :tags: sql - :tickets: - - the "else\_" parameter to the case statement now properly works when - set to zero. - - .. change:: - :tags: orm - :tickets: - - the full featureset of the SelectResults extension has been merged - into a new set of methods available off of Query. These methods - all provide "generative" behavior, whereby the Query is copied - and a new one returned with additional criterion added. - The new methods include: - - * filter() - applies select criterion to the query - * filter_by() - applies "by"-style criterion to the query - * avg() - return the avg() function on the given column - * join() - join to a property (or across a list of properties) - * outerjoin() - like join() but uses LEFT OUTER JOIN - * limit()/offset() - apply LIMIT/OFFSET range-based access - which applies limit/offset: session.query(Foo)[3:5] - * distinct() - apply DISTINCT - * list() - evaluate the criterion and return results - - no incompatible changes have been made to Query's API and no methods - have been deprecated. Existing methods like select(), select_by(), - get(), get_by() all execute the query at once and return results - like they always did. join_to()/join_via() are still there although - the generative join()/outerjoin() methods are easier to use. - - .. change:: - :tags: orm - :tickets: - - the return value for multiple mappers used with instances() now - returns a cartesian product of the requested list of mappers, - represented as a list of tuples. this corresponds to the documented - behavior. So that instances match up properly, the "uniquing" is - disabled when this feature is used. - - .. change:: - :tags: orm - :tickets: - - Query has add_entity() and add_column() generative methods. these - will add the given mapper/class or ColumnElement to the query at - compile time, and apply them to the instances() method. the user is - responsible for constructing reasonable join conditions (otherwise - you can get full cartesian products). result set is the list of - tuples, non-uniqued. - - .. change:: - :tags: orm - :tickets: - - strings and columns can also be sent to the \*args of instances() - where those exact result columns will be part of the result tuples. - - .. change:: - :tags: orm - :tickets: - - a full select() construct can be passed to query.select() (which - worked anyway), but also query.selectfirst(), query.selectone() - which will be used as is (i.e. no query is compiled). works - similarly to sending the results to instances(). - - .. change:: - :tags: orm - :tickets: 495 - - eager loading will not "aliasize" "order by" clauses that were - placed in the select statement by something other than the eager - loader itself, to fix possibility of dupe columns as illustrated in. however, this means you have to be more careful with - the columns placed in the "order by" of Query.select(), that you - have explicitly named them in your criterion (i.e. you can't rely on - the eager loader adding them in for you) - - .. change:: - :tags: orm - :tickets: - - added a handy multi-use "identity_key()" method to Session, allowing - the generation of identity keys for primary key values, instances, - and rows, courtesy Daniel Miller - - .. change:: - :tags: orm - :tickets: 249 - - many-to-many table will be properly handled even for operations that - occur on the "backref" side of the operation - - .. change:: - :tags: orm - :tickets: 492 - - added "refresh-expire" cascade. allows refresh() and - expire() calls to propagate along relationships. - - .. change:: - :tags: orm - :tickets: 493 - - more fixes to polymorphic relations, involving proper lazy-clause - generation on many-to-one relationships to polymorphic mappers. also fixes to detection of "direction", more specific - targeting of columns that belong to the polymorphic union vs. those - that don't. - - .. change:: - :tags: orm - :tickets: - - some fixes to relationship calcs when using "viewonly=True" to pull - in other tables into the join condition which aren't parent of the - relationship's parent/child mappings - - .. change:: - :tags: orm - :tickets: - - flush fixes on cyclical-referential relationships that contain - references to other instances outside of the cyclical chain, when - some of the objects in the cycle are not actually part of the flush - - .. change:: - :tags: orm - :tickets: 500 - - put an aggressive check for "flushing object A with a collection of - B's, but you put a C in the collection" error condition - **even if - C is a subclass of B**, unless B's mapper loads polymorphically. - Otherwise, the collection will later load a "B" which should be a - "C" (since its not polymorphic) which breaks in bi-directional - relationships (i.e. C has its A, but A's backref will lazyload it as - a different instance of type "B") This check is going - to bite some of you who do this without issues, so the error message - will also document a flag "enable_typechecks=False" to disable this - checking. But be aware that bi-directional relationships in - particular become fragile without this check. - - .. change:: - :tags: extensions - :tickets: 472 - - options() method on SelectResults now implemented "generatively" - like the rest of the SelectResults methods. But - you're going to just use Query now anyway. - - .. change:: - :tags: extensions - :tickets: - - query() method is added by assignmapper. this helps with - navigating to all the new generative methods on Query. - - .. change:: - :tags: ms-sql - :tickets: - - removed seconds input on DATE column types (probably - should remove the time altogether) - - .. change:: - :tags: ms-sql - :tickets: - - null values in float fields no longer raise errors - - .. change:: - :tags: ms-sql - :tickets: - - LIMIT with OFFSET now raises an error (MS-SQL has no OFFSET support) - - .. change:: - :tags: ms-sql - :tickets: 509 - - added an facility to use the MSSQL type VARCHAR(max) instead of TEXT - for large unsized string fields. Use the new "text_as_varchar" to - turn it on. - - .. change:: - :tags: ms-sql - :tickets: - - ORDER BY clauses without a LIMIT are now stripped in subqueries, as - MS-SQL forbids this usage - - .. change:: - :tags: ms-sql - :tickets: 480 - - cleanup of module importing code; specifiable DB-API module; more - explicit ordering of module preferences. - - .. change:: - :tags: oracle - :tickets: - - got binary working for any size input ! cx_oracle works fine, - it was my fault as BINARY was being passed and not BLOB for - setinputsizes (also unit tests weren't even setting input sizes). - - .. change:: - :tags: oracle - :tickets: - - also fixed CLOB read/write on a separate changeset. - - .. change:: - :tags: oracle - :tickets: - - auto_setinputsizes defaults to True for Oracle, fixed cases where - it improperly propagated bad types. - - .. change:: - :tags: mysql - :tickets: - - added a catchall \**kwargs to MSString, to help reflection of - obscure types (like "varchar() binary" in MS 4.0) - - .. change:: - :tags: mysql - :tickets: - - added explicit MSTimeStamp type which takes effect when using - types.TIMESTAMP. - -.. changelog:: - :version: 0.3.5 - :released: Thu Feb 22 2007 - - .. change:: - :tags: sql - :tickets: - - the value of "case_sensitive" defaults to True now, regardless of the - casing of the identifier, unless specifically set to False. this is - because the object might be label'ed as something else which does - contain mixed case, and propagating "case_sensitive=False" breaks that. - Other fixes to quoting when using labels and "fake" column objects - - .. change:: - :tags: sql - :tickets: - - added a "supports_execution()" method to ClauseElement, so that - individual kinds of clauses can express if they are appropriate for - executing...such as, you can execute a "select", but not a "Table" or a - "Join". - - .. change:: - :tags: sql - :tickets: - - fixed argument passing to straight textual execute() on engine, - connection. can handle \*args or a list instance for positional, \**kwargs - or a dict instance for named args, or a list of list or dicts to invoke - executemany() - - .. change:: - :tags: sql - :tickets: - - small fix to BoundMetaData to accept unicode or string URLs - - .. change:: - :tags: sql - :tickets: 466 - - fixed named PrimaryKeyConstraint generation courtesy - andrija at gmail - - .. change:: - :tags: sql - :tickets: 464 - - fixed generation of CHECK constraints on columns - - .. change:: - :tags: sql - :tickets: - - fixes to tometadata() operation to propagate Constraints at column and - table level - - .. change:: - :tags: oracle - :tickets: 436 - - when returning "rowid" as the ORDER BY column or in use with ROW_NUMBER - OVER, oracle dialect checks the selectable its being applied to and will - switch to table PK if not applicable, i.e. for a UNION. checking for - DISTINCT, GROUP BY (other places that rowid is invalid) still a TODO. - allows polymorphic mappings to function. - - .. change:: - :tags: oracle - :tickets: - - sequences on a non-pk column will properly fire off on INSERT - - .. change:: - :tags: oracle - :tickets: 435 - - added PrefetchingResultProxy support to pre-fetch LOB columns when they - are known to be present, fixes - - .. change:: - :tags: oracle - :tickets: 379 - - implemented reflection of tables based on synonyms, including across - dblinks - - .. change:: - :tags: oracle - :tickets: 363 - - issues a log warning when a related table can't be reflected due to - certain permission errors - - .. change:: - :tags: mysql - :tickets: - - fix to reflection on older DB's that might return array() type for - "show variables like" statements - - .. change:: - :tags: postgres - :tickets: 442 - - better reflection of sequences for alternate-schema Tables - - .. change:: - :tags: postgres - :tickets: - - sequences on a non-pk column will properly fire off on INSERT - - .. change:: - :tags: postgres - :tickets: 460, 444 - - added PGInterval type, PGInet type - - .. change:: - :tags: mssql - :tickets: 419 - - preliminary support for pyodbc (Yay!) - - .. change:: - :tags: mssql - :tickets: 298 - - better support for NVARCHAR types added - - .. change:: - :tags: mssql - :tickets: - - fix for commit logic on pymssql - - .. change:: - :tags: mssql - :tickets: 456 - - fix for query.get() with schema - - .. change:: - :tags: mssql - :tickets: 473 - - fix for non-integer relationships - - .. change:: - :tags: mssql - :tickets: 419 - - DB-API module now selectable at run-time - - .. change:: - :tags: tickets:422, 481, 415, mssql - :tickets: - - now passes many more unit tests - - .. change:: - :tags: mssql - :tickets: 479 - - better unittest compatibility with ANSI functions - - .. change:: - :tags: mssql - :tickets: 415 - - improved support for implicit sequence PK columns with auto-insert - - .. change:: - :tags: mssql - :tickets: 371 - - fix for blank password in adodbapi - - .. change:: - :tags: mssql - :tickets: 481 - - fixes to get unit tests working with pyodbc - - .. change:: - :tags: mssql - :tickets: - - fix to auto_identity_insert on db-url query - - .. change:: - :tags: mssql - :tickets: - - added query_timeout to db-url query params. currently works only for - pymssql - - .. change:: - :tags: mssql - :tickets: - - tested with pymssql 0.8.0 (which is now LGPL) - - .. change:: - :tags: orm, bugs - :tickets: 441, 448, 439 - - another refactoring to relationship calculation. Allows more accurate - ORM behavior with relationships from/to/between mappers, particularly - polymorphic mappers, also their usage with Query, SelectResults. tickets - include,,. - - .. change:: - :tags: orm, bugs - :tickets: - - removed deprecated method of specifying custom collections on classes; - you must now use the "collection_class" option. the old way was - beginning to produce conflicts when people used assign_mapper(), which - now patches an "options" method, in conjunction with a relationship - named "options". (relationships take precedence over monkeypatched - assign_mapper methods). - - .. change:: - :tags: orm, bugs - :tickets: 454 - - extension() query option propagates to Mapper._instance() method so that - all loading-related methods get called - - .. change:: - :tags: orm, bugs - :tickets: - - eager relation to an inheriting mapper won't fail if no rows returned for - the relationship. - - .. change:: - :tags: orm, bugs - :tickets: 486 - - eager relation loading bug fixed for eager relation on multiple - descendant classes - - .. change:: - :tags: orm, bugs - :tickets: 423 - - fix for very large topological sorts, courtesy ants.aasma at gmail - - .. change:: - :tags: orm, bugs - :tickets: - - eager loading is slightly more strict about detecting "self-referential" - relationships, specifically between polymorphic mappers. this results in - an "eager degrade" to lazy loading. - - .. change:: - :tags: orm, bugs - :tickets: 449 - - improved support for complex queries embedded into "where" criterion for - query.select() - - .. change:: - :tags: orm, bugs - :tickets: 485 - - mapper options like eagerload(), lazyload(), deferred(), will work for - "synonym()" relationships - - .. change:: - :tags: orm, bugs - :tickets: 445 - - fixed bug where cascade operations incorrectly included deleted - collection items in the cascade - - .. change:: - :tags: orm, bugs - :tickets: 478 - - fixed relationship deletion error when one-to-many child item is moved - to a new parent in a single unit of work - - .. change:: - :tags: orm, bugs - :tickets: - - fixed relationship deletion error where parent/child with a single - column as PK/FK on the child would raise a "blank out the primary key" - error, if manually deleted or "delete" cascade without "delete-orphan" - was used - - .. change:: - :tags: orm, bugs - :tickets: - - fix to deferred so that load operation doesn't mistakenly occur when only - PK col attributes are set - - .. change:: - :tags: orm, enhancements - :tickets: 385 - - implemented foreign_keys argument to mapper. use in - conjunction with primaryjoin/secondaryjoin arguments to specify/override - foreign keys defined on the Table instance. - - .. change:: - :tags: orm, enhancements - :tickets: - - contains_eager('foo') automatically implies eagerload('foo') - - .. change:: - :tags: orm, enhancements - :tickets: - - added "alias" argument to contains_eager(). use it to specify the string - name or Alias instance of an alias used in the query for the eagerly - loaded child items. easier to use than "decorator" - - .. change:: - :tags: orm, enhancements - :tickets: - - added "contains_alias()" option for result set mapping to an alias of - the mapped table - - .. change:: - :tags: orm, enhancements - :tickets: 468 - - added support for py2.5 "with" statement with SessionTransaction - - .. change:: - :tags: extensions - :tickets: - - added distinct() method to SelectResults. generally should only make a - difference when using count(). - - .. change:: - :tags: extensions - :tickets: 472 - - added options() method to SelectResults, equivalent to query.options() - - .. change:: - :tags: extensions - :tickets: 462 - - added optional __table_opts__ dictionary to ActiveMapper, will send kw - options to Table objects - - .. change:: - :tags: extensions - :tickets: 467 - - added selectfirst(), selectfirst_by() to assign_mapper - -.. changelog:: - :version: 0.3.4 - :released: Tue Jan 23 2007 - - .. change:: - :tags: general - :tickets: - - global "insure"->"ensure" change. in US english "insure" is actually - largely interchangeable with "ensure" (so says the dictionary), so I'm not - completely illiterate, but its definitely sub-optimal to "ensure" which is - non-ambiguous. - - .. change:: - :tags: sql - :tickets: - - added "fetchmany()" support to ResultProxy - - .. change:: - :tags: sql - :tickets: - - added support for column "key" attribute to be usable in - row[]/row. - - .. change:: - :tags: sql - :tickets: - - changed "BooleanExpression" to subclass from "BinaryExpression", so that - boolean expressions can also follow column-clause behaviors (i.e. label(), - etc). - - .. change:: - :tags: sql - :tickets: - - trailing underscores are trimmed from func. calls, such as func.if_() - - .. change:: - :tags: sql - :tickets: - - fix to correlation of subqueries when the column list of the select - statement is constructed with individual calls to append_column(); this - fixes an ORM bug whereby nested select statements were not getting - correlated with the main select generated by the Query object. - - .. change:: - :tags: sql - :tickets: - - another fix to subquery correlation so that a subquery which has only one - FROM element will *not* correlate that single element, since at least one - FROM element is required in a query. - - .. change:: - :tags: sql - :tickets: 414 - - default "timezone" setting is now False. this corresponds to Python's - datetime behavior as well as Postgres' timestamp/time types (which is the - only timezone-sensitive dialect at the moment) - - .. change:: - :tags: sql - :tickets: - - the "op()" function is now treated as an "operation", rather than a - "comparison". the difference is, an operation produces a BinaryExpression - from which further operations can occur whereas comparison produces the - more restrictive BooleanExpression - - .. change:: - :tags: sql - :tickets: - - trying to redefine a reflected primary key column as non-primary key raises - an error - - .. change:: - :tags: sql - :tickets: - - type system slightly modified to support TypeDecorators that can be - overridden by the dialect (ok, that's not very clear, it allows the mssql - tweak below to be possible) - - .. change:: - :tags: mssql - :tickets: - - added an NVarchar type (produces NVARCHAR), also MSUnicode which provides - Unicode-translation for the NVarchar regardless of dialect convert_unicode - setting. - - .. change:: - :tags: postgres - :tickets: 424 - - fix to the initial checkfirst for tables to take current schema into - account - - .. change:: - :tags: postgres - :tickets: - - postgres has an optional "server_side_cursors=True" flag which will utilize - server side cursors. these are appropriate for fetching only partial - results and are necessary for working with very large unbounded result - sets. While we'd like this to be the default behavior, different - environments seem to have different results and the causes have not been - isolated so we are leaving the feature off by default for now. Uses an - apparently undocumented psycopg2 behavior recently discovered on the - psycopg mailing list. - - .. change:: - :tags: postgres - :tickets: - - added "BIGSERIAL" support for postgres table with - PGBigInteger/autoincrement - - .. change:: - :tags: postgres - :tickets: 402 - - fixes to postgres reflection to better handle when schema names are - present; thanks to jason (at) ncsmags.com - - .. change:: - :tags: mysql - :tickets: 420 - - mysql is inconsistent with what kinds of quotes it uses in foreign keys - during a SHOW CREATE TABLE, reflection updated to accommodate for all three - styles - - .. change:: - :tags: mysql - :tickets: 418 - - mysql table create options work on a generic passthru now, i.e. Table(..., - mysql_engine='InnoDB', mysql_collate="latin1_german2_ci", - mysql_auto_increment="5", mysql_...), helps - - .. change:: - :tags: firebird - :tickets: 408 - - order of constraint creation puts primary key first before all other - constraints; required for firebird, not a bad idea for others - - .. change:: - :tags: firebird - :tickets: 409 - - Firebird fix to autoload multifield foreign keys - - .. change:: - :tags: firebird - :tickets: 409 - - Firebird NUMERIC type properly handles a type without precision - - .. change:: - :tags: oracle - :tickets: - - *slight* support for binary, but still need to figure out how to insert - reasonably large values (over 4K). requires auto_setinputsizes=True sent to - create_engine(), rows must be fully fetched individually, etc. - - .. change:: - :tags: orm - :tickets: - - poked the first hole in the can of worms: saying - query.select_by(somerelationname=someinstance) will create the join of the - primary key columns represented by "somerelationname"'s mapper to the - actual primary key in "someinstance". - - .. change:: - :tags: orm - :tickets: - - reworked how relations interact with "polymorphic" mappers, i.e. mappers - that have a select_table as well as polymorphic flags. better determination - of proper join conditions, interaction with user- defined join conditions, - and support for self-referential polymorphic mappers. - - .. change:: - :tags: orm - :tickets: - - related to polymorphic mapping relations, some deeper error checking when - compiling relations, to detect an ambiguous "primaryjoin" in the case that - both sides of the relationship have foreign key references in the primary - join condition. also tightened down conditions used to locate "relation - direction", associating the "foreignkey" of the relationship with the - "primaryjoin" - - .. change:: - :tags: orm - :tickets: - - a little bit of improvement to the concept of a "concrete" inheritance - mapping, though that concept is not well fleshed out yet (added test case - to support concrete mappers on top of a polymorphic base). - - .. change:: - :tags: orm - :tickets: - - fix to "proxy=True" behavior on synonym() - - .. change:: - :tags: orm - :tickets: 427 - - fixed bug where delete-orphan basically didn't work with many-to-many - relationships, backref presence generally hid the symptom - - .. change:: - :tags: orm - :tickets: - - added a mutex to the mapper compilation step. ive been reluctant to add any - kind of threading anything to SA but this is one spot that its really - needed since mappers are typically "global", and while their state does not - change during normal operation, the initial compilation step does modify - internal state significantly, and this step usually occurs not at - module-level initialization time (unless you call compile()) but at - first-request time - - .. change:: - :tags: orm - :tickets: - - basic idea of "session.merge()" actually implemented. needs more testing. - - .. change:: - :tags: orm - :tickets: - - added "compile_mappers()" function as a shortcut to compiling all mappers - - .. change:: - :tags: orm - :tickets: - - fix to MapperExtension create_instance so that entity_name properly - associated with new instance - - .. change:: - :tags: orm - :tickets: - - speed enhancements to ORM object instantiation, eager loading of rows - - .. change:: - :tags: orm - :tickets: 406 - - invalid options sent to 'cascade' string will raise an exception - - .. change:: - :tags: orm - :tickets: 407 - - fixed bug in mapper refresh/expire whereby eager loaders didn't properly - re-populate item lists - - .. change:: - :tags: orm - :tickets: 413 - - fix to post_update to ensure rows are updated even for non insert/delete - scenarios - - .. change:: - :tags: orm - :tickets: 412 - - added an error message if you actually try to modify primary key values on - an entity and then flush it - - .. change:: - :tags: extensions - :tickets: 426 - - added "validate=False" argument to assign_mapper, if True will ensure that - only mapped attributes are named - - .. change:: - :tags: extensions - :tickets: - - assign_mapper gets "options", "instances" functions added (i.e. - MyClass.instances()) - -.. changelog:: - :version: 0.3.3 - :released: Fri Dec 15 2006 - - .. change:: - :tags: - :tickets: - - string-based FROM clauses fixed, i.e. select(..., from_obj=["sometext"]) - - .. change:: - :tags: - :tickets: - - fixes to passive_deletes flag, lazy=None (noload) flag - - .. change:: - :tags: - :tickets: - - added example/docs for dealing with large collections - - .. change:: - :tags: - :tickets: - - added object_session() method to sqlalchemy namespace - - .. change:: - :tags: - :tickets: - - fixed QueuePool bug whereby its better able to reconnect to a database - that was not reachable (thanks to Sébastien Lelong), also fixed dispose() - method - - .. change:: - :tags: - :tickets: 396 - - patch that makes MySQL rowcount work correctly! - - .. change:: - :tags: - :tickets: - - fix to MySQL catch of 2006/2014 errors to properly re-raise OperationalError - exception - -.. changelog:: - :version: 0.3.2 - :released: Sun Dec 10 2006 - - .. change:: - :tags: - :tickets: 387 - - major connection pool bug fixed. fixes MySQL out of sync - errors, will also prevent transactions getting rolled back - accidentally in all DBs - - .. change:: - :tags: - :tickets: - - major speed enhancements vs. 0.3.1, to bring speed - back to 0.2.8 levels - - .. change:: - :tags: - :tickets: - - made conditional dozens of debug log calls that were - time-intensive to generate log messages - - .. change:: - :tags: - :tickets: - - fixed bug in cascade rules whereby the entire object graph - could be unnecessarily cascaded on the save/update cascade - - .. change:: - :tags: - :tickets: - - various speedups in attributes module - - .. change:: - :tags: - :tickets: 388 - - identity map in Session is by default *no longer weak referencing*. - to have it be weak referencing, use create_session(weak_identity_map=True) - fixes - - .. change:: - :tags: - :tickets: - - MySQL detects errors 2006 (server has gone away) and 2014 - (commands out of sync) and invalidates the connection on which it occurred. - - .. change:: - :tags: - :tickets: 307 - - MySQL bool type fix: - - .. change:: - :tags: - :tickets: 382, 349 - - postgres reflection fixes: - - .. change:: - :tags: - :tickets: 247 - - added keywords for EXCEPT, INTERSECT, EXCEPT ALL, INTERSECT ALL - - .. change:: - :tags: - :tickets: 2110 - - assign_mapper in assignmapper extension returns the created mapper - - .. change:: - :tags: - :tickets: - - added label() function to Select class, when scalar=True is used - to create a scalar subquery - i.e. "select x, y, (select max(foo) from table) AS foomax from table" - - .. change:: - :tags: - :tickets: - - added onupdate and ondelete keyword arguments to ForeignKey; propagate - to underlying ForeignKeyConstraint if present. (don't propagate in the - other direction, however) - - .. change:: - :tags: - :tickets: - - fix to session.update() to preserve "dirty" status of incoming object - - .. change:: - :tags: - :tickets: - - sending a selectable to an IN via the in_() function no longer creates - a "union" out of multiple selects; only one selectable to a the in_() function - is allowed now (make a union yourself if union is needed) - - .. change:: - :tags: - :tickets: - - improved support for disabling save-update cascade via cascade="none" etc. - - .. change:: - :tags: - :tickets: - - added "remote_side" argument to relation(), used only with self-referential - mappers to force the direction of the parent/child relationship. replaces - the usage of the "foreignkey" parameter for "switching" the direction. - "foreignkey" argument is deprecated for all uses and will eventually - be replaced by an argument dedicated to ForeignKey specification on mappers. - -.. changelog:: - :version: 0.3.1 - :released: Mon Nov 13 2006 - - .. change:: - :tags: engine/pool - :tickets: - - some new Pool utility classes, updated docs - - .. change:: - :tags: engine/pool - :tickets: - - "use_threadlocal" on Pool defaults to False (same as create_engine) - - .. change:: - :tags: engine/pool - :tickets: - - fixed direct execution of Compiled objects - - .. change:: - :tags: engine/pool - :tickets: - - create_engine() reworked to be strict about incoming \**kwargs. all keyword - arguments must be consumed by one of the dialect, connection pool, and engine - constructors, else a TypeError is thrown which describes the full set of - invalid kwargs in relation to the selected dialect/pool/engine configuration. - - .. change:: - :tags: databases/types - :tickets: - - MySQL catches exception on "describe" and reports as NoSuchTableError - - .. change:: - :tags: databases/types - :tickets: - - further fixes to sqlite booleans, weren't working as defaults - - .. change:: - :tags: databases/types - :tickets: - - fix to postgres sequence quoting when using schemas - - .. change:: - :tags: orm - :tickets: - - the "delete" cascade will load in all child objects, if they were not - loaded already. this can be turned off (i.e. the old behavior) by setting - passive_deletes=True on a relation(). - - .. change:: - :tags: orm - :tickets: - - adjustments to reworked eager query generation to not fail on circular - eager-loaded relationships (like backrefs) - - .. change:: - :tags: orm - :tickets: - - fixed bug where eagerload() (nor lazyload()) option didn't properly - instruct the Query whether or not to use "nesting" when producing a - LIMIT query. - - .. change:: - :tags: orm - :tickets: 360 - - fixed bug in circular dependency sorting at flush time; if object A - contained a cyclical many-to-one relationship to object B, and object B - was just attached to object A, *but* object B itself wasn't changed, - the many-to-one synchronize of B's primary key attribute to A's foreign key - attribute wouldn't occur. - - .. change:: - :tags: orm - :tickets: 325 - - implemented from_obj argument for query.count, improves count function - on selectresults - - .. change:: - :tags: orm - :tickets: - - added an assertion within the "cascade" step of ORM relationships to check - that the class of object attached to a parent object is appropriate - (i.e. if A.items stores B objects, raise an error if a C is appended to A.items) - - .. change:: - :tags: orm - :tickets: - - new extension sqlalchemy.ext.associationproxy, provides transparent - "association object" mappings. new example - examples/association/proxied_association.py illustrates. - - .. change:: - :tags: orm - :tickets: - - improvement to single table inheritance to load full hierarchies beneath - the target class - - .. change:: - :tags: orm - :tickets: 362 - - fix to subtle condition in topological sort where a node could appear twice, - for - - .. change:: - :tags: orm - :tickets: 365 - - additional rework to topological sort, refactoring, for - - .. change:: - :tags: orm - :tickets: - - "delete-orphan" for a certain type can be set on more than one parent class; - the instance is an "orphan" only if its not attached to *any* of those parents - -.. changelog:: - :version: 0.3.0 - :released: Sun Oct 22 2006 - - .. change:: - :tags: general - :tickets: - - logging is now implemented via standard python "logging" module. - "echo" keyword parameters are still functional but set/unset - log levels for their respective classes/instances. all logging - can be controlled directly through the Python API by setting - INFO and DEBUG levels for loggers in the "sqlalchemy" namespace. - class-level logging is under "sqlalchemy..", - instance-level logging under "sqlalchemy...0x..<00-FF>". - Test suite includes "--log-info" and "--log-debug" arguments - which work independently of --verbose/--quiet. Logging added - to orm to allow tracking of mapper configurations, row iteration. - - .. change:: - :tags: general - :tickets: - - the documentation-generation system has been overhauled to be - much simpler in design and more integrated with Markdown - - .. change:: - :tags: sqlite - :tickets: - - sqlite boolean datatype converts False/True to 0/1 by default - - .. change:: - :tags: sqlite - :tickets: 335 - - fixes to Date/Time (SLDate/SLTime) types; works as good as postgres - now - - .. change:: - :tags: ms-sql - :tickets: - - fixes bug 261 (table reflection broken for MS-SQL case-sensitive - databases) - - .. change:: - :tags: ms-sql - :tickets: - - can now specify port for pymssql - - .. change:: - :tags: ms-sql - :tickets: - - introduces new "auto_identity_insert" option for auto-switching - between "SET IDENTITY_INSERT" mode when values specified for IDENTITY columns - - .. change:: - :tags: ms-sql - :tickets: - - now supports multi-column foreign keys - - .. change:: - :tags: ms-sql - :tickets: - - fix to reflecting date/datetime columns - - .. change:: - :tags: ms-sql - :tickets: - - NCHAR and NVARCHAR type support added - - .. change:: - :tags: oracle - :tickets: - - Oracle has experimental support for cx_Oracle.TIMESTAMP, which requires - a setinputsizes() call on the cursor that is now enabled via the - 'auto_setinputsizes' flag to the oracle dialect. - - .. change:: - :tags: firebird - :tickets: - - aliases do not use "AS" - - .. change:: - :tags: firebird - :tickets: - - correctly raises NoSuchTableError when reflecting non-existent table - - .. change:: - :tags: schema - :tickets: - - a fair amount of cleanup to the schema package, removal of ambiguous - methods, methods that are no longer needed. slightly more constrained - usage, greater emphasis on explicitness - - .. change:: - :tags: schema - :tickets: - - the "primary_key" attribute of Table and other selectables becomes - a setlike ColumnCollection object; is ordered but not numerically - indexed. a comparison clause between two pks that are derived from the - same underlying tables (i.e. such as two Alias objects) can be generated - via table1.primary_key==table2.primary_key - - .. change:: - :tags: schema - :tickets: - - ForeignKey(Constraint) supports "use_alter=True", to create/drop a foreign key - via ALTER. this allows circular foreign key relationships to be set up. - - .. change:: - :tags: schema - :tickets: - - append_item() methods removed from Table and Column; preferably - construct Table/Column/related objects inline, but if needed use - append_column(), append_foreign_key(), append_constraint(), etc. - - .. change:: - :tags: schema - :tickets: - - table.create() no longer returns the Table object, instead has no - return value. the usual case is that tables are created via metadata, - which is preferable since it will handle table dependencies. - - .. change:: - :tags: schema - :tickets: - - added UniqueConstraint (goes at Table level), CheckConstraint - (goes at Table or Column level). - - .. change:: - :tags: schema - :tickets: - - index=False/unique=True on Column now creates a UniqueConstraint, - index=True/unique=False creates a plain Index, - index=True/unique=True on Column creates a unique Index. 'index' - and 'unique' keyword arguments to column are now boolean only; for - explicit names and groupings of indexes or unique constraints, use the - UniqueConstraint/Index constructs explicitly. - - .. change:: - :tags: schema - :tickets: - - added autoincrement=True to Column; will disable schema generation - of SERIAL/AUTO_INCREMENT/identity seq for postgres/mysql/mssql if - explicitly set to False - - .. change:: - :tags: schema - :tickets: - - TypeEngine objects now have methods to deal with copying and comparing - values of their specific type. Currently used by the ORM, see below. - - .. change:: - :tags: schema - :tickets: - - fixed condition that occurred during reflection when a primary key - column was explicitly overridden, where the PrimaryKeyConstraint would - get both the reflected and the programmatic column doubled up - - .. change:: - :tags: schema - :tickets: - - the "foreign_key" attribute on Column and ColumnElement in general - is deprecated, in favor of the "foreign_keys" list/set-based attribute, - which takes into account multiple foreign keys on one column. - "foreign_key" will return the first element in the "foreign_keys" list/set - or None if the list is empty. - - .. change:: - :tags: connections/pooling/execution - :tickets: - - connection pool tracks open cursors and automatically closes them - if connection is returned to pool with cursors still opened. Can be - affected by options which cause it to raise an error instead, or to - do nothing. fixes issues with MySQL, others - - .. change:: - :tags: connections/pooling/execution - :tickets: - - fixed bug where Connection wouldn't lose its Transaction - after commit/rollback - - .. change:: - :tags: connections/pooling/execution - :tickets: - - added scalar() method to ComposedSQLEngine, ResultProxy - - .. change:: - :tags: connections/pooling/execution - :tickets: - - ResultProxy will close() the underlying cursor when the ResultProxy - itself is closed. this will auto-close cursors for ResultProxy objects - that have had all their rows fetched (or had scalar() called). - - .. change:: - :tags: connections/pooling/execution - :tickets: - - ResultProxy.fetchall() internally uses DBAPI fetchall() for better efficiency, - added to mapper iteration as well (courtesy Michael Twomey) - - .. change:: - :tags: construction, sql - :tickets: 292 - - changed "for_update" parameter to accept False/True/"nowait" - and "read", the latter two of which are interpreted only by - Oracle and MySQL - - .. change:: - :tags: construction, sql - :tickets: - - added extract() function to sql dialect - (SELECT extract(field FROM expr)) - - .. change:: - :tags: construction, sql - :tickets: - - BooleanExpression includes new "negate" argument to specify - the appropriate negation operator if one is available. - - .. change:: - :tags: construction, sql - :tickets: - - calling a negation on an "IN" or "IS" clause will result in - "NOT IN", "IS NOT" (as opposed to NOT (x IN y)). - - .. change:: - :tags: construction, sql - :tickets: 172 - - Function objects know what to do in a FROM clause now. their - behavior should be the same, except now you can also do things like - select(['*'], from_obj=[func.my_function()]) to get multiple - columns from the result, or even use sql.column() constructs to name the - return columns - - .. change:: - :tags: orm - :tickets: - - attribute tracking modified to be more intelligent about detecting - changes, particularly with mutable types. TypeEngine objects now - take a greater role in defining how to compare two scalar instances, - including the addition of a MutableType mixin which is implemented by - PickleType. unit-of-work now tracks the "dirty" list as an expression - of all persistent objects where the attribute manager detects changes. - The basic issue that's fixed is detecting changes on PickleType - objects, but also generalizes type handling and "modified" object - checking to be more complete and extensible. - - .. change:: - :tags: orm - :tickets: - - a wide refactoring to "attribute loader" and "options" architectures. - ColumnProperty and PropertyLoader define their loading behavior via switchable - "strategies", and MapperOptions no longer use mapper/property copying - in order to function; they are instead propagated via QueryContext - and SelectionContext objects at query/instances time. - All of the internal copying of mappers and properties that was used to handle - inheritance as well as options() has been removed; the structure - of mappers and properties is much simpler than before and is clearly laid out - in the new 'interfaces' module. - - .. change:: - :tags: orm - :tickets: - - related to the mapper/property overhaul, internal refactoring to - mapper instances() method to use a SelectionContext object to track - state during the operation. - SLIGHT API BREAKAGE: the append_result() and populate_instances() - methods on MapperExtension have a slightly different method signature - now as a result of the change; hoping that these methods are not - in widespread use as of yet. - - .. change:: - :tags: orm - :tickets: - - instances() method moved to Query now, backwards-compatible - version remains on Mapper. - - .. change:: - :tags: orm - :tickets: - - added contains_eager() MapperOption, used in conjunction with - instances() to specify properties that should be eagerly loaded - from the result set, using their plain column names by default, or translated - given an custom row-translation function. - - .. change:: - :tags: orm - :tickets: - - more rearrangements of unit-of-work commit scheme to better allow - dependencies within circular flushes to work properly...updated - task traversal/logging implementation - - .. change:: - :tags: orm - :tickets: 321 - - polymorphic mappers (i.e. using inheritance) now produces INSERT - statements in order of tables across all inherited classes - - .. change:: - :tags: orm - :tickets: - - added an automatic "row switch" feature to mapping, which will - detect a pending instance/deleted instance pair with the same - identity key and convert the INSERT/DELETE to a single UPDATE - - .. change:: - :tags: orm - :tickets: - - "association" mappings simplified to take advantage of - automatic "row switch" feature - - .. change:: - :tags: orm - :tickets: 212 - - "custom list classes" is now implemented via the "collection_class" - keyword argument to relation(). the old way still works but is - deprecated - - .. change:: - :tags: orm - :tickets: - - added "viewonly" flag to relation(), allows construction of - relations that have no effect on the flush() process. - - .. change:: - :tags: orm - :tickets: 292 - - added "lockmode" argument to base Query select/get functions, - including "with_lockmode" function to get a Query copy that has - a default locking mode. Will translate "read"/"update" - arguments into a for_update argument on the select side. - - .. change:: - :tags: orm - :tickets: - - implemented "version check" logic in Query/Mapper, used - when version_id_col is in effect and query.with_lockmode() - is used to get() an instance that's already loaded - - .. change:: - :tags: orm - :tickets: 208 - - post_update behavior improved; does a better job at not - updating too many rows, updates only required columns - - .. change:: - :tags: orm - :tickets: 308 - - adjustments to eager loading so that its "eager chain" is - kept separate from the normal mapper setup, thereby - preventing conflicts with lazy loader operation, fixes - - .. change:: - :tags: orm - :tickets: - - fix to deferred group loading - - .. change:: - :tags: orm - :tickets: 346 - - session.flush() won't close a connection it opened - - .. change:: - :tags: orm - :tickets: - - added "batch=True" flag to mapper; if False, save_obj - will fully save one object at a time including calls - to before_XXXX and after_XXXX - - .. change:: - :tags: orm - :tickets: - - added "column_prefix=None" argument to mapper; prepends the - given string (typically '_') to column-based attributes automatically - set up from the mapper's Table - - .. change:: - :tags: orm - :tickets: 315 - - specifying joins in the from_obj argument of query.select() will - replace the main table of the query, if the table is somewhere within - the given from_obj. this makes it possible to produce custom joins and - outerjoins in queries without the main table getting added twice. - - .. change:: - :tags: orm - :tickets: - - eagerloading is adjusted to more thoughtfully attach its LEFT OUTER JOINs - to the given query, looking for custom "FROM" clauses that may have - already been set up. - - .. change:: - :tags: orm - :tickets: - - added join_to and outerjoin_to transformative methods to SelectResults, - to build up join/outerjoin conditions based on property names. also - added select_from to explicitly set from_obj parameter. - - .. change:: - :tags: orm - :tickets: - - removed "is_primary" flag from mapper. diff --git a/doc/build/changelog/changelog_04.rst b/doc/build/changelog/changelog_04.rst deleted file mode 100644 index b0312b0921a..00000000000 --- a/doc/build/changelog/changelog_04.rst +++ /dev/null @@ -1,4197 +0,0 @@ - -============= -0.4 Changelog -============= - - -.. changelog:: - :version: 0.4.8 - :released: Sun Oct 12 2008 - - .. change:: - :tags: orm - :tickets: 1039 - - Fixed bug regarding inherit_condition passed - with "A=B" versus "B=A" leading to errors - - .. change:: - :tags: orm - :tickets: - - Changes made to new, dirty and deleted - collections in - SessionExtension.before_flush() will take - effect for that flush. - - .. change:: - :tags: orm - :tickets: - - Added label() method to InstrumentedAttribute - to establish forwards compatibility with 0.5. - - .. change:: - :tags: sql - :tickets: 1074 - - column.in_(someselect) can now be used as - a columns-clause expression without the subquery - bleeding into the FROM clause - - .. change:: - :tags: mysql - :tickets: 1146 - - Added MSMediumInteger type. - - .. change:: - :tags: sqlite - :tickets: 968 - - Supplied a custom strftime() function which - handles dates before 1900. - - .. change:: - :tags: sqlite - :tickets: - - String's (and Unicode's, UnicodeText's, etc.) - convert_unicode logic disabled in the sqlite dialect, - to adjust for pysqlite 2.5.0's new requirement that - only Python unicode objects are accepted; - https://itsystementwicklung.de/pipermail/list-pysqlite/2008-March/000018.html - - .. change:: - :tags: oracle - :tickets: 1155 - - has_sequence() now takes schema name into account - - .. change:: - :tags: oracle - :tickets: 1121 - - added BFILE to the list of reflected types - -.. changelog:: - :version: 0.4.7p1 - :released: Thu Jul 31 2008 - - .. change:: - :tags: orm - :tickets: - - Added "add()" and "add_all()" to scoped_session - methods. Workaround for 0.4.7:: - - from sqlalchemy.orm.scoping import ScopedSession, instrument - - setattr(ScopedSession, "add", instrument("add")) - setattr(ScopedSession, "add_all", instrument("add_all")) - - .. change:: - :tags: orm - :tickets: - - Fixed non-2.3 compatible usage of set() and generator - expression within relation(). - -.. changelog:: - :version: 0.4.7 - :released: Sat Jul 26 2008 - - .. change:: - :tags: orm - :tickets: 1058 - - The contains() operator when used with many-to-many - will alias() the secondary (association) table so - that multiple contains() calls will not conflict - with each other - - .. change:: - :tags: orm - :tickets: - - fixed bug preventing merge() from functioning in - conjunction with a comparable_property() - - .. change:: - :tags: orm - :tickets: - - the enable_typechecks=False setting on relation() - now only allows subtypes with inheriting mappers. - Totally unrelated types, or subtypes not set up with - mapper inheritance against the target mapper are - still not allowed. - - .. change:: - :tags: orm - :tickets: 976 - - Added is_active flag to Sessions to detect when - a transaction is in progress. This - flag is always True with a "transactional" - (in 0.5 a non-"autocommit") Session. - - .. change:: - :tags: sql - :tickets: - - Fixed bug when calling select([literal('foo')]) - or select([bindparam('foo')]). - - .. change:: - :tags: schema - :tickets: 571 - - create_all(), drop_all(), create(), drop() all raise - an error if the table name or schema name contains - more characters than that dialect's configured - character limit. Some DB's can handle too-long - table names during usage, and SQLA can handle this - as well. But various reflection/ - checkfirst-during-create scenarios fail since we are - looking for the name within the DB's catalog tables. - - .. change:: - :tags: schema - :tickets: 571, 820 - - The index name generated when you say "index=True" - on a Column is truncated to the length appropriate - for the dialect. Additionally, an Index with a too- - long name cannot be explicitly dropped with - Index.drop(), similar to. - - .. change:: - :tags: postgres - :tickets: - - Repaired server_side_cursors to properly detect - text() clauses. - - .. change:: - :tags: postgres - :tickets: 1092 - - Added PGCidr type. - - .. change:: - :tags: mysql - :tickets: - - Added 'CALL' to the list of SQL keywords which return - result rows. - - .. change:: - :tags: oracle - :tickets: - - Oracle get_default_schema_name() "normalizes" the name - before returning, meaning it returns a lower-case name - when the identifier is detected as case insensitive. - - .. change:: - :tags: oracle - :tickets: 709 - - creating/dropping tables takes schema name into account - when searching for the existing table, so that tables - in other owner namespaces with the same name do not - conflict - - .. change:: - :tags: oracle - :tickets: 1062 - - Cursors now have "arraysize" set to 50 by default on - them, the value of which is configurable using the - "arraysize" argument to create_engine() with the - Oracle dialect. This to account for cx_oracle's default - setting of "1", which has the effect of many round trips - being sent to Oracle. This actually works well in - conjunction with BLOB/CLOB-bound cursors, of which - there are any number available but only for the life of - that row request (so BufferedColumnRow is still needed, - but less so). - - .. change:: - :tags: oracle - :tickets: - - sqlite - - add SLFloat type, which matches the SQLite REAL - type affinity. Previously, only SLNumeric was provided - which fulfills NUMERIC affinity, but that's not the - same as REAL. - -.. changelog:: - :version: 0.4.6 - :released: Sat May 10 2008 - - .. change:: - :tags: orm - :tickets: - - Fix to the recent relation() refactoring which fixes - exotic viewonly relations which join between local and - remote table multiple times, with a common column shared - between the joins. - - .. change:: - :tags: orm - :tickets: - - Also re-established viewonly relation() configurations - that join across multiple tables. - - .. change:: - :tags: orm - :tickets: 610 - - Added experimental relation() flag to help with - primaryjoins across functions, etc., - _local_remote_pairs=[tuples]. This complements a complex - primaryjoin condition allowing you to provide the - individual column pairs which comprise the relation's - local and remote sides. Also improved lazy load SQL - generation to handle placing bind params inside of - functions and other expressions. (partial progress - towards) - - .. change:: - :tags: orm - :tickets: 1036 - - repaired single table inheritance such that you - can single-table inherit from a joined-table inheriting - mapper without issue. - - .. change:: - :tags: orm - :tickets: 1027 - - Fixed "concatenate tuple" bug which could occur with - Query.order_by() if clause adaption had taken place. - - .. change:: - :tags: orm - :tickets: - - Removed ancient assertion that mapped selectables require - "alias names" - the mapper creates its own alias now if - none is present. Though in this case you need to use the - class, not the mapped selectable, as the source of column - attributes - so a warning is still issued. - - .. change:: - :tags: orm - :tickets: - - fixes to the "exists" function involving inheritance (any(), - has(), ~contains()); the full target join will be rendered - into the EXISTS clause for relations that link to subclasses. - - .. change:: - :tags: orm - :tickets: - - restored usage of append_result() extension method for primary - query rows, when the extension is present and only a single- - entity result is being returned. - - .. change:: - :tags: orm - :tickets: - - Also re-established viewonly relation() configurations that - join across multiple tables. - - .. change:: - :tags: orm - :tickets: - - removed ancient assertion that mapped selectables require - "alias names" - the mapper creates its own alias now if - none is present. Though in this case you need to use - the class, not the mapped selectable, as the source of - column attributes - so a warning is still issued. - - .. change:: - :tags: orm - :tickets: 1015 - - refined mapper._save_obj() which was unnecessarily calling - __ne__() on scalar values during flush - - .. change:: - :tags: orm - :tickets: 1019 - - added a feature to eager loading whereby subqueries set - as column_property() with explicit label names (which is not - necessary, btw) will have the label anonymized when - the instance is part of the eager join, to prevent - conflicts with a subquery or column of the same name - on the parent object. - - .. change:: - :tags: orm - :tickets: - - set-based collections \|=, -=, ^= and &= are stricter about - their operands and only operate on sets, frozensets or - subclasses of the collection type. Previously, they would - accept any duck-typed set. - - .. change:: - :tags: orm - :tickets: - - added an example dynamic_dict/dynamic_dict.py, illustrating - a simple way to place dictionary behavior on top of - a dynamic_loader. - - .. change:: - :tags: declarative, extension - :tickets: - - Joined table inheritance mappers use a slightly relaxed - function to create the "inherit condition" to the parent - table, so that other foreign keys to not-yet-declared - Table objects don't trigger an error. - - .. change:: - :tags: declarative, extension - :tickets: - - fixed reentrant mapper compile hang when - a declared attribute is used within ForeignKey, - ie. ForeignKey(MyOtherClass.someattribute) - - .. change:: - :tags: sql - :tickets: - - Added COLLATE support via the .collate() - expression operator and collate(, ) sql - function. - - .. change:: - :tags: sql - :tickets: - - Fixed bug with union() when applied to non-Table connected - select statements - - .. change:: - :tags: sql - :tickets: 1014 - - improved behavior of text() expressions when used as - FROM clauses, such as select().select_from(text("sometext")) - - .. change:: - :tags: sql - :tickets: 1021 - - Column.copy() respects the value of "autoincrement", - fixes usage with Migrate - - .. change:: - :tags: engines - :tickets: - - Pool listeners can now be provided as a dictionary of - callables or a (possibly partial) duck-type of - PoolListener, your choice. - - .. change:: - :tags: engines - :tickets: - - added "rollback_returned" option to Pool which will - disable the rollback() issued when connections are - returned. This flag is only safe to use with a database - which does not support transactions (i.e. MySQL/MyISAM). - - .. change:: - :tags: ext - :tickets: - - set-based association proxies \|=, -=, ^= and &= are - stricter about their operands and only operate on sets, - frozensets or other association proxies. Previously, they - would accept any duck-typed set. - - .. change:: - :tags: mssql - :tickets: 1005 - - Added "odbc_autotranslate" parameter to engine / dburi - parameters. Any given string will be passed through to the - ODBC connection string as: - - "AutoTranslate=%s" % odbc_autotranslate - - .. change:: - :tags: mssql - :tickets: - - Added "odbc_options" parameter to engine / dburi - parameters. The given string is simply appended to the - SQLAlchemy-generated odbc connection string. - - This should obviate the need of adding a myriad of ODBC - options in the future. - - .. change:: - :tags: firebird - :tickets: - - Handle the "SUBSTRING(:string FROM :start FOR :length)" - builtin. - -.. changelog:: - :version: 0.4.5 - :released: Fri Apr 04 2008 - - .. change:: - :tags: orm - :tickets: - - A small change in behavior to session.merge() - existing - objects are checked for based on primary key attributes, not - necessarily _instance_key. So the widely requested - capability, that: - - x = MyObject(id=1) - x = sess.merge(x) - - will in fact load MyObject with id #1 from the database if - present, is now available. merge() still copies the state - of the given object to the persistent one, so an example - like the above would typically have copied "None" from all - attributes of "x" onto the persistent copy. These can be - reverted using session.expire(x). - - .. change:: - :tags: orm - :tickets: - - Also fixed behavior in merge() whereby collection elements - present on the destination but not the merged collection - were not being removed from the destination. - - .. change:: - :tags: orm - :tickets: 995 - - Added a more aggressive check for "uncompiled mappers", - helps particularly with declarative layer - - .. change:: - :tags: orm - :tickets: - - The methodology behind "primaryjoin"/"secondaryjoin" has - been refactored. Behavior should be slightly more - intelligent, primarily in terms of error messages which - have been pared down to be more readable. In a slight - number of scenarios it can better resolve the correct - foreign key than before. - - .. change:: - :tags: orm - :tickets: - - Added comparable_property(), adds query Comparator - behavior to regular, unmanaged Python properties - - .. change:: - :tags: orm, Company.employees.of_type(Engineer), 'machines' - :tickets: - - the functionality of query.with_polymorphic() has - been added to mapper() as a configuration option. - - It's set via several forms: - with_polymorphic='*' - with_polymorphic=[mappers] - with_polymorphic=('*', selectable) - with_polymorphic=([mappers], selectable) - - This controls the default polymorphic loading strategy - for inherited mappers. When a selectable is not given, - outer joins are created for all joined-table inheriting - mappers requested. Note that the auto-create of joins - is not compatible with concrete table inheritance. - - The existing select_table flag on mapper() is now - deprecated and is synonymous with - with_polymorphic('*', select_table). Note that the - underlying "guts" of select_table have been - completely removed and replaced with the newer, - more flexible approach. - - The new approach also automatically allows eager loads - to work for subclasses, if they are present, for - example:: - - sess.query(Company).options(eagerload_all()) - - to load Company objects, their employees, and the - 'machines' collection of employees who happen to be - Engineers. A "with_polymorphic" Query option should be - introduced soon as well which would allow per-Query - control of with_polymorphic() on relations. - - .. change:: - :tags: orm - :tickets: - - added two "experimental" features to Query, - "experimental" in that their specific name/behavior - is not carved in stone just yet: _values() and - _from_self(). We'd like feedback on these. - - - _values(\*columns) is given a list of column - expressions, and returns a new Query that only - returns those columns. When evaluated, the return - value is a list of tuples just like when using - add_column() or add_entity(), the only difference is - that "entity zero", i.e. the mapped class, is not - included in the results. This means it finally makes - sense to use group_by() and having() on Query, which - have been sitting around uselessly until now. - - A future change to this method may include that its - ability to join, filter and allow other options not - related to a "resultset" are removed, so the feedback - we're looking for is how people want to use - _values()...i.e. at the very end, or do people prefer - to continue generating after it's called. - - - _from_self() compiles the SELECT statement for the - Query (minus any eager loaders), and returns a new - Query that selects from that SELECT. So basically you - can query from a Query without needing to extract the - SELECT statement manually. This gives meaning to - operations like query[3:5]._from_self().filter(some - criterion). There's not much controversial here - except that you can quickly create highly nested - queries that are less efficient, and we want feedback - on the naming choice. - - .. change:: - :tags: orm - :tickets: - - query.order_by() and query.group_by() will accept - multiple arguments using \*args (like select() - already does). - - .. change:: - :tags: orm - :tickets: - - Added some convenience descriptors to Query: - query.statement returns the full SELECT construct, - query.whereclause returns just the WHERE part of the - SELECT construct. - - .. change:: - :tags: orm - :tickets: - - Fixed/covered case when using a False/0 value as a - polymorphic discriminator. - - .. change:: - :tags: orm - :tickets: - - Fixed bug which was preventing synonym() attributes from - being used with inheritance - - .. change:: - :tags: orm - :tickets: 996 - - Fixed SQL function truncation of trailing underscores - - .. change:: - :tags: orm - :tickets: - - When attributes are expired on a pending instance, an - error will not be raised when the "refresh" action is - triggered and no result is found. - - .. change:: - :tags: orm - :tickets: - - Session.execute can now find binds from metadata - - .. change:: - :tags: orm - :tickets: - - Adjusted the definition of "self-referential" to be any - two mappers with a common parent (this affects whether or - not aliased=True is required when joining with Query). - - .. change:: - :tags: orm - :tickets: - - Made some fixes to the "from_joinpoint" argument to - query.join() so that if the previous join was aliased and - this one isn't, the join still happens successfully. - - .. change:: - :tags: orm - :tickets: 895 - - Assorted "cascade deletes" fixes: - - Fixed "cascade delete" operation of dynamic relations, - which had only been implemented for foreign-key - nulling behavior in 0.4.2 and not actual cascading - deletes - - - Delete cascade without delete-orphan cascade on a - many-to-one will not delete orphans which were - disconnected from the parent before session.delete() - is called on the parent (one-to-many already had - this). - - - Delete cascade with delete-orphan will delete orphans - whether or not it remains attached to its also-deleted - parent. - - - delete-orphan cascade is properly detected on relations - that are present on superclasses when using inheritance. - - .. change:: - :tags: orm - :tickets: - - Fixed order_by calculation in Query to properly alias - mapper-config'ed order_by when using select_from() - - .. change:: - :tags: orm - :tickets: - - Refactored the diffing logic that kicks in when replacing - one collection with another into collections.bulk_replace, - useful to anyone building multi-level collections. - - .. change:: - :tags: orm - :tickets: - - Cascade traversal algorithm converted from recursive to - iterative to support deep object graphs. - - .. change:: - :tags: sql - :tickets: 999 - - schema-qualified tables now will place the schemaname - ahead of the tablename in all column expressions as well - as when generating column labels. This prevents cross- - schema name collisions in all cases - - .. change:: - :tags: sql - :tickets: - - can now allow selects which correlate all FROM clauses - and have no FROM themselves. These are typically - used in a scalar context, i.e. SELECT x, (SELECT x WHERE y) - FROM table. Requires explicit correlate() call. - - .. change:: - :tags: sql - :tickets: - - 'name' is no longer a required constructor argument for - Column(). It (and .key) may now be deferred until the - column is added to a Table. - - .. change:: - :tags: sql - :tickets: 791, 993 - - like(), ilike(), contains(), startswith(), endswith() take - an optional keyword argument "escape=", which - is set as the escape character using the syntax "x LIKE y - ESCAPE ''". - - .. change:: - :tags: sql - :tickets: - - random() is now a generic sql function and will compile to - the database's random implementation, if any. - - .. change:: - :tags: sql - :tickets: - - update().values() and insert().values() take keyword - arguments. - - .. change:: - :tags: sql - :tickets: - - Fixed an issue in select() regarding its generation of - FROM clauses, in rare circumstances two clauses could be - produced when one was intended to cancel out the other. - Some ORM queries with lots of eager loads might have seen - this symptom. - - .. change:: - :tags: sql - :tickets: - - The case() function now also takes a dictionary as its - whens parameter. It also interprets the "THEN" - expressions as values by default, meaning case([(x==y, - "foo")]) will interpret "foo" as a bound value, not a SQL - expression. use text(expr) for literal SQL expressions in - this case. For the criterion itself, these may be literal - strings only if the "value" keyword is present, otherwise - SA will force explicit usage of either text() or - literal(). - - .. change:: - :tags: oracle - :tickets: - - The "owner" keyword on Table is now deprecated, and is - exactly synonymous with the "schema" keyword. Tables can - now be reflected with alternate "owner" attributes, - explicitly stated on the Table object or not using - "schema". - - .. change:: - :tags: oracle - :tickets: - - All of the "magic" searching for synonyms, DBLINKs etc. - during table reflection are disabled by default unless you - specify "oracle_resolve_synonyms=True" on the Table - object. Resolving synonyms necessarily leads to some - messy guessing which we'd rather leave off by default. - When the flag is set, tables and related tables will be - resolved against synonyms in all cases, meaning if a - synonym exists for a particular table, reflection will use - it when reflecting related tables. This is stickier - behavior than before which is why it's off by default. - - .. change:: - :tags: declarative, extension - :tickets: - - The "synonym" function is now directly usable with - "declarative". Pass in the decorated property using the - "descriptor" keyword argument, e.g.: somekey = - synonym('_somekey', descriptor=property(g, s)) - - .. change:: - :tags: declarative, extension - :tickets: - - The "deferred" function is usable with "declarative". - Simplest usage is to declare deferred and Column together, - e.g.: data = deferred(Column(Text)) - - .. change:: - :tags: declarative, extension - :tickets: - - Declarative also gained @synonym_for(...) and - @comparable_using(...), front-ends for synonym and - comparable_property. - - .. change:: - :tags: declarative, extension - :tickets: 995 - - Improvements to mapper compilation when using declarative; - already-compiled mappers will still trigger compiles of - other uncompiled mappers when used - - .. change:: - :tags: declarative, extension - :tickets: - - Declarative will complete setup for Columns lacking names, - allows a more DRY syntax. - - class Foo(Base): - __tablename__ = 'foos' - id = Column(Integer, primary_key=True) - - .. change:: - :tags: declarative, extension - :tickets: - - inheritance in declarative can be disabled when sending - "inherits=None" to __mapper_args__. - - .. change:: - :tags: declarative, extension - :tickets: - - declarative_base() takes optional kwarg "mapper", which - is any callable/class/method that produces a mapper, - such as declarative_base(mapper=scopedsession.mapper). - This property can also be set on individual declarative - classes using the "__mapper_cls__" property. - - .. change:: - :tags: postgres - :tickets: 1001 - - Got PG server side cursors back into shape, added fixed - unit tests as part of the default test suite. Added - better uniqueness to the cursor ID - - .. change:: - :tags: oracle - :tickets: - - The "owner" keyword on Table is now deprecated, and is - exactly synonymous with the "schema" keyword. Tables can - now be reflected with alternate "owner" attributes, - explicitly stated on the Table object or not using - "schema". - - .. change:: - :tags: oracle - :tickets: - - All of the "magic" searching for synonyms, DBLINKs etc. - during table reflection are disabled by default unless you - specify "oracle_resolve_synonyms=True" on the Table - object. Resolving synonyms necessarily leads to some - messy guessing which we'd rather leave off by default. - When the flag is set, tables and related tables will be - resolved against synonyms in all cases, meaning if a - synonym exists for a particular table, reflection will use - it when reflecting related tables. This is stickier - behavior than before which is why it's off by default. - - .. change:: - :tags: mssql - :tickets: 979 - - Reflected tables will now automatically load other tables - which are referenced by Foreign keys in the auto-loaded - table,. - - .. change:: - :tags: mssql - :tickets: 916 - - Added executemany check to skip identity fetch,. - - .. change:: - :tags: mssql - :tickets: 884 - - Added stubs for small date type. - - .. change:: - :tags: mssql - :tickets: - - Added a new 'driver' keyword parameter for the pyodbc dialect. - Will substitute into the ODBC connection string if given, - defaults to 'SQL Server'. - - .. change:: - :tags: mssql - :tickets: - - Added a new 'max_identifier_length' keyword parameter for - the pyodbc dialect. - - .. change:: - :tags: mssql - :tickets: - - Improvements to pyodbc + Unix. If you couldn't get that - combination to work before, please try again. - - .. change:: - :tags: mysql - :tickets: - - The connection.info keys the dialect uses to cache server - settings have changed and are now namespaced. - -.. changelog:: - :version: 0.4.4 - :released: Wed Mar 12 2008 - - .. change:: - :tags: sql - :tickets: 975 - - Can again create aliases of selects against textual FROM - clauses. - - .. change:: - :tags: sql - :tickets: - - The value of a bindparam() can be a callable, in which - case it's evaluated at statement execution time to get the - value. - - .. change:: - :tags: sql - :tickets: 978 - - Added exception wrapping/reconnect support to result set - fetching. Reconnect works for those databases that raise - a catchable data error during results (i.e. doesn't work - on MySQL) - - .. change:: - :tags: sql - :tickets: 936 - - Implemented two-phase API for "threadlocal" engine, via - engine.begin_twophase(), engine.prepare() - - .. change:: - :tags: sql - :tickets: 986 - - Fixed bug which was preventing UNIONS from being - cloneable. - - .. change:: - :tags: sql - :tickets: - - Added "bind" keyword argument to insert(), update(), - delete() and DDL(). The .bind property is now assignable - on those statements as well as on select(). - - .. change:: - :tags: sql - :tickets: - - Insert statements can now be compiled with extra "prefix" - words between INSERT and INTO, for vendor extensions like - MySQL's INSERT IGNORE INTO table. - - .. change:: - :tags: orm - :tickets: - - any(), has(), contains(), ~contains(), attribute level == - and != now work properly with self-referential relations - - the clause inside the EXISTS is aliased on the "remote" - side to distinguish it from the parent table. This - applies to single table self-referential as well as - inheritance-based self-referential. - - .. change:: - :tags: orm - :tickets: 985 - - Repaired behavior of == and != operators at the relation() - level when compared against NULL for one-to-one relations - - .. change:: - :tags: orm - :tickets: - - Fixed bug whereby session.expire() attributes were not - loading on an polymorphically-mapped instance mapped by a - select_table mapper. - - .. change:: - :tags: orm - :tickets: - - Added query.with_polymorphic() - specifies a list of - classes which descend from the base class, which will be - added to the FROM clause of the query. Allows subclasses - to be used within filter() criterion as well as eagerly - loads the attributes of those subclasses. - - .. change:: - :tags: orm - :tickets: - - Your cries have been heard: removing a pending item from - an attribute or collection with delete-orphan expunges the - item from the session; no FlushError is raised. Note that - if you session.save()'ed the pending item explicitly, the - attribute/collection removal still knocks it out. - - .. change:: - :tags: orm - :tickets: - - session.refresh() and session.expire() raise an error when - called on instances which are not persistent within the - session - - .. change:: - :tags: orm - :tickets: - - Fixed potential generative bug when the same Query was - used to generate multiple Query objects using join(). - - .. change:: - :tags: orm - :tickets: - - Fixed bug which was introduced in 0.4.3, whereby loading - an already-persistent instance mapped with joined table - inheritance would trigger a useless "secondary" load from - its joined table, when using the default "select" - polymorphic_fetch. This was due to attributes being - marked as expired during its first load and not getting - unmarked from the previous "secondary" load. Attributes - are now unexpired based on presence in __dict__ after any - load or commit operation succeeds. - - .. change:: - :tags: orm - :tickets: - - Deprecated Query methods apply_sum(), apply_max(), - apply_min(), apply_avg(). Better methodologies are - coming.... - - .. change:: - :tags: orm - :tickets: - - relation() can accept a callable for its first argument, - which returns the class to be related. This is in place - to assist declarative packages to define relations without - classes yet being in place. - - .. change:: - :tags: orm - :tickets: - - Added a new "higher level" operator called "of_type()": - used in join() as well as with any() and has(), qualifies - the subclass which will be used in filter criterion, e.g.: - - query.filter(Company.employees.of_type(Engineer). - any(Engineer.name=='foo')) - - or - - query.join(Company.employees.of_type(Engineer)). - filter(Engineer.name=='foo') - - .. change:: - :tags: orm - :tickets: - - Preventive code against a potential lost-reference bug in - flush(). - - .. change:: - :tags: orm - :tickets: - - Expressions used in filter(), filter_by() and others, when - they make usage of a clause generated from a relation - using the identity of a child object (e.g., - filter(Parent.child==)), evaluate the actual - primary key value of at execution time so that - the autoflush step of the Query can complete, thereby - populating the PK value of in the case that - was pending. - - .. change:: - :tags: orm - :tickets: - - setting the relation()-level order by to a column in the - many-to-many "secondary" table will now work with eager - loading, previously the "order by" wasn't aliased against - the secondary table's alias. - - .. change:: - :tags: orm - :tickets: - - Synonyms riding on top of existing descriptors are now - full proxies to those descriptors. - - .. change:: - :tags: dialects - :tickets: - - Invalid SQLite connection URLs now raise an error. - - .. change:: - :tags: dialects - :tickets: 981 - - postgres TIMESTAMP renders correctly - - .. change:: - :tags: dialects - :tickets: - - postgres PGArray is a "mutable" type by default; when used - with the ORM, mutable-style equality/ copy-on-write - techniques are used to test for changes. - - .. change:: - :tags: extensions - :tickets: - - a new super-small "declarative" extension has been added, - which allows Table and mapper() configuration to take - place inline underneath a class declaration. This - extension differs from ActiveMapper and Elixir in that it - does not redefine any SQLAlchemy semantics at all; literal - Column, Table and relation() constructs are used to define - the class behavior and table definition. - -.. changelog:: - :version: 0.4.3 - :released: Thu Feb 14 2008 - - .. change:: - :tags: sql - :tickets: - - Added "schema.DDL", an executable free-form DDL statement. - DDLs can be executed in isolation or attached to Table or - MetaData instances and executed automatically when those - objects are created and/or dropped. - - .. change:: - :tags: sql - :tickets: - - Table columns and constraints can be overridden on a an - existing table (such as a table that was already reflected) - using the 'useexisting=True' flag, which now takes into - account the arguments passed along with it. - - .. change:: - :tags: sql - :tickets: - - Added a callable-based DDL events interface, adds hooks - before and after Tables and MetaData create and drop. - - .. change:: - :tags: sql - :tickets: - - Added generative where() method to delete() and - update() constructs which return a new object with criterion - joined to existing criterion via AND, just like - select().where(). - - .. change:: - :tags: sql - :tickets: 727 - - Added "ilike()" operator to column operations. Compiles to - ILIKE on postgres, lower(x) LIKE lower(y) on all - others. - - .. change:: - :tags: sql - :tickets: 943 - - Added "now()" as a generic function; on SQLite, Oracle - and MSSQL compiles as "CURRENT_TIMESTAMP"; "now()" on - all others. - - .. change:: - :tags: sql - :tickets: 962 - - The startswith(), endswith(), and contains() operators now - concatenate the wildcard operator with the given operand in - SQL, i.e. "'%' || " in all cases, accept - text('something') operands properly - - .. change:: - :tags: sql - :tickets: 962 - - cast() accepts text('something') and other non-literal - operands properly - - .. change:: - :tags: sql - :tickets: - - fixed bug in result proxy where anonymously generated - column labels would not be accessible using their straight - string name - - .. change:: - :tags: sql - :tickets: - - Deferrable constraints can now be defined. - - .. change:: - :tags: sql - :tickets: 915 - - Added "autocommit=True" keyword argument to select() and - text(), as well as generative autocommit() method on - select(); for statements which modify the database through - some user-defined means other than the usual INSERT/UPDATE/ - DELETE etc. This flag will enable "autocommit" behavior - during execution if no transaction is in progress. - - .. change:: - :tags: sql - :tickets: - - The '.c.' attribute on a selectable now gets an entry for - every column expression in its columns clause. Previously, - "unnamed" columns like functions and CASE statements weren't - getting put there. Now they will, using their full string - representation if no 'name' is available. - - .. change:: - :tags: sql - :tickets: - - a CompositeSelect, i.e. any union(), union_all(), - intersect(), etc. now asserts that each selectable contains - the same number of columns. This conforms to the - corresponding SQL requirement. - - .. change:: - :tags: sql - :tickets: - - The anonymous 'label' generated for otherwise unlabeled - functions and expressions now propagates outwards at compile - time for expressions like select([select([func.foo()])]). - - .. change:: - :tags: sql - :tickets: - - Building on the above ideas, CompositeSelects now build up - their ".c." collection based on the names present in the - first selectable only; corresponding_column() now works - fully for all embedded selectables. - - .. change:: - :tags: sql - :tickets: - - Oracle and others properly encode SQL used for defaults like - sequences, etc., even if no unicode idents are used since - identifier preparer may return a cached unicode identifier. - - .. change:: - :tags: sql - :tickets: - - Column and clause comparisons to datetime objects on the - left hand side of the expression now work (d < table.c.col). - (datetimes on the RHS have always worked, the LHS exception - is a quirk of the datetime implementation.) - - .. change:: - :tags: orm - :tickets: - - Every Session.begin() must now be accompanied by a - corresponding commit() or rollback() unless the session is - closed with Session.close(). This also includes the begin() - which is implicit to a session created with - transactional=True. The biggest change introduced here is - that when a Session created with transactional=True raises - an exception during flush(), you must call - Session.rollback() or Session.close() in order for that - Session to continue after an exception. - - .. change:: - :tags: orm - :tickets: 961 - - Fixed merge() collection-doubling bug when merging transient - entities with backref'ed collections. - - .. change:: - :tags: orm - :tickets: - - merge(dont_load=True) does not accept transient entities, - this is in continuation with the fact that - merge(dont_load=True) does not accept any "dirty" objects - either. - - .. change:: - :tags: orm - :tickets: - - Added standalone "query" class attribute generated by a - scoped_session. This provides MyClass.query without using - Session.mapper. Use via: - - MyClass.query = Session.query_property() - - .. change:: - :tags: orm - :tickets: - - The proper error message is raised when trying to access - expired instance attributes with no session present - - .. change:: - :tags: orm - :tickets: - - dynamic_loader() / lazy="dynamic" now accepts and uses - the order_by parameter in the same way in which it works - with relation(). - - .. change:: - :tags: orm - :tickets: - - Added expire_all() method to Session. Calls expire() for - all persistent instances. This is handy in conjunction - with... - - .. change:: - :tags: orm - :tickets: - - Instances which have been partially or fully expired will - have their expired attributes populated during a regular - Query operation which affects those objects, preventing a - needless second SQL statement for each instance. - - .. change:: - :tags: orm - :tickets: 938 - - Dynamic relations, when referenced, create a strong - reference to the parent object so that the query still has a - parent to call against even if the parent is only created - (and otherwise dereferenced) within the scope of a single - expression. - - .. change:: - :tags: orm - :tickets: - - Added a mapper() flag "eager_defaults". When set to True, - defaults that are generated during an INSERT or UPDATE - operation are post-fetched immediately, instead of being - deferred until later. This mimics the old 0.3 behavior. - - .. change:: - :tags: orm - :tickets: - - query.join() can now accept class-mapped attributes as - arguments. These can be used in place or in any combination - with strings. In particular this allows construction of - joins to subclasses on a polymorphic relation, i.e.: - - query(Company).join(['employees', Engineer.name]) - - .. change:: - :tags: orm, ('employees', people.join(engineer)), Engineer.name - :tickets: - - query.join() can also accept tuples of attribute name/some - selectable as arguments. This allows construction of joins - *from* subclasses of a polymorphic relation, i.e.: - - query(Company).\ - join( - - ) - - .. change:: - :tags: orm - :tickets: - - General improvements to the behavior of join() in - conjunction with polymorphic mappers, i.e. joining from/to - polymorphic mappers and properly applying aliases. - - .. change:: - :tags: orm - :tickets: 933 - - Fixed/improved behavior when a mapper determines the natural - "primary key" of a mapped join, it will more effectively - reduce columns which are equivalent via foreign key - relation. This affects how many arguments need to be sent - to query.get(), among other things. - - .. change:: - :tags: orm - :tickets: 946 - - The lazy loader can now handle a join condition where the - "bound" column (i.e. the one that gets the parent id sent as - a bind parameter) appears more than once in the join - condition. Specifically this allows the common task of a - relation() which contains a parent-correlated subquery, such - as "select only the most recent child item". - - .. change:: - :tags: orm - :tickets: - - Fixed bug in polymorphic inheritance where an incorrect - exception is raised when base polymorphic_on column does not - correspond to any columns within the local selectable of an - inheriting mapper more than one level deep - - .. change:: - :tags: orm - :tickets: - - Fixed bug in polymorphic inheritance which made it difficult - to set a working "order_by" on a polymorphic mapper. - - .. change:: - :tags: orm - :tickets: - - Fixed a rather expensive call in Query that was slowing down - polymorphic queries. - - .. change:: - :tags: orm - :tickets: 954 - - "Passive defaults" and other "inline" defaults can now be - loaded during a flush() call if needed; in particular, this - allows constructing relations() where a foreign key column - references a server-side-generated, non-primary-key - column. - - .. change:: - :tags: orm - :tickets: - - Additional Session transaction fixes/changes: - - Fixed bug with session transaction management: parent - transactions weren't started on the connection when - adding a connection to a nested transaction. - - - session.transaction now always refers to the innermost - active transaction, even when commit/rollback are called - directly on the session transaction object. - - - Two-phase transactions can now be prepared. - - - When preparing a two-phase transaction fails on one - connection, all the connections are rolled back. - - - session.close() didn't close all transactions when - nested transactions were used. - - - rollback() previously erroneously set the current - transaction directly to the parent of the transaction - that could be rolled back to. Now it rolls back the next - transaction up that can handle it, but sets the current - transaction to its parent and inactivates the - transactions in between. Inactive transactions can only - be rolled back or closed, any other call results in an - error. - - - autoflush for commit() wasn't flushing for simple - subtransactions. - - - unitofwork flush didn't close the failed transaction - when the session was not in a transaction and committing - the transaction failed. - - .. change:: - :tags: orm - :tickets: 964, 940 - - Miscellaneous tickets: - - .. change:: - :tags: general - :tickets: - - Fixed a variety of hidden and some not-so-hidden - compatibility issues for Python 2.3, thanks to new support - for running the full test suite on 2.3. - - .. change:: - :tags: general - :tickets: - - Warnings are now issued as type exceptions.SAWarning. - - .. change:: - :tags: dialects - :tickets: - - Better support for schemas in SQLite (linked in by ATTACH - DATABASE ... AS name). In some cases in the past, schema - names were omitted from generated SQL for SQLite. This is - no longer the case. - - .. change:: - :tags: dialects - :tickets: - - table_names on SQLite now picks up temporary tables as well. - - .. change:: - :tags: dialects - :tickets: - - Auto-detect an unspecified MySQL ANSI_QUOTES mode during - reflection operations, support for changing the mode - midstream. Manual mode setting is still required if no - reflection is used. - - .. change:: - :tags: dialects - :tickets: - - Fixed reflection of TIME columns on SQLite. - - .. change:: - :tags: dialects - :tickets: 580 - - Finally added PGMacAddr type to postgres - - .. change:: - :tags: dialects - :tickets: - - Reflect the sequence associated to a PK field (typically - with a BEFORE INSERT trigger) under Firebird - - .. change:: - :tags: dialects - :tickets: 941 - - Oracle assembles the correct columns in the result set - column mapping when generating a LIMIT/OFFSET subquery, - allows columns to map properly to result sets even if - long-name truncation kicks in - - .. change:: - :tags: dialects - :tickets: - - MSSQL now includes EXEC in the _is_select regexp, which - should allow row-returning stored procedures to be used. - - .. change:: - :tags: dialects - :tickets: - - MSSQL now includes an experimental implementation of - LIMIT/OFFSET using the ANSI SQL row_number() function, so it - requires MSSQL-2005 or higher. To enable the feature, add - "has_window_funcs" to the keyword arguments for connect, or - add "?has_window_funcs=1" to your dburi query arguments. - - .. change:: - :tags: ext - :tickets: - - Changed ext.activemapper to use a non-transactional session - for the objectstore. - - .. change:: - :tags: ext - :tickets: - - Fixed output order of "['a'] + obj.proxied" binary operation - on association-proxied lists. - -.. changelog:: - :version: 0.4.2p3 - :released: Wed Jan 09 2008 - - .. change:: - :tags: general - :tickets: - - sub version numbering scheme changed to suite - setuptools version number rules; easy_install -u - should now get this version over 0.4.2. - - .. change:: - :tags: sql - :tickets: 912 - - Text type is properly exported now and does not - raise a warning on DDL create; String types with no - length only raise warnings during CREATE TABLE - - .. change:: - :tags: sql - :tickets: - - new UnicodeText type is added, to specify an - encoded, unlengthed Text type - - .. change:: - :tags: sql - :tickets: - - fixed bug in union() so that select() statements - which don't derive from FromClause objects can be - unioned - - .. change:: - :tags: orm - :tickets: - - fixed bug with session.dirty when using "mutable - scalars" (such as PickleTypes) - - .. change:: - :tags: orm - :tickets: - - added a more descriptive error message when flushing - on a relation() that has non-locally-mapped columns - in its primary or secondary join condition - - .. change:: - :tags: dialects - :tickets: - - Fixed reflection of mysql empty string column - defaults. - - .. change:: - :tags: sql - :tickets: 912 - - changed name of TEXT to Text since its a "generic" - type; TEXT name is deprecated until 0.5. The - "upgrading" behavior of String to Text when no - length is present is also deprecated until 0.5; will - issue a warning when used for CREATE TABLE - statements (String with no length for SQL expression - purposes is still fine) - - .. change:: - :tags: sql - :tickets: 924 - - generative select.order_by(None) / group_by(None) - was not managing to reset order by/group by - criterion, fixed - - .. change:: - :tags: orm - :tickets: - - suppressing *all* errors in - InstanceState.__cleanup() now. - - .. change:: - :tags: orm - :tickets: 922 - - fixed an attribute history bug whereby assigning a - new collection to a collection-based attribute which - already had pending changes would generate incorrect - history - - .. change:: - :tags: orm - :tickets: 925 - - fixed delete-orphan cascade bug whereby setting the - same object twice to a scalar attribute could log it - as an orphan - - .. change:: - :tags: orm - :tickets: - - Fixed cascades on a += assignment to a list-based - relation. - - .. change:: - :tags: orm - :tickets: 919 - - synonyms can now be created against props that don't - exist yet, which are later added via add_property(). - This commonly includes backrefs. (i.e. you can make - synonyms for backrefs without worrying about the - order of operations) - - .. change:: - :tags: orm - :tickets: - - fixed bug which could occur with polymorphic "union" - mapper which falls back to "deferred" loading of - inheriting tables - - .. change:: - :tags: orm - :tickets: - - the "columns" collection on a mapper/mapped class - (i.e. 'c') is against the mapped table, not the - select_table in the case of polymorphic "union" - loading (this shouldn't be noticeable). - - .. change:: - :tags: ext - :tickets: - - '+', '*', '+=' and '\*=' support for association - proxied lists. - - .. change:: - :tags: dialects - :tickets: 923 - - mssql - narrowed down the test for "date"/"datetime" - in MSDate/ MSDateTime subclasses so that incoming - "datetime" objects don't get mis-interpreted as - "date" objects and vice versa. - - .. change:: - :tags: orm - :tickets: - - fixed fairly critical bug whereby the same instance could be listed - more than once in the unitofwork.new collection; most typically - reproduced when using a combination of inheriting mappers and - ScopedSession.mapper, as the multiple __init__ calls per instance - could save() the object with distinct _state objects - - .. change:: - :tags: orm - :tickets: - - added very rudimentary yielding iterator behavior to Query. Call - query.yield_per() and evaluate the Query in an - iterative context; every collection of N rows will be packaged up - and yielded. Use this method with extreme caution since it does - not attempt to reconcile eagerly loaded collections across - result batch boundaries, nor will it behave nicely if the same - instance occurs in more than one batch. This means that an eagerly - loaded collection will get cleared out if it's referenced in more than - one batch, and in all cases attributes will be overwritten on instances - that occur in more than one batch. - - .. change:: - :tags: orm - :tickets: 920 - - Fixed in-place set mutation operators for set collections and association - proxied sets. - - .. change:: - :tags: dialects - :tickets: 913 - - Fixed the missing call to subtype result processor for the PGArray - type. - -.. changelog:: - :version: 0.4.2 - :released: Wed Jan 02 2008 - - .. change:: - :tags: sql - :tickets: 615 - - generic functions ! we introduce a database of known SQL functions, such - as current_timestamp, coalesce, and create explicit function objects - representing them. These objects have constrained argument lists, are - type aware, and can compile in a dialect-specific fashion. So saying - func.char_length("foo", "bar") raises an error (too many args), - func.coalesce(datetime.date(2007, 10, 5), datetime.date(2005, 10, 15)) - knows that its return type is a Date. We only have a few functions - represented so far but will continue to add to the system - - .. change:: - :tags: sql - :tickets: - - auto-reconnect support improved; a Connection can now automatically - reconnect after its underlying connection is invalidated, without - needing to connect() again from the engine. This allows an ORM session - bound to a single Connection to not need a reconnect. - Open transactions on the Connection must be rolled back after an invalidation - of the underlying connection else an error is raised. Also fixed - bug where disconnect detect was not being called for cursor(), rollback(), - or commit(). - - .. change:: - :tags: sql - :tickets: - - added new flag to String and create_engine(), - assert_unicode=(True|False|'warn'\|None). Defaults to `False` or `None` on - create_engine() and String, `'warn'` on the Unicode type. When `True`, - results in all unicode conversion operations raising an exception when a - non-unicode bytestring is passed as a bind parameter. 'warn' results - in a warning. It is strongly advised that all unicode-aware applications - make proper use of Python unicode objects (i.e. u'hello' and not 'hello') - so that data round trips accurately. - - .. change:: - :tags: sql - :tickets: - - generation of "unique" bind parameters has been simplified to use the same - "unique identifier" mechanisms as everything else. This doesn't affect - user code, except any code that might have been hardcoded against the generated - names. Generated bind params now have the form "_", - whereas before only the second bind of the same name would have this form. - - .. change:: - :tags: sql - :tickets: - - select().as_scalar() will raise an exception if the select does not have - exactly one expression in its columns clause. - - .. change:: - :tags: sql - :tickets: - - bindparam() objects themselves can be used as keys for execute(), i.e. - statement.execute({bind1:'foo', bind2:'bar'}) - - .. change:: - :tags: sql - :tickets: - - added new methods to TypeDecorator, process_bind_param() and - process_result_value(), which automatically take advantage of the processing - of the underlying type. Ideal for using with Unicode or Pickletype. - TypeDecorator should now be the primary way to augment the behavior of any - existing type including other TypeDecorator subclasses such as PickleType. - - .. change:: - :tags: sql - :tickets: - - selectables (and others) will issue a warning when two columns in - their exported columns collection conflict based on name. - - .. change:: - :tags: sql - :tickets: 890 - - tables with schemas can still be used in sqlite, firebird, - schema name just gets dropped - - .. change:: - :tags: sql - :tickets: - - changed the various "literal" generation functions to use an anonymous - bind parameter. not much changes here except their labels now look - like ":param_1", ":param_2" instead of ":literal" - - .. change:: - :tags: sql - :tickets: - - column labels in the form "tablename.columname", i.e. with a dot, are now - supported. - - .. change:: - :tags: sql - :tickets: - - from_obj keyword argument to select() can be a scalar or a list. - - .. change:: - :tags: orm - :tickets: 871 - - a major behavioral change to collection-based backrefs: they no - longer trigger lazy loads ! "reverse" adds and removes - are queued up and are merged with the collection when it is - actually read from and loaded; but do not trigger a load beforehand. - For users who have noticed this behavior, this should be much more - convenient than using dynamic relations in some cases; for those who - have not, you might notice your apps using a lot fewer queries than - before in some situations. - - .. change:: - :tags: orm - :tickets: - - mutable primary key support is added. primary key columns can be - changed freely, and the identity of the instance will change upon - flush. In addition, update cascades of foreign key referents (primary - key or not) along relations are supported, either in tandem with the - database's ON UPDATE CASCADE (required for DB's like Postgres) or - issued directly by the ORM in the form of UPDATE statements, by setting - the flag "passive_cascades=False". - - .. change:: - :tags: orm - :tickets: 490 - - inheriting mappers now inherit the MapperExtensions of their parent - mapper directly, so that all methods for a particular MapperExtension - are called for subclasses as well. As always, any MapperExtension - can return either EXT_CONTINUE to continue extension processing - or EXT_STOP to stop processing. The order of mapper resolution is: - . - - Note that if you instantiate the same extension class separately - and then apply it individually for two mappers in the same inheritance - chain, the extension will be applied twice to the inheriting class, - and each method will be called twice. - - To apply a mapper extension explicitly to each inheriting class but - have each method called only once per operation, use the same - instance of the extension for both mappers. - - .. change:: - :tags: orm - :tickets: 907 - - MapperExtension.before_update() and after_update() are now called - symmetrically; previously, an instance that had no modified column - attributes (but had a relation() modification) could be called with - before_update() but not after_update() - - .. change:: - :tags: orm - :tickets: - - columns which are missing from a Query's select statement - now get automatically deferred during load. - - .. change:: - :tags: orm - :tickets: 908 - - mapped classes which extend "object" and do not provide an - __init__() method will now raise TypeError if non-empty \*args - or \**kwargs are present at instance construction time (and are - not consumed by any extensions such as the scoped_session mapper), - consistent with the behavior of normal Python classes - - .. change:: - :tags: orm - :tickets: 899 - - fixed Query bug when filter_by() compares a relation against None - - .. change:: - :tags: orm - :tickets: - - improved support for pickling of mapped entities. Per-instance - lazy/deferred/expired callables are now serializable so that - they serialize and deserialize with _state. - - .. change:: - :tags: orm - :tickets: 801 - - new synonym() behavior: an attribute will be placed on the mapped - class, if one does not exist already, in all cases. if a property - already exists on the class, the synonym will decorate the property - with the appropriate comparison operators so that it can be used in - column expressions just like any other mapped attribute (i.e. usable in - filter(), etc.) the "proxy=True" flag is deprecated and no longer means - anything. Additionally, the flag "map_column=True" will automatically - generate a ColumnProperty corresponding to the name of the synonym, - i.e.: 'somename':synonym('_somename', map_column=True) will map the - column named 'somename' to the attribute '_somename'. See the example - in the mapper docs. - - .. change:: - :tags: orm - :tickets: - - Query.select_from() now replaces all existing FROM criterion with - the given argument; the previous behavior of constructing a list - of FROM clauses was generally not useful as is required - filter() calls to create join criterion, and new tables introduced - within filter() already add themselves to the FROM clause. The - new behavior allows not just joins from the main table, but select - statements as well. Filter criterion, order bys, eager load - clauses will be "aliased" against the given statement. - - .. change:: - :tags: orm - :tickets: - - this month's refactoring of attribute instrumentation changes - the "copy-on-load" behavior we've had since midway through 0.3 - with "copy-on-modify" in most cases. This takes a sizable chunk - of latency out of load operations and overall does less work - as only attributes which are actually modified get their - "committed state" copied. Only "mutable scalar" attributes - (i.e. a pickled object or other mutable item), the reason for - the copy-on-load change in the first place, retain the old - behavior. - - .. change:: - :tags: attrname, orm - :tickets: - - a slight behavioral change to attributes is, del'ing an attribute - does *not* cause the lazyloader of that attribute to fire off again; - the "del" makes the effective value of the attribute "None". To - re-trigger the "loader" for an attribute, use - session.expire(instance,). - - .. change:: - :tags: orm - :tickets: - - query.filter(SomeClass.somechild == None), when comparing - a many-to-one property to None, properly generates "id IS NULL" - including that the NULL is on the right side. - - .. change:: - :tags: orm - :tickets: - - query.order_by() takes into account aliased joins, i.e. - query.join('orders', aliased=True).order_by(Order.id) - - .. change:: - :tags: orm - :tickets: - - eagerload(), lazyload(), eagerload_all() take an optional - second class-or-mapper argument, which will select the mapper - to apply the option towards. This can select among other - mappers which were added using add_entity(). - - .. change:: - :tags: orm - :tickets: - - eagerloading will work with mappers added via add_entity(). - - .. change:: - :tags: orm - :tickets: - - added "cascade delete" behavior to "dynamic" relations just like - that of regular relations. if passive_deletes flag (also just added) - is not set, a delete of the parent item will trigger a full load of - the child items so that they can be deleted or updated accordingly. - - .. change:: - :tags: orm - :tickets: - - also with dynamic, implemented correct count() behavior as well - as other helper methods. - - .. change:: - :tags: orm - :tickets: - - fix to cascades on polymorphic relations, such that cascades - from an object to a polymorphic collection continue cascading - along the set of attributes specific to each element in the collection. - - .. change:: - :tags: orm - :tickets: 893 - - query.get() and query.load() do not take existing filter or other - criterion into account; these methods *always* look up the given id - in the database or return the current instance from the identity map, - disregarding any existing filter, join, group_by or other criterion - which has been configured. - - .. change:: - :tags: orm - :tickets: 883 - - added support for version_id_col in conjunction with inheriting mappers. - version_id_col is typically set on the base mapper in an inheritance - relationship where it takes effect for all inheriting mappers. - - .. change:: - :tags: orm - :tickets: - - relaxed rules on column_property() expressions having labels; any - ColumnElement is accepted now, as the compiler auto-labels non-labeled - ColumnElements now. a selectable, like a select() statement, still - requires conversion to ColumnElement via as_scalar() or label(). - - .. change:: - :tags: orm - :tickets: - - fixed backref bug where you could not del instance.attr if attr - was None - - .. change:: - :tags: orm - :tickets: - - several ORM attributes have been removed or made private: - mapper.get_attr_by_column(), mapper.set_attr_by_column(), - mapper.pks_by_table, mapper.cascade_callable(), - MapperProperty.cascade_callable(), mapper.canload(), - mapper.save_obj(), mapper.delete_obj(), mapper._mapper_registry, - attributes.AttributeManager - - .. change:: - :tags: orm - :tickets: - - Assigning an incompatible collection type to a relation attribute now - raises TypeError instead of sqlalchemy's ArgumentError. - - .. change:: - :tags: orm - :tickets: 886 - - Bulk assignment of a MappedCollection now raises an error if a key in the - incoming dictionary does not match the key that the collection's keyfunc - would use for that value. - - .. change:: - :tags: orm, newval1, newval2 - :tickets: - - Custom collections can now specify a @converter method to translate - objects used in "bulk" assignment into a stream of values, as in: - - .. sourcecode:: text - - obj.col = - # or - obj.dictcol = {'foo': newval1, 'bar': newval2} - - The MappedCollection uses this hook to ensure that incoming key/value - pairs are sane from the collection's perspective. - - .. change:: - :tags: orm - :tickets: 872 - - fixed endless loop issue when using lazy="dynamic" on both - sides of a bi-directional relationship - - .. change:: - :tags: orm - :tickets: 904 - - more fixes to the LIMIT/OFFSET aliasing applied with Query + eagerloads, - in this case when mapped against a select statement - - .. change:: - :tags: orm - :tickets: - - fix to self-referential eager loading such that if the same mapped - instance appears in two or more distinct sets of columns in the same - result set, its eagerly loaded collection will be populated regardless - of whether or not all of the rows contain a set of "eager" columns for - that collection. this would also show up as a KeyError when fetching - results with join_depth turned on. - - .. change:: - :tags: orm - :tickets: - - fixed bug where Query would not apply a subquery to the SQL when LIMIT - was used in conjunction with an inheriting mapper where the eager - loader was only in the parent mapper. - - .. change:: - :tags: orm - :tickets: - - clarified the error message which occurs when you try to update() - an instance with the same identity key as an instance already present - in the session. - - .. change:: - :tags: orm - :tickets: - - some clarifications and fixes to merge(instance, dont_load=True). - fixed bug where lazy loaders were getting disabled on returned instances. - Also, we currently do not support merging an instance which has uncommitted - changes on it, in the case that dont_load=True is used....this will - now raise an error. This is due to complexities in merging the - "committed state" of the given instance to correctly correspond to the - newly copied instance, as well as other modified state. - Since the use case for dont_load=True is caching, the given instances - shouldn't have any uncommitted changes on them anyway. - We also copy the instances over without using any events now, so that - the 'dirty' list on the new session remains unaffected. - - .. change:: - :tags: orm - :tickets: - - fixed bug which could arise when using session.begin_nested() in conjunction - with more than one level deep of enclosing session.begin() statements - - .. change:: - :tags: orm - :tickets: 914 - - fixed session.refresh() with instance that has custom entity_name - - .. change:: - :tags: dialects - :tickets: - - sqlite SLDate type will not erroneously render "microseconds" portion - of a datetime or time object. - - .. change:: - :tags: dialects - :tickets: 902 - - oracle - - added disconnect detection support for Oracle - - some cleanup to binary/raw types so that cx_oracle.LOB is detected - on an ad-hoc basis - - .. change:: - :tags: dialects - :tickets: 824, 839, 842, 901 - - MSSQL - - PyODBC no longer has a global "set nocount on". - - Fix non-identity integer PKs on autoload - - Better support for convert_unicode - - Less strict date conversion for pyodbc/adodbapi - - Schema-qualified tables / autoload - - .. change:: - :tags: firebird, backend - :tickets: 410 - - does properly reflect domains (partially fixing) and - PassiveDefaults - - .. change:: - :tags: 3562, firebird, backend - :tickets: - - reverted to use default poolclass (was set to SingletonThreadPool in - 0.4.0 for test purposes) - - .. change:: - :tags: firebird, backend - :tickets: - - map func.length() to 'char_length' (easily overridable with the UDF - 'strlen' on old versions of Firebird) - -.. changelog:: - :version: 0.4.1 - :released: Sun Nov 18 2007 - - .. change:: - :tags: sql - :tickets: - - the "shortname" keyword parameter on bindparam() has been - deprecated. - - .. change:: - :tags: sql - :tickets: - - Added contains operator (generates a "LIKE %%" clause). - - .. change:: - :tags: sql - :tickets: - - anonymous column expressions are automatically labeled. - e.g. select([x* 5]) produces "SELECT x * 5 AS anon_1". - This allows the labelname to be present in the cursor.description - which can then be appropriately matched to result-column processing - rules. (we can't reliably use positional tracking for result-column - matches since text() expressions may represent multiple columns). - - .. change:: - :tags: sql - :tickets: - - operator overloading is now controlled by TypeEngine objects - the - one built-in operator overload so far is String types overloading - '+' to be the string concatenation operator. - User-defined types can also define their own operator overloading - by overriding the adapt_operator(self, op) method. - - .. change:: - :tags: sql - :tickets: 819 - - untyped bind parameters on the right side of a binary expression - will be assigned the type of the left side of the operation, to better - enable the appropriate bind parameter processing to take effect - - .. change:: - :tags: sql - :tickets: 833 - - Removed regular expression step from most statement compilations. - Also fixes - - .. change:: - :tags: sql - :tickets: - - Fixed empty (zero column) sqlite inserts, allowing inserts on - autoincrementing single column tables. - - .. change:: - :tags: sql - :tickets: - - Fixed expression translation of text() clauses; this repairs various - ORM scenarios where literal text is used for SQL expressions - - .. change:: - :tags: sql - :tickets: - - Removed ClauseParameters object; compiled.params returns a regular - dictionary now, as well as result.last_inserted_params() / - last_updated_params(). - - .. change:: - :tags: sql - :tickets: - - Fixed INSERT statements w.r.t. primary key columns that have - SQL-expression based default generators on them; SQL expression - executes inline as normal but will not trigger a "postfetch" condition - for the column, for those DB's who provide it via cursor.lastrowid - - .. change:: - :tags: sql - :tickets: 844 - - func. objects can be pickled/unpickled - - .. change:: - :tags: sql - :tickets: - - rewrote and simplified the system used to "target" columns across - selectable expressions. On the SQL side this is represented by the - "corresponding_column()" method. This method is used heavily by the ORM - to "adapt" elements of an expression to similar, aliased expressions, - as well as to target result set columns originally bound to a - table or selectable to an aliased, "corresponding" expression. The new - rewrite features completely consistent and accurate behavior. - - .. change:: - :tags: sql - :tickets: 573 - - Added a field ("info") for storing arbitrary data on schema items - - .. change:: - :tags: sql - :tickets: - - The "properties" collection on Connections has been renamed "info" to - match schema's writable collections. Access is still available via - the "properties" name until 0.5. - - .. change:: - :tags: sql - :tickets: - - fixed the close() method on Transaction when using strategy='threadlocal' - - .. change:: - :tags: sql - :tickets: 853 - - fix to compiled bind parameters to not mistakenly populate None - - .. change:: - :tags: sql - :tickets: - - ._execute_clauseelement becomes a public method - Connectable.execute_clauseelement - - .. change:: - :tags: orm - :tickets: 843 - - eager loading with LIMIT/OFFSET applied no longer adds the primary - table joined to a limited subquery of itself; the eager loads now - join directly to the subquery which also provides the primary table's - columns to the result set. This eliminates a JOIN from all eager loads - with LIMIT/OFFSET. - - .. change:: - :tags: orm - :tickets: 802 - - session.refresh() and session.expire() now support an additional argument - "attribute_names", a list of individual attribute keynames to be refreshed - or expired, allowing partial reloads of attributes on an already-loaded - instance. - - .. change:: - :tags: orm - :tickets: 767 - - added op() operator to instrumented attributes; i.e. - User.name.op('ilike')('%somename%') - - .. change:: - :tags: orm - :tickets: 676 - - Mapped classes may now define __eq__, __hash__, and __nonzero__ methods - with arbitrary semantics. The orm now handles all mapped instances on - an identity-only basis. (e.g. 'is' vs '==') - - .. change:: - :tags: orm - :tickets: - - the "properties" accessor on Mapper is removed; it now throws an informative - exception explaining the usage of mapper.get_property() and - mapper.iterate_properties - - .. change:: - :tags: orm - :tickets: - - added having() method to Query, applies HAVING to the generated statement - in the same way as filter() appends to the WHERE clause. - - .. change:: - :tags: orm - :tickets: 777 - - The behavior of query.options() is now fully based on paths, i.e. an - option such as eagerload_all('x.y.z.y.x') will apply eagerloading to - only those paths, i.e. and not 'x.y.x'; eagerload('children.children') - applies only to exactly two-levels deep, etc. - - .. change:: - :tags: orm - :tickets: - - PickleType will compare using `==` when set up with mutable=False, - and not the `is` operator. To use `is` or any other comparator, send - in a custom comparison function using PickleType(comparator=my_custom_comparator). - - .. change:: - :tags: orm - :tickets: 848 - - query doesn't throw an error if you use distinct() and an order_by() - containing UnaryExpressions (or other) together - - .. change:: - :tags: orm - :tickets: 786 - - order_by() expressions from joined tables are properly added to columns - clause when using distinct() - - .. change:: - :tags: orm - :tickets: 858 - - fixed error where Query.add_column() would not accept a class-bound - attribute as an argument; Query also raises an error if an invalid - argument was sent to add_column() (at instances() time) - - .. change:: - :tags: orm - :tickets: - - added a little more checking for garbage-collection dereferences in - InstanceState.__cleanup() to reduce "gc ignored" errors on app - shutdown - - .. change:: - :tags: orm - :tickets: - - The session API has been solidified: - - .. change:: - :tags: orm - :tickets: 840 - - It's an error to session.save() an object which is already - persistent - - .. change:: - :tags: orm - :tickets: - - It's an error to session.delete() an object which is *not* - persistent. - - .. change:: - :tags: orm - :tickets: - - session.update() and session.delete() raise an error when updating - or deleting an instance that is already in the session with a - different identity. - - .. change:: - :tags: orm - :tickets: - - The session checks more carefully when determining "object X already - in another session"; e.g. if you pickle a series of objects and - unpickle (i.e. as in a Pylons HTTP session or similar), they can go - into a new session without any conflict - - .. change:: - :tags: orm - :tickets: - - merge() includes a keyword argument "dont_load=True". setting this - flag will cause the merge operation to not load any data from the - database in response to incoming detached objects, and will accept - the incoming detached object as though it were already present in - that session. Use this to merge detached objects from external - caching systems into the session. - - .. change:: - :tags: orm - :tickets: - - Deferred column attributes no longer trigger a load operation when the - attribute is assigned to. In those cases, the newly assigned value - will be present in the flushes' UPDATE statement unconditionally. - - .. change:: - :tags: orm - :tickets: 834 - - Fixed a truncation error when re-assigning a subset of a collection - (obj.relation = obj.relation[1:]) - - .. change:: - :tags: orm - :tickets: 832 - - De-cruftified backref configuration code, backrefs which step on - existing properties now raise an error - - .. change:: - :tags: orm - :tickets: 831 - - Improved behavior of add_property() etc., fixed involving - synonym/deferred. - - .. change:: - :tags: orm - :tickets: - - Fixed clear_mappers() behavior to better clean up after itself. - - .. change:: - :tags: orm - :tickets: 841 - - Fix to "row switch" behavior, i.e. when an INSERT/DELETE is combined - into a single UPDATE; many-to-many relations on the parent object - update properly. - - .. change:: - :tags: orm - :tickets: - - Fixed __hash__ for association proxy- these collections are unhashable, - just like their mutable Python counterparts. - - .. change:: - :tags: orm - :tickets: - - Added proxying of save_or_update, __contains__ and __iter__ methods for - scoped sessions. - - .. change:: - :tags: orm - :tickets: 852 - - fixed very hard-to-reproduce issue where by the FROM clause of Query - could get polluted by certain generative calls - - .. change:: - :tags: dialects - :tickets: - - Added experimental support for MaxDB (versions >= 7.6.03.007 only). - - .. change:: - :tags: dialects - :tickets: - - oracle will now reflect "DATE" as an OracleDateTime column, not - OracleDate - - .. change:: - :tags: dialects - :tickets: 847 - - added awareness of schema name in oracle table_names() function, - fixes metadata.reflect(schema='someschema') - - .. change:: - :tags: dialects - :tickets: - - MSSQL anonymous labels for selection of functions made deterministic - - .. change:: - :tags: dialects - :tickets: - - sqlite will reflect "DECIMAL" as a numeric column. - - .. change:: - :tags: dialects - :tickets: 828 - - Made access dao detection more reliable - - .. change:: - :tags: dialects - :tickets: - - Renamed the Dialect attribute 'preexecute_sequences' to - 'preexecute_pk_sequences'. An attribute proxy is in place for - out-of-tree dialects using the old name. - - .. change:: - :tags: dialects - :tickets: - - Added test coverage for unknown type reflection. Fixed sqlite/mysql - handling of type reflection for unknown types. - - .. change:: - :tags: dialects - :tickets: - - Added REAL for mysql dialect (for folks exploiting the - REAL_AS_FLOAT sql mode). - - .. change:: - :tags: dialects - :tickets: - - mysql Float, MSFloat and MSDouble constructed without arguments - now produce no-argument DDL, e.g.'FLOAT'. - - .. change:: - :tags: misc - :tickets: - - Removed unused util.hash(). - -.. changelog:: - :version: 0.4.0 - :released: Wed Oct 17 2007 - - .. change:: - :tags: - :tickets: - - (see 0.4.0beta1 for the start of major changes against 0.3, - as well as https://www.sqlalchemy.org/trac/wiki/WhatsNewIn04 ) - - .. change:: - :tags: - :tickets: 785 - - Added initial Sybase support (mxODBC so far) - - .. change:: - :tags: - :tickets: - - Added partial index support for PostgreSQL. Use the postgres_where keyword - on the Index. - - .. change:: - :tags: - :tickets: 817 - - string-based query param parsing/config file parser understands - wider range of string values for booleans - - .. change:: - :tags: - :tickets: 813 - - backref remove object operation doesn't fail if the other-side - collection doesn't contain the item, supports noload collections - - .. change:: - :tags: - :tickets: 818 - - removed __len__ from "dynamic" collection as it would require issuing - a SQL "count()" operation, thus forcing all list evaluations to issue - redundant SQL - - .. change:: - :tags: - :tickets: 816 - - inline optimizations added to locate_dirty() which can greatly speed up - repeated calls to flush(), as occurs with autoflush=True - - .. change:: - :tags: - :tickets: - - The IdentifierPreprarer's _requires_quotes test is now regex based. Any - out-of-tree dialects that provide custom sets of legal_characters or - illegal_initial_characters will need to move to regexes or override - _requires_quotes. - - .. change:: - :tags: - :tickets: - - Firebird has supports_sane_rowcount and supports_sane_multi_rowcount set - to False due to ticket #370 (right way). - - .. change:: - :tags: - :tickets: - - Improvements and fixes on Firebird reflection: - * FBDialect now mimics OracleDialect, regarding case-sensitivity of TABLE and - COLUMN names (see 'case_sensitive remotion' topic on this current file). - * FBDialect.table_names() doesn't bring system tables (ticket:796). - * FB now reflects Column's nullable property correctly. - - .. change:: - :tags: - :tickets: - - Fixed SQL compiler's awareness of top-level column labels as used - in result-set processing; nested selects which contain the same column - names don't affect the result or conflict with result-column metadata. - - .. change:: - :tags: - :tickets: - - query.get() and related functions (like many-to-one lazyloading) - use compile-time-aliased bind parameter names, to prevent - name conflicts with bind parameters that already exist in the - mapped selectable. - - .. change:: - :tags: - :tickets: 795 - - Fixed three- and multi-level select and deferred inheritance loading - (i.e. abc inheritance with no select_table). - - .. change:: - :tags: - :tickets: - - Ident passed to id_chooser in shard.py always a list. - - .. change:: - :tags: - :tickets: - - The no-arg ResultProxy._row_processor() is now the class attribute - `_process_row`. - - .. change:: - :tags: - :tickets: 797 - - Added support for returning values from inserts and updates for - PostgreSQL 8.2+. - - .. change:: - :tags: - :tickets: - - PG reflection, upon seeing the default schema name being used explicitly - as the "schema" argument in a Table, will assume that this is the - user's desired convention, and will explicitly set the "schema" argument - in foreign-key-related reflected tables, thus making them match only - with Table constructors that also use the explicit "schema" argument - (even though its the default schema). - In other words, SA assumes the user is being consistent in this usage. - - .. change:: - :tags: - :tickets: 808 - - fixed sqlite reflection of BOOL/BOOLEAN - - .. change:: - :tags: - :tickets: - - Added support for UPDATE with LIMIT on mysql. - - .. change:: - :tags: - :tickets: 803 - - null foreign key on a m2o doesn't trigger a lazyload - - .. change:: - :tags: - :tickets: 800 - - oracle does not implicitly convert to unicode for non-typed result - sets (i.e. when no TypeEngine/String/Unicode type is even being used; - previously it was detecting DBAPI types and converting regardless). - should fix - - .. change:: - :tags: - :tickets: 806 - - fix to anonymous label generation of long table/column names - - .. change:: - :tags: - :tickets: - - Firebird dialect now uses SingletonThreadPool as poolclass. - - .. change:: - :tags: - :tickets: - - Firebird now uses dialect.preparer to format sequences names - - .. change:: - :tags: - :tickets: 810 - - Fixed breakage with postgres and multiple two-phase transactions. Two-phase - commits and rollbacks didn't automatically end up with a new transaction - as the usual dbapi commits/rollbacks do. - - .. change:: - :tags: - :tickets: - - Added an option to the _ScopedExt mapper extension to not automatically - save new objects to session on object initialization. - - .. change:: - :tags: - :tickets: - - fixed Oracle non-ansi join syntax - - .. change:: - :tags: - :tickets: - - PickleType and Interval types (on db not supporting it natively) are now - slightly faster. - - .. change:: - :tags: - :tickets: - - Added Float and Time types to Firebird (FBFloat and FBTime). Fixed - BLOB SUB_TYPE for TEXT and Binary types. - - .. change:: - :tags: - :tickets: - - Changed the API for the in\_ operator. in_() now accepts a single argument - that is a sequence of values or a selectable. The old API of passing in - values as varargs still works but is deprecated. - -.. changelog:: - :version: 0.4.0beta6 - :released: Thu Sep 27 2007 - - .. change:: - :tags: - :tickets: - - The Session identity map is now *weak referencing* by default, use - weak_identity_map=False to use a regular dict. The weak dict we are using - is customized to detect instances which are "dirty" and maintain a - temporary strong reference to those instances until changes are flushed. - - .. change:: - :tags: - :tickets: 758 - - Mapper compilation has been reorganized such that most compilation occurs - upon mapper construction. This allows us to have fewer calls to - mapper.compile() and also to allow class-based properties to force a - compilation (i.e. User.addresses == 7 will compile all mappers; this is). The only caveat here is that an inheriting mapper now - looks for its inherited mapper upon construction; so mappers within - inheritance relationships need to be constructed in inheritance order - (which should be the normal case anyway). - - .. change:: - :tags: - :tickets: - - added "FETCH" to the keywords detected by Postgres to indicate a - result-row holding statement (i.e. in addition to "SELECT"). - - .. change:: - :tags: - :tickets: - - Added full list of SQLite reserved keywords so that they get escaped - properly. - - .. change:: - :tags: - :tickets: - - Tightened up the relationship between the Query's generation of "eager - load" aliases, and Query.instances() which actually grabs the eagerly - loaded rows. If the aliases were not specifically generated for that - statement by EagerLoader, the EagerLoader will not take effect when the - rows are fetched. This prevents columns from being grabbed accidentally - as being part of an eager load when they were not meant for such, which - can happen with textual SQL as well as some inheritance situations. It's - particularly important since the "anonymous aliasing" of columns uses - simple integer counts now to generate labels. - - .. change:: - :tags: - :tickets: - - Removed "parameters" argument from clauseelement.compile(), replaced with - "column_keys". The parameters sent to execute() only interact with the - insert/update statement compilation process in terms of the column names - present but not the values for those columns. Produces more consistent - execute/executemany behavior, simplifies things a bit internally. - - .. change:: - :tags: - :tickets: 560 - - Added 'comparator' keyword argument to PickleType. By default, "mutable" - PickleType does a "deep compare" of objects using their dumps() - representation. But this doesn't work for dictionaries. Pickled objects - which provide an adequate __eq__() implementation can be set up with - "PickleType(comparator=operator.eq)" - - .. change:: - :tags: - :tickets: - - Added session.is_modified(obj) method; performs the same "history" - comparison operation as occurs within a flush operation; setting - include_collections=False gives the same result as is used when the flush - determines whether or not to issue an UPDATE for the instance's row. - - .. change:: - :tags: - :tickets: 584, 761 - - Added "schema" argument to Sequence; use this with Postgres /Oracle when - the sequence is located in an alternate schema. Implements part of, should fix. - - .. change:: - :tags: - :tickets: - - Fixed reflection of the empty string for mysql enums. - - .. change:: - :tags: - :tickets: 794 - - Changed MySQL dialect to use the older LIMIT , syntax - instead of LIMIT OFFSET for folks using 3.23. - - .. change:: - :tags: - :tickets: - - Added 'passive_deletes="all"' flag to relation(), disables all nulling-out - of foreign key attributes during a flush where the parent object is - deleted. - - .. change:: - :tags: - :tickets: - - Column defaults and onupdates, executing inline, will add parenthesis for - subqueries and other parenthesis-requiring expressions - - .. change:: - :tags: - :tickets: 793 - - The behavior of String/Unicode types regarding that they auto-convert to - TEXT/CLOB when no length is present now occurs *only* for an exact type of - String or Unicode with no arguments. If you use VARCHAR or NCHAR - (subclasses of String/Unicode) with no length, they will be interpreted by - the dialect as VARCHAR/NCHAR; no "magic" conversion happens there. This - is less surprising behavior and in particular this helps Oracle keep - string-based bind parameters as VARCHARs and not CLOBs. - - .. change:: - :tags: - :tickets: 771 - - Fixes to ShardedSession to work with deferred columns. - - .. change:: - :tags: - :tickets: - - User-defined shard_chooser() function must accept "clause=None" argument; - this is the ClauseElement passed to session.execute(statement) and can be - used to determine correct shard id (since execute() doesn't take an - instance.) - - .. change:: - :tags: - :tickets: 764 - - Adjusted operator precedence of NOT to match '==' and others, so that - ~(x y) produces NOT (x y), which is better compatible - with older MySQL versions.. This doesn't apply to "~(x==y)" - as it does in 0.3 since ~(x==y) compiles to "x != y", but still applies - to operators like BETWEEN. - - .. change:: - :tags: - :tickets: 757, 768, 779, 728 - - Other tickets:,,. - -.. changelog:: - :version: 0.4.0beta5 - :released: - - .. change:: - :tags: - :tickets: 754 - - Connection pool fixes; the better performance of beta4 remains but fixes - "connection overflow" and other bugs which were present (like). - - .. change:: - :tags: - :tickets: 769 - - Fixed bugs in determining proper sync clauses from custom inherit - conditions. - - .. change:: - :tags: - :tickets: 763 - - Extended 'engine_from_config' coercion for QueuePool size / overflow. - - .. change:: - :tags: - :tickets: 748 - - mysql views can be reflected again. - - .. change:: - :tags: - :tickets: - - AssociationProxy can now take custom getters and setters. - - .. change:: - :tags: - :tickets: - - Fixed malfunctioning BETWEEN in orm queries. - - .. change:: - :tags: - :tickets: 762 - - Fixed OrderedProperties pickling - - .. change:: - :tags: - :tickets: - - SQL-expression defaults and sequences now execute "inline" for all - non-primary key columns during an INSERT or UPDATE, and for all columns - during an executemany()-style call. inline=True flag on any insert/update - statement also forces the same behavior with a single execute(). - result.postfetch_cols() is a collection of columns for which the previous - single insert or update statement contained a SQL-side default expression. - - .. change:: - :tags: - :tickets: 759 - - Fixed PG executemany() behavior. - - .. change:: - :tags: - :tickets: - - postgres reflects tables with autoincrement=False for primary key columns - which have no defaults. - - .. change:: - :tags: - :tickets: - - postgres no longer wraps executemany() with individual execute() calls, - instead favoring performance. "rowcount"/"concurrency" checks with - deleted items (which use executemany) are disabled with PG since psycopg2 - does not report proper rowcount for executemany(). - - .. change:: - :tags: tickets, fixed - :tickets: 742 - - - - .. change:: - :tags: tickets, fixed - :tickets: 748 - - - - .. change:: - :tags: tickets, fixed - :tickets: 760 - - - - .. change:: - :tags: tickets, fixed - :tickets: 762 - - - - .. change:: - :tags: tickets, fixed - :tickets: 763 - - - -.. changelog:: - :version: 0.4.0beta4 - :released: Wed Aug 22 2007 - - .. change:: - :tags: - :tickets: - - Tidied up what ends up in your namespace when you 'from sqlalchemy import \*': - - .. change:: - :tags: - :tickets: - - 'table' and 'column' are no longer imported. They remain available by - direct reference (as in 'sql.table' and 'sql.column') or a glob import - from the sql package. It was too easy to accidentally use a - sql.expressions.table instead of schema.Table when just starting out - with SQLAlchemy, likewise column. - - .. change:: - :tags: - :tickets: - - Internal-ish classes like ClauseElement, FromClause, NullTypeEngine, - etc., are also no longer imported into your namespace - - .. change:: - :tags: - :tickets: - - The 'Smallinteger' compatibility name (small i!) is no longer imported, - but remains in schema.py for now. SmallInteger (big I!) is still - imported. - - .. change:: - :tags: - :tickets: - - The connection pool uses a "threadlocal" strategy internally to return - the same connection already bound to a thread, for "contextual" connections; - these are the connections used when you do a "connectionless" execution - like insert().execute(). This is like a "partial" version of the - "threadlocal" engine strategy but without the thread-local transaction part - of it. We're hoping it reduces connection pool overhead as well as - database usage. However, if it proves to impact stability in a negative way, - we'll roll it right back. - - .. change:: - :tags: - :tickets: - - Fix to bind param processing such that "False" values (like blank strings) - still get processed/encoded. - - .. change:: - :tags: - :tickets: 752 - - Fix to select() "generative" behavior, such that calling column(), - select_from(), correlate(), and with_prefix() does not modify the - original select object - - .. change:: - :tags: - :tickets: - - Added a "legacy" adapter to types, such that user-defined TypeEngine - and TypeDecorator classes which define convert_bind_param() and/or - convert_result_value() will continue to function. Also supports - calling the super() version of those methods. - - .. change:: - :tags: - :tickets: - - Added session.prune(), trims away instances cached in a session that - are no longer referenced elsewhere. (A utility for strong-ref - identity maps). - - .. change:: - :tags: - :tickets: - - Added close() method to Transaction. Closes out a transaction using - rollback if it's the outermost transaction, otherwise just ends - without affecting the outer transaction. - - .. change:: - :tags: - :tickets: - - Transactional and non-transactional Session integrates better with - bound connection; a close() will ensure that connection - transactional state is the same as that which existed on it before - being bound to the Session. - - .. change:: - :tags: - :tickets: 735 - - Modified SQL operator functions to be module-level operators, - allowing SQL expressions to be pickleable. - - .. change:: - :tags: - :tickets: - - Small adjustment to mapper class.__init__ to allow for Py2.6 - object.__init__() behavior. - - .. change:: - :tags: - :tickets: - - Fixed 'prefix' argument for select() - - .. change:: - :tags: - :tickets: - - Connection.begin() no longer accepts nested=True, this logic is now - all in begin_nested(). - - .. change:: - :tags: - :tickets: - - Fixes to new "dynamic" relation loader involving cascades - - .. change:: - :tags: tickets, fixed - :tickets: 735 - - - - .. change:: - :tags: tickets, fixed - :tickets: 752 - - - -.. changelog:: - :version: 0.4.0beta3 - :released: Thu Aug 16 2007 - - .. change:: - :tags: - :tickets: - - SQL types optimization: - - .. change:: - :tags: - :tickets: - - New performance tests show a combined mass-insert/mass-select test as - having 68% fewer function calls than the same test run against 0.3. - - .. change:: - :tags: - :tickets: - - General performance improvement of result set iteration is around 10-20%. - - .. change:: - :tags: - :tickets: - - In types.AbstractType, convert_bind_param() and convert_result_value() - have migrated to callable-returning bind_processor() and - result_processor() methods. If no callable is returned, no pre/post - processing function is called. - - .. change:: - :tags: - :tickets: - - Hooks added throughout base/sql/defaults to optimize the calling of bind - param/result processors so that method call overhead is minimized. - - .. change:: - :tags: - :tickets: - - Support added for executemany() scenarios such that unneeded "last row id" - logic doesn't kick in, parameters aren't excessively traversed. - - .. change:: - :tags: - :tickets: - - Added 'inherit_foreign_keys' arg to mapper(). - - .. change:: - :tags: - :tickets: - - Added support for string date passthrough in sqlite. - - .. change:: - :tags: tickets, fixed - :tickets: 738 - - - - .. change:: - :tags: tickets, fixed - :tickets: 739 - - - - .. change:: - :tags: tickets, fixed - :tickets: 743 - - - - .. change:: - :tags: tickets, fixed - :tickets: 744 - - - -.. changelog:: - :version: 0.4.0beta2 - :released: Tue Aug 14 2007 - - .. change:: - :tags: oracle, improvements. - :tickets: - - Auto-commit after LOAD DATA INFILE for mysql. - - .. change:: - :tags: oracle, improvements. - :tickets: - - A rudimental SessionExtension class has been added, allowing user-defined - functionality to take place at flush(), commit(), and rollback() boundaries. - - .. change:: - :tags: oracle, improvements. - :tickets: - - Added engine_from_config() function for helping to create_engine() from an - .ini style config. - - .. change:: - :tags: oracle, improvements. - :tickets: - - base_mapper() becomes a plain attribute. - - .. change:: - :tags: oracle, improvements. - :tickets: - - session.execute() and scalar() can search for a Table with which to bind from - using the given ClauseElement. - - .. change:: - :tags: oracle, improvements. - :tickets: - - Session automatically extrapolates tables from mappers with binds, also uses - base_mapper so that inheritance hierarchies bind automatically. - - .. change:: - :tags: oracle, improvements. - :tickets: - - Moved ClauseVisitor traversal back to inlined non-recursive. - - .. change:: - :tags: tickets, fixed - :tickets: 730 - - - - .. change:: - :tags: tickets, fixed - :tickets: 732 - - - - .. change:: - :tags: tickets, fixed - :tickets: 733 - - - - .. change:: - :tags: tickets, fixed - :tickets: 734 - - - -.. changelog:: - :version: 0.4.0beta1 - :released: Sun Aug 12 2007 - - .. change:: - :tags: orm - :tickets: - - Speed! Along with recent speedups to ResultProxy, total number of function - calls significantly reduced for large loads. - - .. change:: - :tags: orm - :tickets: - - test/perf/masseagerload.py reports 0.4 as having the fewest number of - function calls across all SA versions (0.1, 0.2, and 0.3). - - .. change:: - :tags: orm - :tickets: 213 - - New collection_class api and implementation. Collections are - now instrumented via decorations rather than proxying. You can now have - collections that manage their own membership, and your class instance will - be directly exposed on the relation property. The changes are transparent - for most users. - - .. change:: - :tags: orm - :tickets: - - InstrumentedList (as it was) is removed, and relation properties no - longer have 'clear()', '.data', or any other added methods beyond those - provided by the collection type. You are free, of course, to add them to - a custom class. - - .. change:: - :tags: orm - :tickets: - - __setitem__-like assignments now fire remove events for the existing - value, if any. - - .. change:: - :tags: orm - :tickets: - - dict-likes used as collection classes no longer need to change __iter__ - semantics- itervalues() is used by default instead. This is a backwards - incompatible change. - - .. change:: - :tags: orm - :tickets: - - Subclassing dict for a mapped collection is no longer needed in most - cases. orm.collections provides canned implementations that key objects - by a specified column or a custom function of your choice. - - .. change:: - :tags: orm - :tickets: - - Collection assignment now requires a compatible type- assigning None to - clear a collection or assigning a list to a dict collection will now - raise an argument error. - - .. change:: - :tags: orm - :tickets: - - AttributeExtension moved to interfaces, and .delete is now .remove The - event method signature has also been swapped around. - - .. change:: - :tags: orm - :tickets: - - Major overhaul for Query: - - .. change:: - :tags: orm - :tickets: - - All selectXXX methods are deprecated. Generative methods are now the - standard way to do things, i.e. filter(), filter_by(), all(), one(), - etc. Deprecated methods are docstring'ed with their new replacements. - - .. change:: - :tags: orm - :tickets: 643 - - Class-level properties are now usable as query elements... no more - '.c.'! "Class.c.propname" is now superseded by "Class.propname". All - clause operators are supported, as well as higher level operators such - as Class.prop== for scalar attributes, - Class.prop.contains() and Class.prop.any() for collection-based attributes (all are also - negatable). Table-based column expressions as well as columns mounted - on mapped classes via 'c' are of course still fully available and can be - freely mixed with the new attributes. - - .. change:: - :tags: orm - :tickets: - - Removed ancient query.select_by_attributename() capability. - - .. change:: - :tags: orm - :tickets: - - The aliasing logic used by eager loading has been generalized, so that - it also adds full automatic aliasing support to Query. It's no longer - necessary to create an explicit Alias to join to the same tables - multiple times; *even for self-referential relationships*. - - - join() and outerjoin() take arguments "aliased=True". Yhis causes - their joins to be built on aliased tables; subsequent calls to - filter() and filter_by() will translate all table expressions (yes, - real expressions using the original mapped Table) to be that of the - Alias for the duration of that join() (i.e. until reset_joinpoint() or - another join() is called). - - - join() and outerjoin() take arguments "id=". When used - with "aliased=True", the id can be referenced by add_entity(cls, - id=) so that you can select the joined instances even if - they're from an alias. - - - join() and outerjoin() now work with self-referential relationships! - Using "aliased=True", you can join as many levels deep as desired, - i.e. query.join(['children', 'children'], aliased=True); filter - criterion will be against the rightmost joined table - - .. change:: - :tags: orm - :tickets: 660 - - Added query.populate_existing(), marks the query to reload all - attributes and collections of all instances touched in the query, - including eagerly-loaded entities. - - .. change:: - :tags: orm - :tickets: - - Added eagerload_all(), allows eagerload_all('x.y.z') to specify eager - loading of all properties in the given path. - - .. change:: - :tags: orm - :tickets: - - Major overhaul for Session: - - .. change:: - :tags: orm - :tickets: - - New function which "configures" a session called "sessionmaker()". Send - various keyword arguments to this function once, returns a new class - which creates a Session against that stereotype. - - .. change:: - :tags: orm - :tickets: - - SessionTransaction removed from "public" API. You now can call begin()/ - commit()/rollback() on the Session itself. - - .. change:: - :tags: orm - :tickets: - - Session also supports SAVEPOINT transactions; call begin_nested(). - - .. change:: - :tags: orm - :tickets: - - Session supports two-phase commit behavior when vertically or - horizontally partitioning (i.e., using more than one engine). Use - twophase=True. - - .. change:: - :tags: orm - :tickets: - - Session flag "transactional=True" produces a session which always places - itself into a transaction when first used. Upon commit(), rollback() or - close(), the transaction ends; but begins again on the next usage. - - .. change:: - :tags: orm - :tickets: - - Session supports "autoflush=True". This issues a flush() before each - query. Use in conjunction with transactional, and you can just - save()/update() and then query, the new objects will be there. Use - commit() at the end (or flush() if non-transactional) to flush remaining - changes. - - .. change:: - :tags: orm - :tickets: - - New scoped_session() function replaces SessionContext and assignmapper. - Builds onto "sessionmaker()" concept to produce a class whose Session() - construction returns the thread-local session. Or, call all Session - methods as class methods, i.e. Session.save(foo); Session.commit(). - just like the old "objectstore" days. - - .. change:: - :tags: orm - :tickets: - - Added new "binds" argument to Session to support configuration of - multiple binds with sessionmaker() function. - - .. change:: - :tags: orm - :tickets: - - A rudimental SessionExtension class has been added, allowing - user-defined functionality to take place at flush(), commit(), and - rollback() boundaries. - - .. change:: - :tags: orm - :tickets: - - Query-based relation()s available with dynamic_loader(). This is a - *writable* collection (supporting append() and remove()) which is also a - live Query object when accessed for reads. Ideal for dealing with very - large collections where only partial loading is desired. - - .. change:: - :tags: orm - :tickets: - - flush()-embedded inline INSERT/UPDATE expressions. Assign any SQL - expression, like "sometable.c.column + 1", to an instance's attribute. - Upon flush(), the mapper detects the expression and embeds it directly in - the INSERT or UPDATE statement; the attribute gets deferred on the - instance so it loads the new value the next time you access it. - - .. change:: - :tags: orm - :tickets: 618 - - A rudimental sharding (horizontal scaling) system is introduced. This - system uses a modified Session which can distribute read and write - operations among multiple databases, based on user-defined functions - defining the "sharding strategy". Instances and their dependents can be - distributed and queried among multiple databases based on attribute - values, round-robin approaches or any other user-defined - system. - - .. change:: - :tags: orm - :tickets: 659 - - Eager loading has been enhanced to allow even more joins in more places. - It now functions at any arbitrary depth along self-referential and - cyclical structures. When loading cyclical structures, specify - "join_depth" on relation() indicating how many times you'd like the table - to join to itself; each level gets a distinct table alias. The alias - names themselves are generated at compile time using a simple counting - scheme now and are a lot easier on the eyes, as well as of course - completely deterministic. - - .. change:: - :tags: orm - :tickets: 211 - - Added composite column properties. This allows you to create a type which - is represented by more than one column, when using the ORM. Objects of - the new type are fully functional in query expressions, comparisons, - query.get() clauses, etc. and act as though they are regular single-column - scalars... except they're not! Use the function composite(cls, \*columns) - inside of the mapper's "properties" dict, and instances of cls will be - created/mapped to a single attribute, comprised of the values corresponding - to \*columns. - - .. change:: - :tags: orm - :tickets: - - Improved support for custom column_property() attributes which feature - correlated subqueries, works better with eager loading now. - - .. change:: - :tags: orm - :tickets: 611 - - Primary key "collapse" behavior; the mapper will analyze all columns in - its given selectable for primary key "equivalence", that is, columns which - are equivalent via foreign key relationship or via an explicit - inherit_condition. primarily for joined-table inheritance scenarios where - different named PK columns in inheriting tables should "collapse" into a - single-valued (or fewer-valued) primary key. Fixes things like. - - .. change:: - :tags: orm - :tickets: - - Joined-table inheritance will now generate the primary key columns of all - inherited classes against the root table of the join only. This implies - that each row in the root table is distinct to a single instance. If for - some rare reason this is not desirable, explicit primary_key settings on - individual mappers will override it. - - .. change:: - :tags: orm - :tickets: - - When "polymorphic" flags are used with joined-table or single-table - inheritance, all identity keys are generated against the root class of the - inheritance hierarchy; this allows query.get() to work polymorphically - using the same caching semantics as a non-polymorphic get. Note that this - currently does not work with concrete inheritance. - - .. change:: - :tags: orm - :tickets: - - Secondary inheritance loading: polymorphic mappers can be constructed - *without* a select_table argument. inheriting mappers whose tables were - not represented in the initial load will issue a second SQL query - immediately, once per instance (i.e. not very efficient for large lists), - in order to load the remaining columns. - - .. change:: - :tags: orm - :tickets: - - Secondary inheritance loading can also move its second query into a - column-level "deferred" load, via the "polymorphic_fetch" argument, which - can be set to 'select' or 'deferred' - - .. change:: - :tags: orm - :tickets: 696 - - It's now possible to map only a subset of available selectable columns - onto mapper properties, using include_columns/exclude_columns.. - - .. change:: - :tags: orm - :tickets: - - Added undefer_group() MapperOption, sets a set of "deferred" columns - joined by a "group" to load as "undeferred". - - .. change:: - :tags: orm - :tickets: - - Rewrite of the "deterministic alias name" logic to be part of the SQL - layer, produces much simpler alias and label names more in the style of - Hibernate - - .. change:: - :tags: sql - :tickets: - - Speed! Clause compilation as well as the mechanics of SQL constructs have - been streamlined and simplified to a significant degree, for a 20-30% - improvement of the statement construction/compilation overhead of 0.3. - - .. change:: - :tags: sql - :tickets: - - All "type" keyword arguments, such as those to bindparam(), column(), - Column(), and func.(), renamed to "type\_". Those objects still - name their "type" attribute as "type". - - .. change:: - :tags: sql - :tickets: - - case_sensitive=(True|False) setting removed from schema items, since - checking this state added a lot of method call overhead and there was no - decent reason to ever set it to False. Table and column names which are - all lower case will be treated as case-insensitive (yes we adjust for - Oracle's UPPERCASE style too). - - .. change:: - :tags: transactions - :tickets: - - Added context manager (with statement) support for transactions. - - .. change:: - :tags: transactions - :tickets: - - Added support for two phase commit, works with mysql and postgres so far. - - .. change:: - :tags: transactions - :tickets: - - Added a subtransaction implementation that uses savepoints. - - .. change:: - :tags: transactions - :tickets: - - Added support for savepoints. - - .. change:: - :tags: metadata - :tickets: - - Tables can be reflected from the database en-masse without declaring - them in advance. MetaData(engine, reflect=True) will load all tables - present in the database, or use metadata.reflect() for finer control. - - .. change:: - :tags: metadata - :tickets: - - DynamicMetaData has been renamed to ThreadLocalMetaData - - .. change:: - :tags: metadata - :tickets: - - The ThreadLocalMetaData constructor now takes no arguments. - - .. change:: - :tags: metadata - :tickets: - - BoundMetaData has been removed- regular MetaData is equivalent - - .. change:: - :tags: metadata - :tickets: 646 - - Numeric and Float types now have an "asdecimal" flag; defaults to True for - Numeric, False for Float. When True, values are returned as - decimal.Decimal objects; when False, values are returned as float(). The - defaults of True/False are already the behavior for PG and MySQL's DBAPI - modules. - - .. change:: - :tags: metadata - :tickets: 475 - - New SQL operator implementation which removes all hardcoded operators from - expression structures and moves them into compilation; allows greater - flexibility of operator compilation; for example, "+" compiles to "||" - when used in a string context, or "concat(a,b)" on MySQL; whereas in a - numeric context it compiles to "+". Fixes. - - .. change:: - :tags: metadata - :tickets: - - "Anonymous" alias and label names are now generated at SQL compilation - time in a completely deterministic fashion... no more random hex IDs - - .. change:: - :tags: metadata - :tickets: - - Significant architectural overhaul to SQL elements (ClauseElement). All - elements share a common "mutability" framework which allows a consistent - approach to in-place modifications of elements as well as generative - behavior. Improves stability of the ORM which makes heavy usage of - mutations to SQL expressions. - - .. change:: - :tags: metadata - :tickets: - - select() and union()'s now have "generative" behavior. Methods like - order_by() and group_by() return a *new* instance - the original instance - is left unchanged. Non-generative methods remain as well. - - .. change:: - :tags: metadata - :tickets: 569, 52 - - The internals of select/union vastly simplified- all decision making - regarding "is subquery" and "correlation" pushed to SQL generation phase. - select() elements are now *never* mutated by their enclosing containers or - by any dialect's compilation process - - .. change:: - :tags: metadata - :tickets: - - select(scalar=True) argument is deprecated; use select(..).as_scalar(). - The resulting object obeys the full "column" interface and plays better - within expressions. - - .. change:: - :tags: metadata - :tickets: 504 - - Added select().with_prefix('foo') allowing any set of keywords to be - placed before the columns clause of the SELECT - - .. change:: - :tags: metadata - :tickets: 686 - - Added array slice support to row[] - - .. change:: - :tags: metadata - :tickets: - - Result sets make a better attempt at matching the DBAPI types present in - cursor.description to the TypeEngine objects defined by the dialect, which - are then used for result-processing. Note this only takes effect for - textual SQL; constructed SQL statements always have an explicit type map. - - .. change:: - :tags: metadata - :tickets: - - Result sets from CRUD operations close their underlying cursor immediately - and will also autoclose the connection if defined for the operation; this - allows more efficient usage of connections for successive CRUD operations - with less chance of "dangling connections". - - .. change:: - :tags: metadata - :tickets: 559 - - Column defaults and onupdate Python functions (i.e. passed to - ColumnDefault) may take zero or one arguments; the one argument is the - ExecutionContext, from which you can call "context.parameters[someparam]" - to access the other bind parameter values affixed to the statement. The connection used for the execution is available as well - so that you can pre-execute statements. - - .. change:: - :tags: metadata - :tickets: - - Added "explicit" create/drop/execute support for sequences (i.e. you can - pass a "connectable" to each of those methods on Sequence). - - .. change:: - :tags: metadata - :tickets: - - Better quoting of identifiers when manipulating schemas. - - .. change:: - :tags: metadata - :tickets: - - Standardized the behavior for table reflection where types can't be - located; NullType is substituted instead, warning is raised. - - .. change:: - :tags: metadata - :tickets: 606 - - ColumnCollection (i.e. the 'c' attribute on tables) follows dictionary - semantics for "__contains__" - - .. change:: - :tags: engines - :tickets: - - Speed! The mechanics of result processing and bind parameter processing - have been overhauled, streamlined and optimized to issue as little method - calls as possible. Bench tests for mass INSERT and mass rowset iteration - both show 0.4 to be over twice as fast as 0.3, using 68% fewer function - calls. - - .. change:: - :tags: engines - :tickets: - - You can now hook into the pool lifecycle and run SQL statements or other - logic at new each DBAPI connection, pool check-out and check-in. - - .. change:: - :tags: engines - :tickets: - - Connections gain a .properties collection, with contents scoped to the - lifetime of the underlying DBAPI connection - - .. change:: - :tags: engines - :tickets: - - Removed auto_close_cursors and disallow_open_cursors arguments from Pool; - reduces overhead as cursors are normally closed by ResultProxy and - Connection. - - .. change:: - :tags: extensions - :tickets: - - proxyengine is temporarily removed, pending an actually working - replacement. - - .. change:: - :tags: extensions - :tickets: - - SelectResults has been replaced by Query. SelectResults / - SelectResultsExt still exist but just return a slightly modified Query - object for backwards-compatibility. join_to() method from SelectResults - isn't present anymore, need to use join(). - - .. change:: - :tags: mysql - :tickets: - - Table and column names loaded via reflection are now Unicode. - - .. change:: - :tags: mysql - :tickets: - - All standard column types are now supported, including SET. - - .. change:: - :tags: mysql - :tickets: - - Table reflection can now be performed in as little as one round-trip. - - .. change:: - :tags: mysql - :tickets: - - ANSI and ANSI_QUOTES sql modes are now supported. - - .. change:: - :tags: mysql - :tickets: - - Indexes are now reflected. - - .. change:: - :tags: postgres - :tickets: - - Added PGArray datatype for using postgres array datatypes. - - .. change:: - :tags: oracle - :tickets: 507 - - Very rudimental support for OUT parameters added; use sql.outparam(name, - type) to set up an OUT parameter, just like bindparam(); after execution, - values are available via result.out_parameters dictionary. diff --git a/doc/build/changelog/changelog_05.rst b/doc/build/changelog/changelog_05.rst deleted file mode 100644 index e998cb4443b..00000000000 --- a/doc/build/changelog/changelog_05.rst +++ /dev/null @@ -1,3775 +0,0 @@ - -============= -0.5 Changelog -============= - - -.. changelog:: - :version: 0.5.9 - :released: - - .. change:: - :tags: sql - :tickets: 1661 - - Fixed erroneous self_group() call in expression package. - -.. changelog:: - :version: 0.5.8 - :released: Sat Jan 16 2010 - - .. change:: - :tags: sql - :tickets: - - The copy() method on Column now supports uninitialized, - unnamed Column objects. This allows easy creation of - declarative helpers which place common columns on multiple - subclasses. - - .. change:: - :tags: sql - :tickets: - - Default generators like Sequence() translate correctly - across a copy() operation. - - .. change:: - :tags: sql - :tickets: - - Sequence() and other DefaultGenerator objects are accepted - as the value for the "default" and "onupdate" keyword - arguments of Column, in addition to being accepted - positionally. - - .. change:: - :tags: sql - :tickets: 1568, 1617 - - Fixed a column arithmetic bug that affected column - correspondence for cloned selectables which contain - free-standing column expressions. This bug is - generally only noticeable when exercising newer - ORM behavior only available in 0.6 via, - but is more correct at the SQL expression level - as well. - - .. change:: - :tags: postgresql - :tickets: 1647 - - The extract() function, which was slightly improved in - 0.5.7, needed a lot more work to generate the correct - typecast (the typecasts appear to be necessary in PG's - EXTRACT quite a lot of the time). The typecast is - now generated using a rule dictionary based - on PG's documentation for date/time/interval arithmetic. - It also accepts text() constructs again, which was broken - in 0.5.7. - - .. change:: - :tags: firebird - :tickets: 1646 - - Recognize more errors as disconnections. - -.. changelog:: - :version: 0.5.7 - :released: Sat Dec 26 2009 - - .. change:: - :tags: orm - :tickets: 1543 - - contains_eager() now works with the automatically - generated subquery that results when you say - "query(Parent).join(Parent.somejoinedsubclass)", i.e. - when Parent joins to a joined-table-inheritance subclass. - Previously contains_eager() would erroneously add the - subclass table to the query separately producing a - cartesian product. An example is in the ticket - description. - - .. change:: - :tags: orm - :tickets: 1553 - - query.options() now only propagate to loaded objects - for potential further sub-loads only for options where - such behavior is relevant, keeping - various unserializable options like those generated - by contains_eager() out of individual instance states. - - .. change:: - :tags: orm - :tickets: 1054 - - Session.execute() now locates table- and - mapper-specific binds based on a passed - in expression which is an insert()/update()/delete() - construct. - - .. change:: - :tags: orm - :tickets: - - Session.merge() now properly overwrites a many-to-one or - uselist=False attribute to None if the attribute - is also None in the given object to be merged. - - .. change:: - :tags: orm - :tickets: 1618 - - Fixed a needless select which would occur when merging - transient objects that contained a null primary key - identifier. - - .. change:: - :tags: orm - :tickets: 1585 - - Mutable collection passed to the "extension" attribute - of relation(), column_property() etc. will not be mutated - or shared among multiple instrumentation calls, preventing - duplicate extensions, such as backref populators, - from being inserted into the list. - - .. change:: - :tags: orm - :tickets: 1504 - - Fixed the call to get_committed_value() on CompositeProperty. - - .. change:: - :tags: orm - :tickets: 1602 - - Fixed bug where Query would crash if a join() with no clear - "left" side were called when a non-mapped column entity - appeared in the columns list. - - .. change:: - :tags: orm - :tickets: 1616, 1480 - - Fixed bug whereby composite columns wouldn't load properly - when configured on a joined-table subclass, introduced in - version 0.5.6 as a result of the fix for. thx to Scott Torborg. - - .. change:: - :tags: orm - :tickets: 1556 - - The "use get" behavior of many-to-one relations, i.e. that a - lazy load will fallback to the possibly cached query.get() - value, now works across join conditions where the two compared - types are not exactly the same class, but share the same - "affinity" - i.e. Integer and SmallInteger. Also allows - combinations of reflected and non-reflected types to work - with 0.5 style type reflection, such as PGText/Text (note 0.6 - reflects types as their generic versions). - - .. change:: - :tags: orm - :tickets: 1436 - - Fixed bug in query.update() when passing Cls.attribute - as keys in the value dict and using synchronize_session='expire' - ('fetch' in 0.6). - - .. change:: - :tags: sql - :tickets: 1603 - - Fixed bug in two-phase transaction whereby commit() method - didn't set the full state which allows subsequent close() - call to succeed. - - .. change:: - :tags: sql - :tickets: - - Fixed the "numeric" paramstyle, which apparently is the - default paramstyle used by Informixdb. - - .. change:: - :tags: sql - :tickets: 1574 - - Repeat expressions in the columns clause of a select - are deduped based on the identity of each clause element, - not the actual string. This allows positional - elements to render correctly even if they all render - identically, such as "qmark" style bind parameters. - - .. change:: - :tags: sql - :tickets: 1632 - - The cursor associated with connection pool connections - (i.e. _CursorFairy) now proxies `__iter__()` to the - underlying cursor correctly. - - .. change:: - :tags: sql - :tickets: 1556 - - types now support an "affinity comparison" operation, i.e. - that an Integer/SmallInteger are "compatible", or - a Text/String, PickleType/Binary, etc. Part of. - - .. change:: - :tags: sql - :tickets: 1641 - - Fixed bug preventing alias() of an alias() from being - cloned or adapted (occurs frequently in ORM operations). - - .. change:: - :tags: sqlite - :tickets: 1439 - - sqlite dialect properly generates CREATE INDEX for a table - that is in an alternate schema. - - .. change:: - :tags: postgresql - :tickets: 1085 - - Added support for reflecting the DOUBLE PRECISION type, - via a new postgres.PGDoublePrecision object. - This is postgresql.DOUBLE_PRECISION in 0.6. - - .. change:: - :tags: postgresql - :tickets: 460 - - Added support for reflecting the INTERVAL YEAR TO MONTH - and INTERVAL DAY TO SECOND syntaxes of the INTERVAL - type. - - .. change:: - :tags: postgresql - :tickets: 1576 - - Corrected the "has_sequence" query to take current schema, - or explicit sequence-stated schema, into account. - - .. change:: - :tags: postgresql - :tickets: 1611 - - Fixed the behavior of extract() to apply operator - precedence rules to the "::" operator when applying - the "timestamp" cast - ensures proper parenthesization. - - .. change:: - :tags: mssql - :tickets: 1561 - - Changed the name of TrustedConnection to - Trusted_Connection when constructing pyodbc connect - arguments - - .. change:: - :tags: oracle - :tickets: 1637 - - The "table_names" dialect function, used by MetaData - .reflect(), omits "index overflow tables", a system - table generated by Oracle when "index only tables" - with overflow are used. These tables aren't accessible - via SQL and can't be reflected. - - .. change:: - :tags: ext - :tickets: 1570, 1523 - - A column can be added to a joined-table declarative - superclass after the class has been constructed - (i.e. via class-level attribute assignment), and - the column will be propagated down to - subclasses. This is the reverse - situation as that of, fixed in 0.5.6. - - .. change:: - :tags: ext - :tickets: 1491 - - Fixed a slight inaccuracy in the sharding example. - Comparing equivalence of columns in the ORM is best - accomplished using col1.shares_lineage(col2). - - .. change:: - :tags: ext - :tickets: 1606 - - Removed unused `load()` method from ShardedQuery. - -.. changelog:: - :version: 0.5.6 - :released: Sat Sep 12 2009 - - .. change:: - :tags: orm - :tickets: 1300 - - Fixed bug whereby inheritance discriminator part of a - composite primary key would fail on updates. - Continuation of. - - .. change:: - :tags: orm - :tickets: 1507 - - Fixed bug which disallowed one side of a many-to-many - bidirectional reference to declare itself as "viewonly" - - .. change:: - :tags: orm - :tickets: 1526 - - Added an assertion that prevents a @validates function - or other AttributeExtension from loading an unloaded - collection such that internal state may be corrupted. - - .. change:: - :tags: orm - :tickets: 1519 - - Fixed bug which prevented two entities from mutually - replacing each other's primary key values within a single - flush() for some orderings of operations. - - .. change:: - :tags: orm - :tickets: 1485 - - Fixed an obscure issue whereby a joined-table subclass - with a self-referential eager load on the base class - would populate the related object's "subclass" table with - data from the "subclass" table of the parent. - - .. change:: - :tags: orm - :tickets: 1477 - - relations() now have greater ability to be "overridden", - meaning a subclass that explicitly specifies a relation() - overriding that of the parent class will be honored - during a flush. This is currently to support - many-to-many relations from concrete inheritance setups. - Outside of that use case, YMMV. - - .. change:: - :tags: orm - :tickets: 1483 - - Squeezed a few more unnecessary "lazy loads" out of - relation(). When a collection is mutated, many-to-one - backrefs on the other side will not fire off to load - the "old" value, unless "single_parent=True" is set. - A direct assignment of a many-to-one still loads - the "old" value in order to update backref collections - on that value, which may be present in the session - already, thus maintaining the 0.5 behavioral contract. - - .. change:: - :tags: orm - :tickets: 1480 - - Fixed bug whereby a load/refresh of joined table - inheritance attributes which were based on - column_property() or similar would fail to evaluate. - - .. change:: - :tags: orm - :tickets: 1488 - - Improved support for MapperProperty objects overriding - that of an inherited mapper for non-concrete - inheritance setups - attribute extensions won't randomly - collide with each other. - - .. change:: - :tags: orm - :tickets: 1487 - - UPDATE and DELETE do not support ORDER BY, LIMIT, OFFSET, - etc. in standard SQL. Query.update() and Query.delete() - now raise an exception if any of limit(), offset(), - order_by(), group_by(), or distinct() have been - called. - - .. change:: - :tags: orm - :tickets: - - Added AttributeExtension to sqlalchemy.orm.__all__ - - .. change:: - :tags: orm - :tickets: 1476 - - Improved error message when query() is called with - a non-SQL /entity expression. - - .. change:: - :tags: orm - :tickets: 1440 - - Using False or 0 as a polymorphic discriminator now - works on the base class as well as a subclass. - - .. change:: - :tags: orm - :tickets: 1424 - - Added enable_assertions(False) to Query which disables - the usual assertions for expected state - used - by Query subclasses to engineer custom state.. See - https://www.sqlalchemy.org/trac/wiki/UsageRecipes/PreFilteredQuery - for an example. - - .. change:: - :tags: orm - :tickets: 1501 - - Fixed recursion issue which occurred if a mapped object's - `__len__()` or `__nonzero__()` method resulted in state - changes. - - .. change:: - :tags: orm - :tickets: 1506 - - Fixed incorrect exception raise in - Weak/StrongIdentityMap.add() - - .. change:: - :tags: orm - :tickets: 1522 - - Fixed the error message for "could not find a FROM clause" - in query.join() which would fail to issue correctly - if the query was against a pure SQL construct. - - .. change:: - :tags: orm - :tickets: 1486 - - Fixed a somewhat hypothetical issue which would result - in the wrong primary key being calculated for a mapper - using the old polymorphic_union function - but this - is old stuff. - - .. change:: - :tags: sql - :tickets: 1373 - - Fixed column.copy() to copy defaults and onupdates. - - .. change:: - :tags: sql - :tickets: - - Fixed a bug in extract() introduced in 0.5.4 whereby - the string "field" argument was getting treated as a - ClauseElement, causing various errors within more - complex SQL transformations. - - .. change:: - :tags: sql - :tickets: 1420 - - Unary expressions such as DISTINCT propagate their - type handling to result sets, allowing conversions like - unicode and such to take place. - - .. change:: - :tags: sql - :tickets: 1482 - - Fixed bug in Table and Column whereby passing empty - dict for "info" argument would raise an exception. - - .. change:: - :tags: oracle - :tickets: 1309 - - Backported 0.6 fix for Oracle alias names not getting - truncated. - - .. change:: - :tags: ext - :tickets: 1446 - - The collection proxies produced by associationproxy are now - pickleable. A user-defined proxy_factory however - is still not pickleable unless it defines __getstate__ - and __setstate__. - - .. change:: - :tags: ext - :tickets: 1468 - - Declarative will raise an informative exception if - __table_args__ is passed as a tuple with no dict argument. - Improved documentation. - - .. change:: - :tags: ext - :tickets: 1527 - - Table objects declared in the MetaData can now be used - in string expressions sent to primaryjoin/secondaryjoin/ - secondary - the name is pulled from the MetaData of the - declarative base. - - .. change:: - :tags: ext - :tickets: 1523 - - A column can be added to a joined-table subclass after - the class has been constructed (i.e. via class-level - attribute assignment). The column is added to the underlying - Table as always, but now the mapper will rebuild its - "join" to include the new column, instead of raising - an error about "no such column, use column_property() - instead". - - .. change:: - :tags: test - :tickets: - - Added examples into the test suite so they get exercised - regularly and cleaned up a couple deprecation warnings. - -.. changelog:: - :version: 0.5.5 - :released: Mon Jul 13 2009 - - .. change:: - :tags: general - :tickets: 970 - - unit tests have been migrated from unittest to nose. See - README.unittests for information on how to run the tests. - - .. change:: - :tags: orm - :tickets: - - The "foreign_keys" argument of relation() will now propagate - automatically to the backref in the same way that primaryjoin - and secondaryjoin do. For the extremely rare use case where - the backref of a relation() has intentionally different - "foreign_keys" configured, both sides now need to be - configured explicitly (if they do in fact require this setting, - see the next note...). - - .. change:: - :tags: orm - :tickets: - - ...the only known (and really, really rare) use case where a - different foreign_keys setting was used on the - forwards/backwards side, a composite foreign key that - partially points to its own columns, has been enhanced such - that the fk->itself aspect of the relation won't be used to - determine relation direction. - - .. change:: - :tags: orm - :tickets: - - Session.mapper is now *deprecated*. - - Call session.add() if you'd like a free-standing object to be - part of your session. Otherwise, a DIY version of - Session.mapper is now documented at - https://www.sqlalchemy.org/trac/wiki/UsageRecipes/SessionAwareMapper - The method will remain deprecated throughout 0.6. - - .. change:: - :tags: orm - :tickets: 1431 - - Fixed Query being able to join() from individual columns of a - joined-table subclass entity, i.e. query(SubClass.foo, - SubClass.bar).join(). In most cases, an error - "Could not find a FROM clause to join from" would be - raised. In a few others, the result would be returned in terms - of the base class rather than the subclass - so applications - which relied on this erroneous result need to be - adjusted. - - .. change:: - :tags: orm - :tickets: 1461 - - Fixed a bug involving contains_eager(), which would apply - itself to a secondary (i.e. lazy) load in a particular rare - case, producing cartesian products. improved the targeting of - query.options() on secondary loads overall. - - .. change:: - :tags: orm - :tickets: - - Fixed bug introduced in 0.5.4 whereby Composite types fail - when default-holding columns are flushed. - - .. change:: - :tags: orm - :tickets: 1426 - - Fixed another 0.5.4 bug whereby mutable attributes - (i.e. PickleType) wouldn't be deserialized correctly when the - whole object was serialized. - - .. change:: - :tags: orm - :tickets: - - Fixed bug whereby session.is_modified() would raise an - exception if any synonyms were in use. - - .. change:: - :tags: orm - :tickets: - - Fixed potential memory leak whereby previously pickled objects - placed back in a session would not be fully garbage collected - unless the Session were explicitly closed out. - - .. change:: - :tags: orm - :tickets: - - Fixed bug whereby list-based attributes, like pickletype and - PGArray, failed to be merged() properly. - - .. change:: - :tags: orm - :tickets: - - Repaired non-working attributes.set_committed_value function. - - .. change:: - :tags: orm - :tickets: - - Trimmed the pickle format for InstanceState which should - further reduce the memory footprint of pickled instances. The - format should be backwards compatible with that of 0.5.4 and - previous. - - .. change:: - :tags: orm - :tickets: 1463 - - sqlalchemy.orm.join and sqlalchemy.orm.outerjoin are now - added to __all__ in sqlalchemy.orm.*. - - .. change:: - :tags: orm - :tickets: 1458 - - Fixed bug where Query exception raise would fail when - a too-short composite primary key value were passed to - get(). - - .. change:: - :tags: sql - :tickets: - - Removed an obscure feature of execute() (including connection, - engine, Session) whereby a bindparam() construct can be sent - as a key to the params dictionary. This usage is undocumented - and is at the core of an issue whereby the bindparam() object - created implicitly by a text() construct may have the same - hash value as a string placed in the params dictionary and may - result in an inappropriate match when computing the final bind - parameters. Internal checks for this condition would add - significant latency to the critical task of parameter - rendering, so the behavior is removed. This is a backwards - incompatible change for any application that may have been - using this feature, however the feature has never been - documented. - - .. change:: - :tags: engine/pool - :tickets: - - Implemented recreate() for StaticPool. - -.. changelog:: - :version: 0.5.4p2 - :released: Tue May 26 2009 - - .. change:: - :tags: sql - :tickets: - - Repaired the printing of SQL exceptions which are not - based on parameters or are not executemany() style. - - .. change:: - :tags: postgresql - :tickets: - - Deprecated the hardcoded TIMESTAMP function, which when - used as func.TIMESTAMP(value) would render "TIMESTAMP value". - This breaks on some platforms as PostgreSQL doesn't allow - bind parameters to be used in this context. The hard-coded - uppercase is also inappropriate and there's lots of other - PG casts that we'd need to support. So instead, use - text constructs i.e. select(["timestamp '12/05/09'"]). - -.. changelog:: - :version: 0.5.4p1 - :released: Mon May 18 2009 - - .. change:: - :tags: orm - :tickets: - - Fixed an attribute error introduced in 0.5.4 which would - occur when merge() was used with an incomplete object. - -.. changelog:: - :version: 0.5.4 - :released: Sun May 17 2009 - - .. change:: - :tags: orm - :tickets: 1398 - - Significant performance enhancements regarding Sessions/flush() - in conjunction with large mapper graphs, large numbers of - objects: - - - Removed all* O(N) scanning behavior from the flush() process, - i.e. operations that were scanning the full session, - including an extremely expensive one that was erroneously - assuming primary key values were changing when this - was not the case. - - * one edge case remains which may invoke a full scan, - if an existing primary key attribute is modified - to a new value. - - - The Session's "weak referencing" behavior is now *full* - - no strong references whatsoever are made to a mapped object - or related items/collections in its __dict__. Backrefs and - other cycles in objects no longer affect the Session's ability - to lose all references to unmodified objects. Objects with - pending changes still are maintained strongly until flush. - - - The implementation also improves performance by moving - the "resurrection" process of garbage collected items - to only be relevant for mappings that map "mutable" - attributes (i.e. PickleType, composite attrs). This removes - overhead from the gc process and simplifies internal - behavior. - - If a "mutable" attribute change is the sole change on an object - which is then dereferenced, the mapper will not have access to - other attribute state when the UPDATE is issued. This may present - itself differently to some MapperExtensions. - - The change also affects the internal attribute API, but not - the AttributeExtension interface nor any of the publicly - documented attribute functions. - - - The unit of work no longer generates a graph of "dependency" - processors for the full graph of mappers during flush(), instead - creating such processors only for those mappers which represent - objects with pending changes. This saves a tremendous number - of method calls in the context of a large interconnected - graph of mappers. - - - Cached a wasteful "table sort" operation that previously - occurred multiple times per flush, also removing significant - method call count from flush(). - - - Other redundant behaviors have been simplified in - mapper._save_obj(). - - .. change:: - :tags: orm - :tickets: - - Modified query_cls on DynamicAttributeImpl to accept a full - mixin version of the AppenderQuery, which allows subclassing - the AppenderMixin. - - .. change:: - :tags: orm - :tickets: 1300 - - The "polymorphic discriminator" column may be part of a - primary key, and it will be populated with the correct - discriminator value. - - .. change:: - :tags: orm - :tickets: - - Fixed the evaluator not being able to evaluate IS NULL clauses. - - .. change:: - :tags: orm - :tickets: 1352 - - Fixed the "set collection" function on "dynamic" relations to - initiate events correctly. Previously a collection could only - be assigned to a pending parent instance, otherwise modified - events would not be fired correctly. Set collection is now - compatible with merge(), fixes. - - .. change:: - :tags: orm - :tickets: - - Allowed pickling of PropertyOption objects constructed with - instrumented descriptors; previously, pickle errors would occur - when pickling an object which was loaded with a descriptor-based - option, such as query.options(eagerload(MyClass.foo)). - - .. change:: - :tags: orm - :tickets: 1357 - - Lazy loader will not use get() if the "lazy load" SQL clause - matches the clause used by get(), but contains some parameters - hardcoded. Previously the lazy strategy would fail with the - get(). Ideally get() would be used with the hardcoded - parameters but this would require further development. - - .. change:: - :tags: orm - :tickets: 1391 - - MapperOptions and other state associated with query.options() - is no longer bundled within callables associated with each - lazy/deferred-loading attribute during a load. - The options are now associated with the instance's - state object just once when it's populated. This removes - the need in most cases for per-instance/attribute loader - objects, improving load speed and memory overhead for - individual instances. - - .. change:: - :tags: orm - :tickets: 1360 - - Fixed another location where autoflush was interfering - with session.merge(). autoflush is disabled completely - for the duration of merge() now. - - .. change:: - :tags: orm - :tickets: 1406 - - Fixed bug which prevented "mutable primary key" dependency - logic from functioning properly on a one-to-one - relation(). - - .. change:: - :tags: orm - :tickets: - - Fixed bug in relation(), introduced in 0.5.3, - whereby a self referential relation - from a base class to a joined-table subclass would - not configure correctly. - - .. change:: - :tags: orm - :tickets: - - Fixed obscure mapper compilation issue when inheriting - mappers are used which would result in un-initialized - attributes. - - .. change:: - :tags: orm - :tickets: - - Fixed documentation for session weak_identity_map - - the default value is True, indicating a weak - referencing map in use. - - .. change:: - :tags: orm - :tickets: 1376 - - Fixed a unit of work issue whereby the foreign - key attribute on an item contained within a collection - owned by an object being deleted would not be set to - None if the relation() was self-referential. - - .. change:: - :tags: orm - :tickets: 1378 - - Fixed Query.update() and Query.delete() failures with eagerloaded - relations. - - .. change:: - :tags: orm - :tickets: - - It is now an error to specify both columns of a binary primaryjoin - condition in the foreign_keys or remote_side collection. Whereas - previously it was just nonsensical, but would succeed in a - non-deterministic way. - - .. change:: - :tags: ticket: 594, 1341, schema - :tickets: - - Added a quote_schema() method to the IdentifierPreparer class - so that dialects can override how schemas get handled. This - enables the MSSQL dialect to treat schemas as multipart - identifiers, such as 'database.owner'. - - .. change:: - :tags: sql - :tickets: - - Back-ported the "compiler" extension from SQLA 0.6. This - is a standardized interface which allows the creation of custom - ClauseElement subclasses and compilers. In particular it's - handy as an alternative to text() when you'd like to - build a construct that has database-specific compilations. - See the extension docs for details. - - .. change:: - :tags: sql - :tickets: 1413 - - Exception messages are truncated when the list of bound - parameters is larger than 10, preventing enormous - multi-page exceptions from filling up screens and logfiles - for large executemany() statements. - - .. change:: - :tags: sql - :tickets: - - ``sqlalchemy.extract()`` is now dialect sensitive and can - extract components of timestamps idiomatically across the - supported databases, including SQLite. - - .. change:: - :tags: sql - :tickets: 1353 - - Fixed __repr__() and other _get_colspec() methods on - ForeignKey constructed from __clause_element__() style - construct (i.e. declarative columns). - - .. change:: - :tags: mysql - :tickets: 1405 - - Reflecting a FOREIGN KEY construct will take into account - a dotted schema.tablename combination, if the foreign key - references a table in a remote schema. - - .. change:: - :tags: mssql - :tickets: - - Modified how savepoint logic works to prevent it from - stepping on non-savepoint oriented routines. Savepoint - support is still very experimental. - - .. change:: - :tags: mssql - :tickets: 1310 - - Added in reserved words for MSSQL that covers version 2008 - and all prior versions. - - .. change:: - :tags: mssql - :tickets: 1343 - - Corrected problem with information schema not working with a - binary collation based database. Cleaned up information schema - since it is only used by mssql now. - - .. change:: - :tags: sqlite - :tickets: 1402 - - Corrected the SLBoolean type so that it properly treats only 1 - as True. - - .. change:: - :tags: sqlite - :tickets: 1273 - - Corrected the float type so that it correctly maps to a - SLFloat type when being reflected. - - .. change:: - :tags: extensions - :tickets: 1379 - - Fixed adding of deferred or other column properties to a - declarative class. - -.. changelog:: - :version: 0.5.3 - :released: Tue Mar 24 2009 - - .. change:: - :tags: orm - :tickets: 1315 - - The "objects" argument to session.flush() is deprecated. - State which represents the linkage between a parent and - child object does not support "flushed" status on - one side of the link and not the other, so supporting - this operation leads to misleading results. - - .. change:: - :tags: orm - :tickets: - - Query now implements __clause_element__() which produces - its selectable, which means a Query instance can be accepted - in many SQL expressions, including col.in_(query), - union(query1, query2), select([foo]).select_from(query), - etc. - - .. change:: - :tags: orm - :tickets: 1337 - - Query.join() can now construct multiple FROM clauses, if - needed. Such as, query(A, B).join(A.x).join(B.y) - might say SELECT A.*, B.* FROM A JOIN X, B JOIN Y. - Eager loading can also tack its joins onto those - multiple FROM clauses. - - .. change:: - :tags: orm - :tickets: 1347 - - Fixed bug in dynamic_loader() where append/remove events - after construction time were not being propagated to the - UOW to pick up on flush(). - - .. change:: - :tags: orm - :tickets: - - Fixed bug where column_prefix wasn't being checked before - not mapping an attribute that already had class-level - name present. - - .. change:: - :tags: orm - :tickets: 1315 - - a session.expire() on a particular collection attribute - will clear any pending backref additions as well, so that - the next access correctly returns only what was present - in the database. Presents some degree of a workaround for, although we are considering removing the - flush([objects]) feature altogether. - - .. change:: - :tags: orm - :tickets: - - Session.scalar() now converts raw SQL strings to text() - the same way Session.execute() does and accepts same - alternative \**kw args. - - .. change:: - :tags: orm - :tickets: - - improvements to the "determine direction" logic of - relation() such that the direction of tricky situations - like mapper(A.join(B)) -> relation-> mapper(B) can be - determined. - - .. change:: - :tags: orm - :tickets: 1306 - - When flushing partial sets of objects using session.flush([somelist]), - pending objects which remain pending after the operation won't - inadvertently be added as persistent. - - .. change:: - :tags: orm - :tickets: 1314 - - Added "post_configure_attribute" method to InstrumentationManager, - so that the "listen_for_events.py" example works again. - - .. change:: - :tags: orm - :tickets: - - a forward and complementing backwards reference which are both - of the same direction, i.e. ONETOMANY or MANYTOONE, - is now detected, and an error message is raised. - Saves crazy CircularDependencyErrors later on. - - .. change:: - :tags: orm - :tickets: - - Fixed bugs in Query regarding simultaneous selection of - multiple joined-table inheritance entities with common base - classes: - - - previously the adaption applied to "B" on - "A JOIN B" would be erroneously partially applied - to "A". - - - comparisons on relations (i.e. A.related==someb) - were not getting adapted when they should. - - - Other filterings, like - query(A).join(A.bs).filter(B.foo=='bar'), were erroneously - adapting "B.foo" as though it were an "A". - - .. change:: - :tags: orm - :tickets: 1325 - - Fixed adaptation of EXISTS clauses via any(), has(), etc. - in conjunction with an aliased object on the left and - of_type() on the right. - - .. change:: - :tags: orm - :tickets: - - Added an attribute helper method ``set_committed_value`` in - sqlalchemy.orm.attributes. Given an object, attribute name, - and value, will set the value on the object as part of its - "committed" state, i.e. state that is understood to have - been loaded from the database. Helps with the creation of - homegrown collection loaders and such. - - .. change:: - :tags: orm - :tickets: - - Query won't fail with weakref error when a non-mapper/class - instrumented descriptor is passed, raises - "Invalid column expression". - - .. change:: - :tags: orm - :tickets: - - Query.group_by() properly takes into account aliasing applied - to the FROM clause, such as with select_from(), using - with_polymorphic(), or using from_self(). - - .. change:: - :tags: sql - :tickets: - - An alias() of a select() will convert to a "scalar subquery" - when used in an unambiguously scalar context, i.e. it's used - in a comparison operation. This applies to - the ORM when using query.subquery() as well. - - .. change:: - :tags: sql - :tickets: 1302 - - Fixed missing _label attribute on Function object, others - when used in a select() with use_labels (such as when used - in an ORM column_property()). - - .. change:: - :tags: sql - :tickets: 1309 - - anonymous alias names now truncate down to the max length - allowed by the dialect. More significant on DBs like - Oracle with very small character limits. - - .. change:: - :tags: sql - :tickets: - - the __selectable__() interface has been replaced entirely - by __clause_element__(). - - .. change:: - :tags: sql - :tickets: 1299 - - The per-dialect cache used by TypeEngine to cache - dialect-specific types is now a WeakKeyDictionary. - This to prevent dialect objects from - being referenced forever for an application that - creates an arbitrarily large number of engines - or dialects. There is a small performance penalty - which will be resolved in 0.6. - - .. change:: - :tags: sqlite - :tickets: - - Fixed SQLite reflection methods so that non-present - cursor.description, which triggers an auto-cursor - close, will be detected so that no results doesn't - fail on recent versions of pysqlite which raise - an error when fetchone() called with no rows present. - - .. change:: - :tags: postgresql - :tickets: - - Index reflection won't fail when an index with - multiple expressions is encountered. - - .. change:: - :tags: postgresql - :tickets: 1327 - - Added PGUuid and PGBit types to - sqlalchemy.databases.postgres. - - .. change:: - :tags: postgresql - :tickets: 1327 - - Refection of unknown PG types won't crash when those - types are specified within a domain. - - .. change:: - :tags: mssql - :tickets: - - Preliminary support for pymssql 1.0.1 - - .. change:: - :tags: mssql - :tickets: - - Corrected issue on mssql where max_identifier_length was - not being respected. - - .. change:: - :tags: extensions - :tickets: - - Fixed a recursive pickling issue in serializer, triggered - by an EXISTS or other embedded FROM construct. - - .. change:: - :tags: extensions - :tickets: - - Declarative locates the "inherits" class using a search - through __bases__, to skip over mixins that are local - to subclasses. - - .. change:: - :tags: extensions - :tickets: - - Declarative figures out joined-table inheritance primary join - condition even if "inherits" mapper argument is given - explicitly. - - .. change:: - :tags: extensions - :tickets: - - Declarative will properly interpret the "foreign_keys" argument - on a backref() if it's a string. - - .. change:: - :tags: extensions - :tickets: - - Declarative will accept a table-bound column as a property - when used in conjunction with __table__, if the column is already - present in __table__. The column will be remapped to the given - key the same way as when added to the mapper() properties dict. - -.. changelog:: - :version: 0.5.2 - :released: Sat Jan 24 2009 - - .. change:: - :tags: orm - :tickets: - - Further refined 0.5.1's warning about delete-orphan cascade - placed on a many-to-many relation. First, the bad news: - the warning will apply to both many-to-many as well as - many-to-one relations. This is necessary since in both - cases, SQLA does not scan the full set of potential parents - when determining "orphan" status - for a persistent object - it only detects an in-python de-association event to establish - the object as an "orphan". Next, the good news: to support - one-to-one via a foreign key or association table, or to - support one-to-many via an association table, a new flag - single_parent=True may be set which indicates objects - linked to the relation are only meant to have a single parent. - The relation will raise an error if multiple parent-association - events occur within Python. - - .. change:: - :tags: orm - :tickets: 1292 - - Adjusted the attribute instrumentation change from 0.5.1 to - fully establish instrumentation for subclasses where the mapper - was created after the superclass had already been fully - instrumented. - - .. change:: - :tags: orm - :tickets: - - Fixed bug in delete-orphan cascade whereby two one-to-one - relations from two different parent classes to the same target - class would prematurely expunge the instance. - - .. change:: - :tags: orm - :tickets: - - Fixed an eager loading bug whereby self-referential eager - loading would prevent other eager loads, self referential or not, - from joining to the parent JOIN properly. Thanks to Alex K - for creating a great test case. - - .. change:: - :tags: orm - :tickets: - - session.expire() and related methods will not expire() unloaded - deferred attributes. This prevents them from being needlessly - loaded when the instance is refreshed. - - .. change:: - :tags: orm - :tickets: 1293 - - query.join()/outerjoin() will now properly join an aliased() - construct to the existing left side, even if query.from_self() - or query.select_from(someselectable) has been called. - - .. change:: - :tags: sql - :tickets: 1284 - - Further fixes to the "percent signs and spaces in column/table - names" functionality. - - .. change:: - :tags: mssql - :tickets: 1291 - - Restored convert_unicode handling. Results were being passed - on through without conversion. - - .. change:: - :tags: mssql - :tickets: 1282 - - Really fixing the decimal handling this time.. - - .. change:: - :tags: Ticket:1289, mssql - :tickets: - - Modified table reflection code to use only kwargs when - constructing tables. - -.. changelog:: - :version: 0.5.1 - :released: Sat Jan 17 2009 - - .. change:: - :tags: orm - :tickets: - - Removed an internal join cache which could potentially leak - memory when issuing query.join() repeatedly to ad-hoc - selectables. - - .. change:: - :tags: orm - :tickets: - - The "clear()", "save()", "update()", "save_or_update()" - Session methods have been deprecated, replaced by - "expunge_all()" and "add()". "expunge_all()" has also - been added to ScopedSession. - - .. change:: - :tags: orm - :tickets: - - Modernized the "no mapped table" exception and added a more - explicit __table__/__tablename__ exception to declarative. - - .. change:: - :tags: orm - :tickets: 1237 - - Concrete inheriting mappers now instrument attributes which - are inherited from the superclass, but are not defined for - the concrete mapper itself, with an InstrumentedAttribute that - issues a descriptive error when accessed. - - .. change:: - :tags: orm - :tickets: 1237, 781 - - Added a new `relation()` keyword `back_populates`. This - allows configuration of backreferences using explicit - relations. This is required when creating - bidirectional relations between a hierarchy of concrete - mappers and another class. - - .. change:: - :tags: orm - :tickets: 1237 - - Test coverage added for `relation()` objects specified on - concrete mappers. - - .. change:: - :tags: orm - :tickets: 1276 - - Query.from_self() as well as query.subquery() both disable - the rendering of eager joins inside the subquery produced. - The "disable all eager joins" feature is available publicly - via a new query.enable_eagerloads() generative. - - .. change:: - :tags: orm - :tickets: - - Added a rudimental series of set operations to Query that - receive Query objects as arguments, including union(), - union_all(), intersect(), except_(), intersect_all(), - except_all(). See the API documentation for - Query.union() for examples. - - .. change:: - :tags: orm - :tickets: - - Fixed bug that prevented Query.join() and eagerloads from - attaching to a query that selected from a union or aliased union. - - .. change:: - :tags: orm - :tickets: 1237 - - A short documentation example added for bidirectional - relations specified on concrete mappers. - - .. change:: - :tags: orm - :tickets: 1269 - - Mappers now instrument class attributes upon construction - with the final InstrumentedAttribute object which remains - persistent. The `_CompileOnAttr`/`__getattribute__()` - methodology has been removed. The net effect is that - Column-based mapped class attributes can now be used fully - at the class level without invoking a mapper compilation - operation, greatly simplifying typical usage patterns - within declarative. - - .. change:: - :tags: orm - :tickets: - - ColumnProperty (and front-end helpers such as ``deferred``) no - longer ignores unknown \**keyword arguments. - - .. change:: - :tags: orm - :tickets: - - Fixed a bug with the unitofwork's "row switch" mechanism, - i.e. the conversion of INSERT/DELETE into an UPDATE, when - combined with joined-table inheritance and an object - which contained no defined values for the child table where - an UPDATE with no SET clause would be rendered. - - .. change:: - :tags: orm - :tickets: 1281 - - Using delete-orphan on a many-to-many relation is deprecated. - This produces misleading or erroneous results since SQLA does - not retrieve the full list of "parents" for m2m. To get delete-orphan - behavior with an m2m table, use an explicit association class - so that the individual association row is treated as a parent. - - .. change:: - :tags: orm - :tickets: 1281 - - delete-orphan cascade always requires delete cascade. Specifying - delete-orphan without delete now raises a deprecation warning. - - .. change:: - :tags: sql - :tickets: 1256 - - Improved the methodology to handling percent signs in column - names from. Added more tests. MySQL and - PostgreSQL dialects still do not issue correct CREATE TABLE - statements for identifiers with percent signs in them. - - .. change:: - :tags: schema - :tickets: 1214 - - Index now accepts column-oriented InstrumentedAttributes - (i.e. column-based mapped class attributes) as column - arguments. - - .. change:: - :tags: schema - :tickets: - - Column with no name (as in declarative) won't raise a - NoneType error when its string output is requested - (such as in a stack trace). - - .. change:: - :tags: schema - :tickets: 1278 - - Fixed bug when overriding a Column with a ForeignKey - on a reflected table, where derived columns (i.e. the - "virtual" columns of a select, etc.) would inadvertently - call upon schema-level cleanup logic intended only - for the original column. - - .. change:: - :tags: declarative - :tickets: - - Can now specify Column objects on subclasses which have no - table of their own (i.e. use single table inheritance). - The columns will be appended to the base table, but only - mapped by the subclass. - - .. change:: - :tags: declarative - :tickets: - - For both joined and single inheriting subclasses, the subclass - will only map those columns which are already mapped on the - superclass and those explicit on the subclass. Other - columns that are present on the `Table` will be excluded - from the mapping by default, which can be disabled - by passing a blank `exclude_properties` collection to the - `__mapper_args__`. This is so that single-inheriting - classes which define their own columns are the only classes - to map those columns. The effect is actually a more organized - mapping than you'd normally get with explicit `mapper()` - calls unless you set up the `exclude_properties` arguments - explicitly. - - .. change:: - :tags: declarative - :tickets: - - It's an error to add new Column objects to a declarative class - that specified an existing table using __table__. - - .. change:: - :tags: mysql - :tickets: - - Added the missing keywords from MySQL 4.1 so they get escaped - properly. - - .. change:: - :tags: mssql - :tickets: 1280 - - Corrected handling of large decimal values with more robust - tests. Removed string manipulation on floats. - - .. change:: - :tags: mssql - :tickets: - - Modified the do_begin handling in mssql to use the Cursor not - the Connection so it is DBAPI compatible. - - .. change:: - :tags: mssql - :tickets: - - Corrected SAVEPOINT support on adodbapi by changing the - handling of savepoint_release, which is unsupported on mssql. - -.. changelog:: - :version: 0.5.0 - :released: Tue Jan 06 2009 - - .. change:: - :tags: general - :tickets: - - Documentation has been converted to Sphinx. In particular, - the generated API documentation has been constructed into a - full blown "API Reference" section which organizes editorial - documentation combined with generated docstrings. Cross - linking between sections and API docs are vastly improved, a - javascript-powered search feature is provided, and a full - index of all classes, functions and members is provided. - - .. change:: - :tags: general - :tickets: - - setup.py now imports setuptools only optionally. If not - present, distutils is used. The new "pip" installer is - recommended over easy_install as it installs in a more - simplified way. - - .. change:: - :tags: general - :tickets: - - added an extremely basic illustration of a PostGIS integration - to the examples folder. - - .. change:: - :tags: orm - :tickets: - - Query.with_polymorphic() now accepts a third argument - "discriminator" which will replace the value of - mapper.polymorphic_on for that query. Mappers themselves no - longer require polymorphic_on to be set, even if the mapper - has a polymorphic_identity. When not set, the mapper will - load non-polymorphically by default. Together, these two - features allow a non-polymorphic concrete inheritance setup to - use polymorphic loading on a per-query basis, since concrete - setups are prone to many issues when used polymorphically in - all cases. - - .. change:: - :tags: orm - :tickets: - - dynamic_loader accepts a query_class= to customize the Query - classes used for both the dynamic collection and the queries - built from it. - - .. change:: - :tags: orm - :tickets: 1079 - - query.order_by() accepts None which will remove any pending - order_by state from the query, as well as cancel out any - mapper/relation configured ordering. This is primarily useful - for overriding the ordering specified on a dynamic_loader(). - - .. change:: - :tags: sql - :tickets: 935 - - RowProxy objects can be used in place of dictionary arguments - sent to connection.execute() and friends. - - .. change:: - :tags: dialect - :tickets: - - Added a new description_encoding attribute on the dialect that - is used for encoding the column name when processing the - metadata. This usually defaults to utf-8. - - .. change:: - :tags: mssql - :tickets: - - Added in a new MSGenericBinary type. This maps to the Binary - type so it can implement the specialized behavior of treating - length specified types as fixed-width Binary types and - non-length types as an unbound variable length Binary type. - - .. change:: - :tags: mssql - :tickets: 1249 - - Added in new types: MSVarBinary and MSImage. - - .. change:: - :tags: mssql - :tickets: - - Added in the MSReal, MSNText, MSSmallDateTime, MSTime, - MSDateTimeOffset, and MSDateTime2 types - - .. change:: - :tags: sqlite - :tickets: 1266 - - Table reflection now stores the actual DefaultClause value for - the column. - - .. change:: - :tags: sqlite - :tickets: - - bugfixes, behavioral changes - - .. change:: - :tags: orm - :tickets: - - Exceptions raised during compile_mappers() are now preserved - to provide "sticky behavior" - if a hasattr() call on a - pre-compiled mapped attribute triggers a failing compile and - suppresses the exception, subsequent compilation is blocked - and the exception will be reiterated on the next compile() - call. This issue occurs frequently when using declarative. - - .. change:: - :tags: orm - :tickets: - - property.of_type() is now recognized on a single-table - inheriting target, when used in the context of - prop.of_type(..).any()/has(), as well as - query.join(prop.of_type(...)). - - .. change:: - :tags: orm - :tickets: - - query.join() raises an error when the target of the join - doesn't match the property-based attribute - while it's - unlikely anyone is doing this, the SQLAlchemy author was - guilty of this particular loosey-goosey behavior. - - .. change:: - :tags: orm - :tickets: 1272 - - Fixed bug when using weak_instance_map=False where modified - events would not be intercepted for a flush(). - - .. change:: - :tags: orm - :tickets: 1268 - - Fixed some deep "column correspondence" issues which could - impact a Query made against a selectable containing multiple - versions of the same table, as well as unions and similar - which contained the same table columns in different column - positions at different levels. - - .. change:: - :tags: orm - :tickets: - - Custom comparator classes used in conjunction with - column_property(), relation() etc. can define new comparison - methods on the Comparator, which will become available via - __getattr__() on the InstrumentedAttribute. In the case of - synonym() or comparable_property(), attributes are resolved - first on the user-defined descriptor, then on the user-defined - comparator. - - .. change:: - :tags: orm - :tickets: 976 - - Added ScopedSession.is_active accessor. - - .. change:: - :tags: orm - :tickets: 1262 - - Can pass mapped attributes and column objects as keys to - query.update({}). - - .. change:: - :tags: orm - :tickets: - - Mapped attributes passed to the values() of an expression - level insert() or update() will use the keys of the mapped - columns, not that of the mapped attribute. - - .. change:: - :tags: orm - :tickets: 1242 - - Corrected problem with Query.delete() and Query.update() not - working properly with bind parameters. - - .. change:: - :tags: orm - :tickets: - - Query.select_from(), from_statement() ensure that the given - argument is a FromClause, or Text/Select/Union, respectively. - - .. change:: - :tags: orm - :tickets: 1253 - - Query() can be passed a "composite" attribute as a column - expression and it will be expanded. Somewhat related to. - - .. change:: - :tags: orm - :tickets: - - Query() is a little more robust when passed various column - expressions such as strings, clauselists, text() constructs - (which may mean it just raises an error more nicely). - - .. change:: - :tags: orm - :tickets: - - first() works as expected with Query.from_statement(). - - .. change:: - :tags: orm - :tickets: - - Fixed bug introduced in 0.5rc4 involving eager loading not - functioning for properties which were added to a mapper - post-compile using add_property() or equivalent. - - .. change:: - :tags: orm - :tickets: - - Fixed bug where many-to-many relation() with viewonly=True - would not correctly reference the link between - secondary->remote. - - .. change:: - :tags: orm - :tickets: 1232 - - Duplicate items in a list-based collection will be maintained - when issuing INSERTs to a "secondary" table in a many-to-many - relation. Assuming the m2m table has a unique or primary key - constraint on it, this will raise the expected constraint - violation instead of silently dropping the duplicate - entries. Note that the old behavior remains for a one-to-many - relation since collection entries in that case don't result in - INSERT statements and SQLA doesn't manually police - collections. - - .. change:: - :tags: orm - :tickets: - - Query.add_column() can accept FromClause objects in the same - manner as session.query() can. - - .. change:: - :tags: orm - :tickets: - - Comparison of many-to-one relation to NULL is properly - converted to IS NOT NULL based on not_(). - - .. change:: - :tags: orm - :tickets: 1087 - - Extra checks added to ensure explicit - primaryjoin/secondaryjoin are ClauseElement instances, to - prevent more confusing errors later on. - - .. change:: - :tags: orm - :tickets: 1236 - - Improved mapper() check for non-class classes. - - .. change:: - :tags: orm - :tickets: 5051 - - comparator_factory argument is now documented and supported by - all MapperProperty types, including column_property(), - relation(), backref(), and synonym(). - - .. change:: - :tags: orm - :tickets: - - Changed the name of PropertyLoader to RelationProperty, to be - consistent with all the other names. PropertyLoader is still - present as a synonym. - - .. change:: - :tags: orm - :tickets: 1099, 1228 - - fixed "double iter()" call causing bus errors in shard API, - removed errant result.close() left over from the 0.4 - version. - - .. change:: - :tags: orm - :tickets: - - made Session.merge cascades not trigger autoflush. Fixes - merged instances getting prematurely inserted with missing - values. - - .. change:: - :tags: orm - :tickets: - - Two fixes to help prevent out-of-band columns from being - rendered in polymorphic_union inheritance scenarios (which - then causes extra tables to be rendered in the FROM clause - causing cartesian products): - - - improvements to "column adaption" for a->b->c inheritance - situations to better locate columns that are related to - one another via multiple levels of indirection, rather - than rendering the non-adapted column. - - - the "polymorphic discriminator" column is only rendered - for the actual mapper being queried against. The column - won't be "pulled in" from a subclass or superclass mapper - since it's not needed. - - .. change:: - :tags: orm - :tickets: 1072 - - Fixed shard_id argument on ShardedSession.execute(). - - .. change:: - :tags: sql - :tickets: 1256 - - Columns can again contain percent signs within their - names. - - .. change:: - :tags: sql - :tickets: - - sqlalchemy.sql.expression.Function is now a public class. It - can be subclassed to provide user-defined SQL functions in an - imperative style, including with pre-established behaviors. - The postgis.py example illustrates one usage of this. - - .. change:: - :tags: sql - :tickets: - - PickleType now favors == comparison by default, if the - incoming object (such as a dict) implements __eq__(). If the - object does not implement __eq__() and mutable=True, a - deprecation warning is raised. - - .. change:: - :tags: sql - :tickets: 1215 - - Fixed the import weirdness in sqlalchemy.sql to not export - __names__. - - .. change:: - :tags: sql - :tickets: 1238 - - Using the same ForeignKey object repeatedly raises an error - instead of silently failing later. - - .. change:: - :tags: sql - :tickets: - - Added NotImplementedError for params() method on - Insert/Update/Delete constructs. These items currently don't - support this functionality, which also would be a little - misleading compared to values(). - - .. change:: - :tags: sql - :tickets: 650 - - Reflected foreign keys will properly locate their referenced - column, even if the column was given a "key" attribute - different from the reflected name. This is achieved via a new - flag on ForeignKey/ForeignKeyConstraint called "link_to_name", - if True means the given name is the referred-to column's name, - not its assigned key. - - .. change:: - :tags: sql - :tickets: 1253 - - select() can accept a ClauseList as a column in the same way - as a Table or other selectable and the interior expressions - will be used as column elements. - - .. change:: - :tags: sql - :tickets: - - the "passive" flag on session.is_modified() is correctly - propagated to the attribute manager. - - .. change:: - :tags: sql - :tickets: - - union() and union_all() will not whack any order_by() that has - been applied to the select()s inside. If you union() a - select() with order_by() (presumably to support LIMIT/OFFSET), - you should also call self_group() on it to apply parenthesis. - - .. change:: - :tags: engine/pool - :tickets: 1246 - - Connection.invalidate() checks for closed status to avoid - attribute errors. - - .. change:: - :tags: engine/pool - :tickets: 1094 - - NullPool supports reconnect on failure behavior. - - .. change:: - :tags: engine/pool - :tickets: 799 - - Added a mutex for the initial pool creation when using - pool.manage(dbapi). This prevents a minor case of "dogpile" - behavior which would otherwise occur upon a heavy load - startup. - - .. change:: - :tags: engine/pool - :tickets: - - _execute_clauseelement() goes back to being a private method. - Subclassing Connection is not needed now that ConnectionProxy - is available. - - .. change:: - :tags: documentation - :tickets: 1149, 1200 - - Tickets. - - .. change:: - :tags: documentation - :tickets: - - Added note about create_session() defaults. - - .. change:: - :tags: documentation - :tickets: - - Added section about metadata.reflect(). - - .. change:: - :tags: documentation - :tickets: - - Updated `TypeDecorator` section. - - .. change:: - :tags: documentation - :tickets: - - Rewrote the "threadlocal" strategy section of the docs due to - recent confusion over this feature. - - .. change:: - :tags: documentation - :tickets: - - Removed badly out of date 'polymorphic_fetch' and - 'select_table' docs from inheritance, reworked the second half - of "joined table inheritance". - - .. change:: - :tags: documentation - :tickets: - - Documented `comparator_factory` kwarg, added new doc section - "Custom Comparators". - - .. change:: - :tags: mssql - :tickets: 1254 - - Refactored the Date/Time types. The ``smalldatetime`` data - type no longer truncates to a date only, and will now be - mapped to the MSSmallDateTime type. - - .. change:: - :tags: mssql - :tickets: - - Corrected an issue with Numerics to accept an int. - - .. change:: - :tags: mssql - :tickets: - - Mapped ``char_length`` to the ``LEN()`` function. - - .. change:: - :tags: mssql - :tickets: - - If an ``INSERT`` includes a subselect the ``INSERT`` is - converted from an ``INSERT INTO VALUES`` construct to a - ``INSERT INTO SELECT`` construct. - - .. change:: - :tags: mssql - :tickets: - - If the column is part of a ``primary_key`` it will be ``NOT - NULL`` since MSSQL doesn't allow ``NULL`` in primary_key - columns. - - .. change:: - :tags: mssql - :tickets: 1249 - - ``MSBinary`` now returns a ``BINARY`` instead of an - ``IMAGE``. This is a backwards incompatible change in that - ``BINARY`` is a fixed length data type whereas ``IMAGE`` is a - variable length data type. - - .. change:: - :tags: mssql - :tickets: 1258 - - ``get_default_schema_name`` is now reflected from the database - based on the user's default schema. This only works with MSSQL - 2005 and later. - - .. change:: - :tags: mssql - :tickets: 1248 - - Added collation support through the use of a new collation - argument. This is supported on the following types: char, - nchar, varchar, nvarchar, text, ntext. - - .. change:: - :tags: mssql - :tickets: - - Changes to the connection string parameters favor DSN as the - default specification for pyodbc. See the mssql.py docstring - for detailed usage instructions. - - .. change:: - :tags: mssql - :tickets: - - Added experimental support of savepoints. It currently does - not work fully with sessions. - - .. change:: - :tags: mssql - :tickets: 1243 - - Support for three levels of column nullability: NULL, NOT - NULL, and the database's configured default. The default - Column configuration (nullable=True) will now generate NULL in - the DDL. Previously no specification was emitted and the - database default would take effect (usually NULL, but not - always). To explicitly request the database default, - configure columns with nullable=None and no specification will - be emitted in DDL. This is backwards incompatible - behavior. - - .. change:: - :tags: postgres - :tickets: 1267 - - "%" signs in text() constructs are automatically escaped to - "%%". Because of the backwards incompatible nature of this - change, a warning is emitted if '%%' is detected in the - string. - - .. change:: - :tags: postgres - :tickets: - - Calling alias.execute() in conjunction with - server_side_cursors won't raise AttributeError. - - .. change:: - :tags: postgres - :tickets: 714 - - Added Index reflection support to PostgreSQL, using a great - patch we long neglected, submitted by Ken - Kuhlman. - - .. change:: - :tags: oracle - :tickets: - - Adjusted the format of create_xid() to repair two-phase - commit. We now have field reports of Oracle two-phase commit - working properly with this change. - - .. change:: - :tags: oracle - :tickets: 1233 - - Added OracleNVarchar type, produces NVARCHAR2, and also - subclasses Unicode so that convert_unicode=True by default. - NVARCHAR2 reflects into this type automatically so these - columns pass unicode on a reflected table with no explicit - convert_unicode=True flags. - - .. change:: - :tags: oracle - :tickets: 1265 - - Fixed bug which was preventing out params of certain types - from being received; thanks a ton to huddlej at wwu.edu ! - - .. change:: - :tags: mysql - :tickets: - - "%" signs in text() constructs are automatically escaped to - "%%". Because of the backwards incompatible nature of this - change, a warning is emitted if '%%' is detected in the - string. - - .. change:: - :tags: mysql - :tickets: 1241 - - Fixed bug in exception raise when FK columns not present - during reflection. - - .. change:: - :tags: mysql - :tickets: - - Fixed bug involving reflection of a remote-schema table with a - foreign key ref to another table in that schema. - - .. change:: - :tags: associationproxy - :tickets: - - The association proxy properties are make themselves available - at the class level, e.g. MyClass.aproxy. Previously this - evaluated to None. - - .. change:: - :tags: declarative - :tickets: - - The full list of arguments accepted as string by backref() - includes 'primaryjoin', 'secondaryjoin', 'secondary', - 'foreign_keys', 'remote_side', 'order_by'. - -.. changelog:: - :version: 0.5.0rc4 - :released: Fri Nov 14 2008 - - .. change:: - :tags: orm - :tickets: - - Query.count() has been enhanced to do the "right thing" in a - wider variety of cases. It can now count multiple-entity - queries, as well as column-based queries. Note that this means - if you say query(A, B).count() without any joining criterion, - it's going to count the cartesian product of A*B. Any query - which is against column-based entities will automatically - issue "SELECT count(1) FROM (SELECT...)" so that the real - rowcount is returned, meaning a query such as - query(func.count(A.name)).count() will return a value of one, - since that query would return one row. - - .. change:: - :tags: orm - :tickets: - - Lots of performance tuning. A rough guesstimate over various - ORM operations places it 10% faster over 0.5.0rc3, 25-30% over - 0.4.8. - - .. change:: - :tags: orm - :tickets: - - bugfixes and behavioral changes - - .. change:: - :tags: general - :tickets: - - global "propigate"->"propagate" change. - - .. change:: - :tags: orm - :tickets: - - Adjustments to the enhanced garbage collection on - InstanceState to better guard against errors due to lost - state. - - .. change:: - :tags: orm - :tickets: 1220 - - Query.get() returns a more informative error message when - executed against multiple entities. - - .. change:: - :tags: orm - :tickets: 1140, 1221 - - Restored NotImplementedError on Cls.relation.in_() - - .. change:: - :tags: orm - :tickets: 1226 - - Fixed PendingDeprecationWarning involving order_by parameter - on relation(). - - .. change:: - :tags: sql - :tickets: - - Removed the 'properties' attribute of the Connection object, - Connection.info should be used. - - .. change:: - :tags: sql - :tickets: - - Restored "active rowcount" fetch before ResultProxy autocloses - the cursor. This was removed in 0.5rc3. - - .. change:: - :tags: sql - :tickets: - - Rearranged the `load_dialect_impl()` method in `TypeDecorator` - such that it will take effect even if the user-defined - `TypeDecorator` uses another `TypeDecorator` as its impl. - - .. change:: - :tags: access - :tickets: - - Added support for Currency type. - - .. change:: - :tags: access - :tickets: 1017 - - Functions were not return their result. - - .. change:: - :tags: access - :tickets: 1017 - - Corrected problem with joins. Access only support LEFT OUTER - or INNER not just JOIN by itself. - - .. change:: - :tags: mssql - :tickets: - - Lots of cleanup and fixes to correct problems with limit and - offset. - - .. change:: - :tags: mssql - :tickets: - - Correct situation where subqueries as part of a binary - expression need to be translated to use the IN and NOT IN - syntax. - - .. change:: - :tags: mssql - :tickets: 1216 - - Fixed E Notation issue that prevented the ability to insert - decimal values less than 1E-6. - - .. change:: - :tags: mssql - :tickets: 1217 - - Corrected problems with reflection when dealing with schemas, - particularly when those schemas are the default - schema. - - .. change:: - :tags: mssql - :tickets: - - Corrected problem with casting a zero length item to a - varchar. It now correctly adjusts the CAST. - - .. change:: - :tags: ext - :tickets: - - Can now use a custom "inherit_condition" in __mapper_args__ - when using declarative. - - .. change:: - :tags: ext - :tickets: - - fixed string-based "remote_side", "order_by" and others not - propagating correctly when used in backref(). - -.. changelog:: - :version: 0.5.0rc3 - :released: Fri Nov 07 2008 - - .. change:: - :tags: orm - :tickets: - - Added two new hooks to SessionExtension: after_bulk_delete() - and after_bulk_update(). after_bulk_delete() is called after - a bulk delete() operation on a query. after_bulk_update() is - called after a bulk update() operation on a query. - - .. change:: - :tags: sql - :tickets: - - SQL compiler optimizations and complexity reduction. The call - count for compiling a typical select() construct is 20% less - versus 0.5.0rc2. - - .. change:: - :tags: sql - :tickets: 1211 - - Dialects can now generate label names of adjustable - length. Pass in the argument "label_length=" to - create_engine() to adjust how many characters max will be - present in dynamically generated column labels, i.e. - "somecolumn AS somelabel". Any value less than 6 will result - in a label of minimal size, consisting of an underscore and a - numeric counter. The compiler uses the value of - dialect.max_identifier_length as a default. - - .. change:: - :tags: ext - :tickets: - - Added a new extension sqlalchemy.ext.serializer. Provides - Serializer/Deserializer "classes" which mirror - Pickle/Unpickle, as well as dumps() and loads(). This - serializer implements an "external object" pickler which keeps - key context-sensitive objects, including engines, sessions, - metadata, Tables/Columns, and mappers, outside of the pickle - stream, and can later restore the pickle using any - engine/metadata/session provider. This is used not for - pickling regular object instances, which are pickleable - without any special logic, but for pickling expression objects - and full Query objects, such that all mapper/engine/session - dependencies can be restored at unpickle time. - - .. change:: - :tags: oracle - :tickets: - - Wrote a docstring for Oracle dialect. Apparently that Ohloh - "few source code comments" label is starting to string :). - - .. change:: - :tags: oracle - :tickets: 536 - - Removed FIRST_ROWS() optimize flag when using LIMIT/OFFSET, - can be re-enabled with optimize_limits=True create_engine() - flag. - - .. change:: - :tags: oracle - :tickets: - - bugfixes and behavioral changes - - .. change:: - :tags: orm - :tickets: - - "not equals" comparisons of simple many-to-one relation to an - instance will not drop into an EXISTS clause and will compare - foreign key columns instead. - - .. change:: - :tags: orm - :tickets: - - Removed not-really-working use cases of comparing a collection - to an iterable. Use contains() to test for collection - membership. - - .. change:: - :tags: orm - :tickets: 1171 - - Improved the behavior of aliased() objects such that they more - accurately adapt the expressions generated, which helps - particularly with self-referential comparisons. - - .. change:: - :tags: orm - :tickets: - - Fixed bug involving primaryjoin/secondaryjoin conditions - constructed from class-bound attributes (as often occurs when - using declarative), which later would be inappropriately - aliased by Query, particularly with the various EXISTS based - comparators. - - .. change:: - :tags: orm - :tickets: - - Fixed bug when using multiple query.join() with an - aliased-bound descriptor which would lose the left alias. - - .. change:: - :tags: orm - :tickets: - - Improved weakref identity map memory management to no longer - require mutexing, resurrects garbage collected instance on a - lazy basis for an InstanceState with pending changes. - - .. change:: - :tags: orm - :tickets: - - InstanceState object now removes circular references to itself - upon disposal to keep it outside of cyclic garbage collection. - - .. change:: - :tags: orm - :tickets: - - relation() won't hide unrelated ForeignKey errors inside of - the "please specify primaryjoin" message when determining join - condition. - - .. change:: - :tags: orm - :tickets: 1218 - - Fixed bug in Query involving order_by() in conjunction with - multiple aliases of the same class (will add tests in) - - .. change:: - :tags: orm - :tickets: - - When using Query.join() with an explicit clause for the ON - clause, the clause will be aliased in terms of the left side - of the join, allowing scenarios like query(Source). - from_self().join((Dest, Source.id==Dest.source_id)) to work - properly. - - .. change:: - :tags: orm - :tickets: - - polymorphic_union() function respects the "key" of each Column - if they differ from the column's name. - - .. change:: - :tags: orm - :tickets: 1183 - - Repaired support for "passive-deletes" on a many-to-one - relation() with "delete" cascade. - - .. change:: - :tags: orm - :tickets: 1213 - - Fixed bug in composite types which prevented a primary-key - composite type from being mutated. - - .. change:: - :tags: orm - :tickets: 1202 - - Added more granularity to internal attribute access, such that - cascade and flush operations will not initialize unloaded - attributes and collections, leaving them intact for a - lazy-load later on. Backref events still initialize attributes - and collections for pending instances. - - .. change:: - :tags: sql - :tickets: 1212 - - Simplified the check for ResultProxy "autoclose without - results" to be based solely on presence of - cursor.description. All the regexp-based guessing about - statements returning rows has been removed. - - .. change:: - :tags: sql - :tickets: 1194 - - Direct execution of a union() construct will properly set up - result-row processing. - - .. change:: - :tags: sql - :tickets: - - The internal notion of an "OID" or "ROWID" column has been - removed. It's basically not used by any dialect, and the - possibility of its usage with psycopg2's cursor.lastrowid is - basically gone now that INSERT..RETURNING is available. - - .. change:: - :tags: sql - :tickets: - - Removed "default_order_by()" method on all FromClause objects. - - .. change:: - :tags: sql - :tickets: - - Repaired the table.tometadata() method so that a passed-in - schema argument is propagated to ForeignKey constructs. - - .. change:: - :tags: sql - :tickets: - - Slightly changed behavior of IN operator for comparing to - empty collections. Now results in inequality comparison - against self. More portable, but breaks with stored procedures - that aren't pure functions. - - .. change:: - :tags: oracle - :tickets: - - Setting the auto_convert_lobs to False on create_engine() will - also instruct the OracleBinary type to return the cx_oracle - LOB object unchanged. - - .. change:: - :tags: mysql - :tickets: - - Fixed foreign key reflection in the edge case where a Table's - explicit schema= is the same as the schema (database) the - connection is attached to. - - .. change:: - :tags: mysql - :tickets: - - No longer expects include_columns in table reflection to be - lower case. - - .. change:: - :tags: ext - :tickets: 1174 - - Fixed bug preventing declarative-bound "column" objects from - being used in column_mapped_collection(). - - .. change:: - :tags: misc - :tickets: 1077 - - util.flatten_iterator() func doesn't interpret strings with - __iter__() methods as iterators, such as in pypy. - -.. changelog:: - :version: 0.5.0rc2 - :released: Sun Oct 12 2008 - - .. change:: - :tags: orm - :tickets: - - Fixed bug involving read/write relation()s that contain - literal or other non-column expressions within their - primaryjoin condition equated to a foreign key column. - - .. change:: - :tags: orm - :tickets: - - "non-batch" mode in mapper(), a feature which allows mapper - extension methods to be called as each instance is - updated/inserted, now honors the insert order of the objects - given. - - .. change:: - :tags: orm - :tickets: - - Fixed RLock-related bug in mapper which could deadlock upon - reentrant mapper compile() calls, something that occurs when - using declarative constructs inside of ForeignKey objects. - - .. change:: - :tags: orm - :tickets: - - ScopedSession.query_property now accepts a query_cls factory, - overriding the session's configured query_cls. - - .. change:: - :tags: orm - :tickets: - - Fixed shared state bug interfering with ScopedSession.mapper's - ability to apply default __init__ implementations on object - subclasses. - - .. change:: - :tags: orm - :tickets: 1177 - - Fixed up slices on Query (i.e. query[x:y]) to work properly - for zero length slices, slices with None on either end. - - .. change:: - :tags: orm - :tickets: - - Added an example illustrating Celko's "nested sets" as a - SQLA mapping. - - .. change:: - :tags: orm - :tickets: - - contains_eager() with an alias argument works even when - the alias is embedded in a SELECT, as when sent to the - Query via query.select_from(). - - .. change:: - :tags: orm - :tickets: 1180 - - contains_eager() usage is now compatible with a Query that - also contains a regular eager load and limit/offset, in that - the columns are added to the Query-generated subquery. - - .. change:: - :tags: orm - :tickets: - - session.execute() will execute a Sequence object passed to - it (regression from 0.4). - - .. change:: - :tags: orm - :tickets: - - Removed the "raiseerror" keyword argument from object_mapper() - and class_mapper(). These functions raise in all cases - if the given class/instance is not mapped. - - .. change:: - :tags: orm - :tickets: - - Fixed session.transaction.commit() on a autocommit=False - session not starting a new transaction. - - .. change:: - :tags: orm - :tickets: - - Some adjustments to Session.identity_map's weak referencing - behavior to reduce asynchronous GC side effects. - - .. change:: - :tags: orm - :tickets: 1182 - - Adjustment to Session's post-flush accounting of newly - "clean" objects to better protect against operating on - objects as they're asynchronously gc'ed. - - .. change:: - :tags: sql - :tickets: 1074 - - column.in_(someselect) can now be used as a columns-clause - expression without the subquery bleeding into the FROM clause - - .. change:: - :tags: sqlite - :tickets: 968 - - Overhauled SQLite date/time bind/result processing to use - regular expressions and format strings, rather than - strptime/strftime, to generically support pre-1900 dates, - dates with microseconds. - - .. change:: - :tags: sqlite - :tickets: - - String's (and Unicode's, UnicodeText's, etc.) convert_unicode - logic disabled in the sqlite dialect, to adjust for pysqlite - 2.5.0's new requirement that only Python unicode objects are - accepted; - https://itsystementwicklung.de/pipermail/list-pysqlite/2008-March/000018.html - - .. change:: - :tags: mysql - :tickets: - - Temporary tables are now reflectable. - - .. change:: - :tags: oracle - :tickets: 1187 - - Oracle will detect string-based statements which contain - comments at the front before a SELECT as SELECT statements. - -.. changelog:: - :version: 0.5.0rc1 - :released: Thu Sep 11 2008 - - .. change:: - :tags: orm - :tickets: - - Query now has delete() and update(values) methods. This allows - to perform bulk deletes/updates with the Query object. - - .. change:: - :tags: orm - :tickets: - - The RowTuple object returned by Query(\*cols) now features - keynames which prefer mapped attribute names over column keys, - column keys over column names, i.e. Query(Class.foo, - Class.bar) will have names "foo" and "bar" even if those are - not the names of the underlying Column objects. Direct Column - objects such as Query(table.c.col) will return the "key" - attribute of the Column. - - .. change:: - :tags: orm - :tickets: - - Added scalar() and value() methods to Query, each return a - single scalar value. scalar() takes no arguments and is - roughly equivalent to first()[0], value() - takes a single column expression and is roughly equivalent to - values(expr).next()[0]. - - .. change:: - :tags: orm - :tickets: - - Improved the determination of the FROM clause when placing SQL - expressions in the query() list of entities. In particular - scalar subqueries should not "leak" their inner FROM objects - out into the enclosing query. - - .. change:: - :tags: orm - :tickets: - - Joins along a relation() from a mapped class to a mapped - subclass, where the mapped subclass is configured with single - table inheritance, will include an IN clause which limits the - subtypes of the joined class to those requested, within the ON - clause of the join. This takes effect for eager load joins as - well as query.join(). Note that in some scenarios the IN - clause will appear in the WHERE clause of the query as well - since this discrimination has multiple trigger points. - - .. change:: - :tags: orm - :tickets: - - AttributeExtension has been refined such that the event - is fired before the mutation actually occurs. Additionally, - the append() and set() methods must now return the given value, - which is used as the value to be used in the mutation operation. - This allows creation of validating AttributeListeners which - raise before the action actually occurs, and which can change - the given value into something else before its used. - - .. change:: - :tags: orm - :tickets: - - column_property(), composite_property(), and relation() now - accept a single or list of AttributeExtensions using the - "extension" keyword argument. - - .. change:: - :tags: orm - :tickets: - - query.order_by().get() silently drops the "ORDER BY" from - the query issued by GET but does not raise an exception. - - .. change:: - :tags: orm - :tickets: - - Added a Validator AttributeExtension, as well as a - @validates decorator which is used in a similar fashion - as @reconstructor, and marks a method as validating - one or more mapped attributes. - - .. change:: - :tags: orm - :tickets: 1140 - - class.someprop.in_() raises NotImplementedError pending the - implementation of "in\_" for relation - - .. change:: - :tags: orm - :tickets: 1127 - - Fixed primary key update for many-to-many collections where - the collection had not been loaded yet - - .. change:: - :tags: orm - :tickets: - - Fixed bug whereby deferred() columns with a group in conjunction - with an otherwise unrelated synonym() would produce - an AttributeError during deferred load. - - .. change:: - :tags: orm - :tickets: 1128 - - The before_flush() hook on SessionExtension takes place before - the list of new/dirty/deleted is calculated for the final - time, allowing routines within before_flush() to further - change the state of the Session before the flush proceeds. - - .. change:: - :tags: orm - :tickets: - - The "extension" argument to Session and others can now - optionally be a list, supporting events sent to multiple - SessionExtension instances. Session places SessionExtensions - in Session.extensions. - - .. change:: - :tags: orm - :tickets: - - Reentrant calls to flush() raise an error. This also serves - as a rudimentary, but not foolproof, check against concurrent - calls to Session.flush(). - - .. change:: - :tags: orm - :tickets: - - Improved the behavior of query.join() when joining to - joined-table inheritance subclasses, using explicit join - criteria (i.e. not on a relation). - - .. change:: - :tags: orm - :tickets: - - @orm.attributes.reconstitute and - MapperExtension.reconstitute have been renamed to - @orm.reconstructor and MapperExtension.reconstruct_instance - - .. change:: - :tags: orm - :tickets: 1129 - - Fixed @reconstructor hook for subclasses which inherit from a - base class. - - .. change:: - :tags: orm - :tickets: 1132 - - The composite() property type now supports a - __set_composite_values__() method on the composite class which - is required if the class represents state using attribute - names other than the column's keynames; default-generated - values now get populated properly upon flush. Also, - composites with attributes set to None compare correctly. - - .. change:: - :tags: orm - :tickets: - - The 3-tuple of iterables returned by attributes.get_history() - may now be a mix of lists and tuples. (Previously members - were always lists.) - - .. change:: - :tags: orm - :tickets: 1151 - - Fixed bug whereby changing a primary key attribute on an - entity where the attribute's previous value had been expired - would produce an error upon flush(). - - .. change:: - :tags: orm - :tickets: - - Fixed custom instrumentation bug whereby get_instance_dict() - was not called for newly constructed instances not loaded - by the ORM. - - .. change:: - :tags: orm - :tickets: 1150 - - Session.delete() adds the given object to the session if - not already present. This was a regression bug from 0.4. - - .. change:: - :tags: orm - :tickets: - - The `echo_uow` flag on `Session` is deprecated, and unit-of-work - logging is now application-level only, not per-session level. - - .. change:: - :tags: orm - :tickets: 1153 - - Removed conflicting `contains()` operator from - `InstrumentedAttribute` which didn't accept `escape` kwaarg. - - .. change:: - :tags: declarative - :tickets: 1161 - - Fixed bug whereby mapper couldn't initialize if a composite - primary key referenced another table that was not defined - yet. - - .. change:: - :tags: declarative - :tickets: - - Fixed exception throw which would occur when string-based - primaryjoin condition was used in conjunction with backref. - - .. change:: - :tags: schema - :tickets: 1033 - - Added "sorted_tables" accessor to MetaData, which returns - Table objects sorted in order of dependency as a list. - This deprecates the MetaData.table_iterator() method. - The "reverse=False" keyword argument has also been - removed from util.sort_tables(); use the Python - 'reversed' function to reverse the results. - - .. change:: - :tags: schema - :tickets: - - The 'length' argument to all Numeric types has been renamed - to 'scale'. 'length' is deprecated and is still accepted - with a warning. - - .. change:: - :tags: schema - :tickets: - - Dropped 0.3-compatibility for user defined types - (convert_result_value, convert_bind_param). - - .. change:: - :tags: sql - :tickets: 1068 - - Temporarily rolled back the "ORDER BY" enhancement from. This feature is on hold pending further - development. - - .. change:: - :tags: sql - :tickets: - - The exists() construct won't "export" its contained list - of elements as FROM clauses, allowing them to be used more - effectively in the columns clause of a SELECT. - - .. change:: - :tags: sql - :tickets: 798 - - and_() and or_() now generate a ColumnElement, allowing - boolean expressions as result columns, i.e. - select([and_(1, 0)]). - - .. change:: - :tags: sql - :tickets: - - Bind params now subclass ColumnElement which allows them to be - selectable by orm.query (they already had most ColumnElement - semantics). - - .. change:: - :tags: sql - :tickets: - - Added select_from() method to exists() construct, which becomes - more and more compatible with a regular select(). - - .. change:: - :tags: sql - :tickets: 1160 - - Added func.min(), func.max(), func.sum() as "generic functions", - which basically allows for their return type to be determined - automatically. Helps with dates on SQLite, decimal types, - others. - - .. change:: - :tags: sql - :tickets: - - added decimal.Decimal as an "auto-detect" type; bind parameters - and generic functions will set their type to Numeric when a - Decimal is used. - - .. change:: - :tags: mysql - :tickets: - - The 'length' argument to MSInteger, MSBigInteger, MSTinyInteger, - MSSmallInteger and MSYear has been renamed to 'display_width'. - - .. change:: - :tags: mysql - :tickets: 1146 - - Added MSMediumInteger type. - - .. change:: - :tags: mysql - :tickets: - - the function func.utc_timestamp() compiles to UTC_TIMESTAMP, without - the parenthesis, which seem to get in the way when using in - conjunction with executemany(). - - .. change:: - :tags: oracle - :tickets: 536 - - limit/offset no longer uses ROW NUMBER OVER to limit rows, - and instead uses subqueries in conjunction with a special - Oracle optimization comment. Allows LIMIT/OFFSET to work - in conjunction with DISTINCT. - - .. change:: - :tags: oracle - :tickets: 1155 - - has_sequence() now takes the current "schema" argument into - account - - .. change:: - :tags: oracle - :tickets: 1121 - - added BFILE to reflected type names - -.. changelog:: - :version: 0.5.0beta3 - :released: Mon Aug 04 2008 - - .. change:: - :tags: orm - :tickets: - - The "entity_name" feature of SQLAlchemy mappers has been - removed. For rationale, see https://tinyurl.com/6nm2ne - - .. change:: - :tags: orm - :tickets: - - the "autoexpire" flag on Session, sessionmaker(), and - scoped_session() has been renamed to "expire_on_commit". It - does not affect the expiration behavior of rollback(). - - .. change:: - :tags: orm - :tickets: - - fixed endless loop bug which could occur within a mapper's - deferred load of inherited attributes. - - .. change:: - :tags: orm - :tickets: - - a legacy-support flag "_enable_transaction_accounting" flag - added to Session which when False, disables all - transaction-level object accounting, including expire on - rollback, expire on commit, new/deleted list maintenance, and - autoflush on begin. - - .. change:: - :tags: orm - :tickets: - - The 'cascade' parameter to relation() accepts None as a value, - which is equivalent to no cascades. - - .. change:: - :tags: orm - :tickets: - - A critical fix to dynamic relations allows the "modified" - history to be properly cleared after a flush(). - - .. change:: - :tags: orm - :tickets: - - user-defined @properties on a class are detected and left in - place during mapper initialization. This means that a - table-bound column of the same name will not be mapped at all - if a @property is in the way (and the column is not remapped - to a different name), nor will an instrumented attribute from - an inherited class be applied. The same rules apply for names - excluded using the include_properties/exclude_properties - collections. - - .. change:: - :tags: orm - :tickets: - - Added a new SessionExtension hook called after_attach(). This - is called at the point of attachment for objects via add(), - add_all(), delete(), and merge(). - - .. change:: - :tags: orm - :tickets: 1111 - - A mapper which inherits from another, when inheriting the - columns of its inherited mapper, will use any reassigned - property names specified in that inheriting mapper. - Previously, if "Base" had reassigned "base_id" to the name - "id", "SubBase(Base)" would still get an attribute called - "base_id". This could be worked around by explicitly stating - the column in each submapper as well but this is fairly - unworkable and also impossible when using declarative. - - .. change:: - :tags: orm - :tickets: - - Fixed a series of potential race conditions in Session whereby - asynchronous GC could remove unmodified, no longer referenced - items from the session as they were present in a list of items - to be processed, typically during session.expunge_all() and - dependent methods. - - .. change:: - :tags: orm - :tickets: - - Some improvements to the _CompileOnAttr mechanism which should - reduce the probability of "Attribute x was not replaced during - compile" warnings. (this generally applies to SQLA hackers, - like Elixir devs). - - .. change:: - :tags: orm - :tickets: - - Fixed bug whereby the "unsaved, pending instance" FlushError - raised for a pending orphan would not take superclass mappers - into account when generating the list of relations responsible - for the error. - - .. change:: - :tags: sql - :tickets: - - func.count() with no arguments renders as COUNT(*), equivalent - to func.count(text('*')). - - .. change:: - :tags: sql - :tickets: 1068 - - simple label names in ORDER BY expressions render as - themselves, and not as a re-statement of their corresponding - expression. This feature is currently enabled only for - SQLite, MySQL, and PostgreSQL. It can be enabled on other - dialects as each is shown to support this - behavior. - - .. change:: - :tags: ext - :tickets: - - Class-bound attributes sent as arguments to relation()'s - remote_side and foreign_keys parameters are now accepted, - allowing them to be used with declarative. Additionally fixed - bugs involving order_by being specified as a class-bound - attribute in conjunction with eager loading. - - .. change:: - :tags: ext - :tickets: - - declarative initialization of Columns adjusted so that - non-renamed columns initialize in the same way as a non - declarative mapper. This allows an inheriting mapper to set - up its same-named "id" columns in particular such that the - parent "id" column is favored over the child column, reducing - database round trips when this value is requested. - - .. change:: - :tags: mysql - :tickets: 1110 - - Quoting of MSEnum values for use in CREATE TABLE is now - optional & will be quoted on demand as required. (Quoting was - always optional for use with existing tables.) - -.. changelog:: - :version: 0.5.0beta2 - :released: Mon Jul 14 2008 - - .. change:: - :tags: orm - :tickets: 870 - - In addition to expired attributes, deferred attributes also - load if their data is present in the result set. - - .. change:: - :tags: orm - :tickets: - - session.refresh() raises an informative error message if the - list of attributes does not include any column-based - attributes. - - .. change:: - :tags: orm - :tickets: - - query() raises an informative error message if no columns or - mappers are specified. - - .. change:: - :tags: orm - :tickets: - - lazy loaders now trigger autoflush before proceeding. This - allows expire() of a collection or scalar relation to function - properly in the context of autoflush. - - .. change:: - :tags: orm - :tickets: 887 - - column_property() attributes which represent SQL expressions - or columns that are not present in the mapped tables (such as - those from views) are automatically expired after an INSERT or - UPDATE, assuming they have not been locally modified, so that - they are refreshed with the most recent data upon access. - - .. change:: - :tags: orm - :tickets: 1082 - - Fixed explicit, self-referential joins between two - joined-table inheritance mappers when using query.join(cls, - aliased=True). - - .. change:: - :tags: orm - :tickets: - - Fixed query.join() when used in conjunction with a - columns-only clause and a SQL-expression ON clause in the - join. - - .. change:: - :tags: orm - :tickets: - - The "allow_column_override" flag from mapper() has been - removed. This flag is virtually always misunderstood. Its - specific functionality is available via the - include_properties/exclude_properties mapper arguments. - - .. change:: - :tags: orm - :tickets: 1066 - - Repaired `__str__()` method on Query. - - .. change:: - :tags: orm - :tickets: - - Session.bind gets used as a default even when table/mapper - specific binds are defined. - - .. change:: - :tags: schema - :tickets: 1075 - - Added prefixes option to `Table` that accepts a list of - strings to insert after CREATE in the CREATE TABLE statement. - - .. change:: - :tags: schema - :tickets: - - Unicode, UnicodeText types now set "assert_unicode" and - "convert_unicode" by default, but accept overriding - \**kwargs for these values. - - .. change:: - :tags: sql - :tickets: - - Added new match() operator that performs a full-text search. - Supported on PostgreSQL, SQLite, MySQL, MS-SQL, and Oracle - backends. - - .. change:: - :tags: sqlite - :tickets: 1090 - - Modified SQLite's representation of "microseconds" to match - the output of str(somedatetime), i.e. in that the microseconds - are represented as fractional seconds in string format. This - makes SQLA's SQLite date type compatible with datetimes that - were saved directly using Pysqlite (which just calls str()). - Note that this is incompatible with the existing microseconds - values in a SQLA 0.4 generated SQLite database file. - - To get the old behavior globally: - - from sqlalchemy.databases.sqlite import DateTimeMixin - DateTimeMixin.__legacy_microseconds__ = True - - To get the behavior on individual DateTime types: - - t = sqlite.SLDateTime() - t.__legacy_microseconds__ = True - - Then use "t" as the type on the Column. - - .. change:: - :tags: sqlite - :tickets: - - SQLite Date, DateTime, and Time types only accept Python - datetime objects now, not strings. If you'd like to format - dates as strings yourself with SQLite, use a String type. If - you'd like them to return datetime objects anyway despite - their accepting strings as input, make a TypeDecorator around - String - SQLA doesn't encourage this pattern. - - .. change:: - :tags: extensions - :tickets: 1096 - - Declarative supports a __table_args__ class variable, which is - either a dictionary, or tuple of the form (arg1, arg2, ..., - {kwarg1:value, ...}) which contains positional + kw arguments - to be passed to the Table constructor. - -.. changelog:: - :version: 0.5.0beta1 - :released: Thu Jun 12 2008 - - .. change:: - :tags: - :tickets: - - The "__init__" trigger/decorator added by mapper now attempts - to exactly mirror the argument signature of the original - __init__. The pass-through for '_sa_session' is no longer - implicit- you must allow for this keyword argument in your - constructor. - - .. change:: - :tags: - :tickets: - - ClassState is renamed to ClassManager. - - .. change:: - :tags: - :tickets: - - Classes may supply their own InstrumentationManager by - providing a __sa_instrumentation_manager__ property. - - .. change:: - :tags: - :tickets: - - Custom instrumentation may use any mechanism to associate a - ClassManager with a class and an InstanceState with an - instance. Attributes on those objects are still the default - association mechanism used by SQLAlchemy's native - instrumentation. - - .. change:: - :tags: - :tickets: - - Moved entity_name, _sa_session_id, and _instance_key from the - instance object to the instance state. These values are still - available in the old way, which is now deprecated, using - descriptors attached to the class. A deprecation warning will - be issued when accessed. - - .. change:: - :tags: - :tickets: - - The _prepare_instrumentation alias for prepare_instrumentation - has been removed. - - .. change:: - :tags: - :tickets: - - sqlalchemy.exceptions has been renamed to sqlalchemy.exc. The - module may be imported under either name. - - .. change:: - :tags: - :tickets: - - ORM-related exceptions are now defined in sqlalchemy.orm.exc. - ConcurrentModificationError, FlushError, and - UnmappedColumnError compatibility aliases are installed in - sqlalchemy.exc during the import of sqlalchemy.orm. - - .. change:: - :tags: - :tickets: - - sqlalchemy.logging has been renamed to sqlalchemy.log. - - .. change:: - :tags: - :tickets: - - The transitional sqlalchemy.log.SADeprecationWarning alias for - the warning's definition in sqlalchemy.exc has been removed. - - .. change:: - :tags: - :tickets: - - exc.AssertionError has been removed and usage replaced with - Python's built-in AssertionError. - - .. change:: - :tags: - :tickets: - - The behavior of MapperExtensions attached to multiple, - entity_name= primary mappers for a single class has been - altered. The first mapper() defined for a class is the only - mapper eligible for the MapperExtension 'instrument_class', - 'init_instance' and 'init_failed' events. This is backwards - incompatible; previously the extensions of last mapper defined - would receive these events. - - .. change:: - :tags: firebird - :tickets: - - Added support for returning values from inserts (2.0+ only), - updates and deletes (2.1+ only). - - .. change:: - :tags: general - :tickets: - - global "propigate"->"propagate" change. - - .. change:: - :tags: orm - :tickets: - - polymorphic_union() function respects the "key" of each - Column if they differ from the column's name. - - .. change:: - :tags: orm - :tickets: 1199 - - Fixed 0.4-only bug preventing composite columns - from working properly with inheriting mappers - - .. change:: - :tags: orm - :tickets: - - Fixed RLock-related bug in mapper which could deadlock upon - reentrant mapper compile() calls, something that occurs when - using declarative constructs inside of ForeignKey objects. - Ported from 0.5. - - .. change:: - :tags: orm - :tickets: 1213 - - Fixed bug in composite types which prevented a primary-key - composite type from being mutated. - - .. change:: - :tags: orm - :tickets: 976 - - Added ScopedSession.is_active accessor. - - .. change:: - :tags: orm - :tickets: 939 - - Class-bound accessor can be used as the argument to - relation() order_by. - - .. change:: - :tags: orm - :tickets: 1072 - - Fixed shard_id argument on ShardedSession.execute(). - - .. change:: - :tags: sql - :tickets: 1246 - - Connection.invalidate() checks for closed status - to avoid attribute errors. - - .. change:: - :tags: sql - :tickets: 1094 - - NullPool supports reconnect on failure behavior. - - .. change:: - :tags: sql - :tickets: 1299 - - The per-dialect cache used by TypeEngine to cache - dialect-specific types is now a WeakKeyDictionary. - This to prevent dialect objects from - being referenced forever for an application that - creates an arbitrarily large number of engines - or dialects. There is a small performance penalty - which will be resolved in 0.6. - - .. change:: - :tags: sql - :tickets: - - Fixed SQLite reflection methods so that non-present - cursor.description, which triggers an auto-cursor - close, will be detected so that no results doesn't - fail on recent versions of pysqlite which raise - an error when fetchone() called with no rows present. - - .. change:: - :tags: postgres - :tickets: 714 - - Added Index reflection support to Postgres, using a - great patch we long neglected, submitted by - Ken Kuhlman. - - .. change:: - :tags: mysql - :tickets: 1241 - - Fixed bug in exception raise when FK columns not present - during reflection. - - .. change:: - :tags: oracle - :tickets: 1265 - - Fixed bug which was preventing out params of certain types - from being received; thanks a ton to huddlej at wwu.edu ! diff --git a/doc/build/changelog/changelog_06.rst b/doc/build/changelog/changelog_06.rst deleted file mode 100644 index 739df36b230..00000000000 --- a/doc/build/changelog/changelog_06.rst +++ /dev/null @@ -1,5411 +0,0 @@ -============= -0.6 Changelog -============= - - -.. changelog:: - :version: 0.6.9 - :released: Sat May 05 2012 - - .. change:: - :tags: general - :tickets: 2279 - - Adjusted the "importlater" mechanism, which is - used internally to resolve import cycles, - such that the usage of __import__ is completed - when the import of sqlalchemy or sqlalchemy.orm - is done, thereby avoiding any usage of __import__ - after the application starts new threads, - fixes. - - .. change:: - :tags: orm - :tickets: 2197 - - Fixed bug whereby the source clause - used by query.join() would be inconsistent - if against a column expression that combined - multiple entities together. - - .. change:: - :tags: orm, bug - :tickets: 2310 - - fixed inappropriate evaluation of user-mapped - object in a boolean context within query.get(). - - .. change:: - :tags: orm - :tickets: 2228 - - Fixed bug apparent only in Python 3 whereby - sorting of persistent + pending objects during - flush would produce an illegal comparison, - if the persistent object primary key - is not a single integer. - - .. change:: - :tags: orm - :tickets: 2234 - - Fixed bug where query.join() + aliased=True - from a joined-inh structure to itself on - relationship() with join condition on the child - table would convert the lead entity into the - joined one inappropriately. - - .. change:: - :tags: orm - :tickets: 2287 - - Fixed bug whereby mapper.order_by attribute would - be ignored in the "inner" query within a - subquery eager load. . - - .. change:: - :tags: orm - :tickets: 2215 - - Fixed bug whereby if a mapped class - redefined __hash__() or __eq__() to something - non-standard, which is a supported use case - as SQLA should never consult these, - the methods would be consulted if the class - was part of a "composite" (i.e. non-single-entity) - result set. - - .. change:: - :tags: orm - :tickets: 2188 - - Fixed subtle bug that caused SQL to blow - up if: column_property() against subquery + - joinedload + LIMIT + order by the column - property() occurred. . - - .. change:: - :tags: orm - :tickets: 2207 - - The join condition produced by with_parent - as well as when using a "dynamic" relationship - against a parent will generate unique - bindparams, rather than incorrectly repeating - the same bindparam. . - - .. change:: - :tags: orm - :tickets: 2199 - - Repaired the "no statement condition" - assertion in Query which would attempt - to raise if a generative method were called - after from_statement() were called.. - - .. change:: - :tags: orm - :tickets: 1776 - - Cls.column.collate("some collation") now - works. - - .. change:: - :tags: orm, bug - :tickets: 2297 - - Fixed the error formatting raised when - a tuple is inadvertently passed to session.query(). - - .. change:: - :tags: engine - :tickets: 2317 - - Backported the fix for introduced - in 0.7.4, which ensures that the connection - is in a valid state before attempting to call - rollback()/prepare()/release() on savepoint - and two-phase transactions. - - .. change:: - :tags: sql - :tickets: 2188 - - Fixed two subtle bugs involving column - correspondence in a selectable, - one with the same labeled subquery repeated, the other - when the label has been "grouped" and - loses itself. Affects. - - .. change:: - :tags: sql - :tickets: - - Fixed bug whereby "warn on unicode" flag - would get set for the String type - when used with certain dialects. This - bug is not in 0.7. - - .. change:: - :tags: sql - :tickets: 2270 - - Fixed bug whereby with_only_columns() method of - Select would fail if a selectable were passed.. However, the FROM behavior is - still incorrect here, so you need 0.7 in - any case for this use case to be usable. - - .. change:: - :tags: schema - :tickets: - - Added an informative error message when - ForeignKeyConstraint refers to a column name in - the parent that is not found. - - .. change:: - :tags: postgresql - :tickets: 2291, 2141 - - Fixed bug related to whereby the - same modified index behavior in PG 9 affected - primary key reflection on a renamed column.. - - .. change:: - :tags: mysql - :tickets: 2186 - - Fixed OurSQL dialect to use ansi-neutral - quote symbol "'" for XA commands instead - of '"'. . - - .. change:: - :tags: mysql - :tickets: 2225 - - a CREATE TABLE will put the COLLATE option - after CHARSET, which appears to be part of - MySQL's arbitrary rules regarding if it will actually - work or not. - - .. change:: - :tags: mssql, bug - :tickets: 2269 - - Decode incoming values when retrieving - list of index names and the names of columns - within those indexes. - - .. change:: - :tags: oracle - :tickets: 2200 - - Added ORA-00028 to disconnect codes, use - cx_oracle _Error.code to get at the code,. - - .. change:: - :tags: oracle - :tickets: 2220 - - repaired the oracle.RAW type which did not - generate the correct DDL. - - .. change:: - :tags: oracle - :tickets: 2212 - - added CURRENT to reserved word list. - - .. change:: - :tags: examples - :tickets: 2266 - - Adjusted dictlike-polymorphic.py example - to apply the CAST such that it works on - PG, other databases. - -.. changelog:: - :version: 0.6.8 - :released: Sun Jun 05 2011 - - .. change:: - :tags: orm - :tickets: 2144 - - Calling query.get() against a column-based entity is - invalid, this condition now raises a deprecation warning. - - .. change:: - :tags: orm - :tickets: 2151 - - a non_primary mapper will inherit the _identity_class - of the primary mapper. This so that a non_primary - established against a class that's normally in an - inheritance mapping will produce results that are - identity-map compatible with that of the primary - mapper - - .. change:: - :tags: orm - :tickets: 2148 - - Backported 0.7's identity map implementation, which - does not use a mutex around removal. This as some users - were still getting deadlocks despite the adjustments - in 0.6.7; the 0.7 approach that doesn't use a mutex - does not appear to produce "dictionary changed size" - issues, the original rationale for the mutex. - - .. change:: - :tags: orm - :tickets: 2163 - - Fixed the error message emitted for "can't - execute syncrule for destination column 'q'; - mapper 'X' does not map this column" to - reference the correct mapper. . - - .. change:: - :tags: orm - :tickets: 2149 - - Fixed bug where determination of "self referential" - relationship would fail with no workaround - for joined-inh subclass related to itself, - or joined-inh subclass related to a subclass - of that with no cols in the sub-sub class - in the join condition. - - .. change:: - :tags: orm - :tickets: 2153 - - mapper() will ignore non-configured foreign keys - to unrelated tables when determining inherit - condition between parent and child class. - This is equivalent to behavior already - applied to declarative. Note that 0.7 has a - more comprehensive solution to this, altering - how join() itself determines an FK error. - - .. change:: - :tags: orm - :tickets: 2171 - - Fixed bug whereby mapper mapped to an anonymous - alias would fail if logging were used, due to - unescaped % sign in the alias name. - - .. change:: - :tags: orm - :tickets: 2170 - - Modify the text of the message which occurs - when the "identity" key isn't detected on - flush, to include the common cause that - the Column isn't set up to detect - auto-increment correctly;. - - .. change:: - :tags: orm - :tickets: 2182 - - Fixed bug where transaction-level "deleted" - collection wouldn't be cleared of expunged - states, raising an error if they later - became transient. - - .. change:: - :tags: sql - :tickets: 2147 - - Fixed bug whereby if FetchedValue was passed - to column server_onupdate, it would not - have its parent "column" assigned, added - test coverage for all column default assignment - patterns. - - .. change:: - :tags: sql - :tickets: 2167 - - Fixed bug whereby nesting a label of a select() - with another label in it would produce incorrect - exported columns. Among other things this would - break an ORM column_property() mapping against - another column_property(). . - - .. change:: - :tags: engine - :tickets: 2178 - - Adjusted the __contains__() method of - a RowProxy result row such that no exception - throw is generated internally; - NoSuchColumnError() also will generate its - message regardless of whether or not the column - construct can be coerced to a string.. - - .. change:: - :tags: postgresql - :tickets: 2141 - - Fixed bug affecting PG 9 whereby index reflection - would fail if against a column whose name - had changed. . - - .. change:: - :tags: postgresql - :tickets: 2175 - - Some unit test fixes regarding numeric arrays, - MATCH operator. A potential floating-point - inaccuracy issue was fixed, and certain tests - of the MATCH operator only execute within an - EN-oriented locale for now. . - - .. change:: - :tags: mssql - :tickets: 2169 - - Fixed bug in MSSQL dialect whereby the aliasing - applied to a schema-qualified table would leak - into enclosing select statements. - - .. change:: - :tags: mssql - :tickets: 2159 - - Fixed bug whereby DATETIME2 type would fail on - the "adapt" step when used in result sets or - bound parameters. This issue is not in 0.7. - -.. changelog:: - :version: 0.6.7 - :released: Wed Apr 13 2011 - - .. change:: - :tags: orm - :tickets: 2087 - - Tightened the iterate vs. remove mutex around the - identity map iteration, attempting to reduce the - chance of an (extremely rare) reentrant gc operation - causing a deadlock. Might remove the mutex in - 0.7. - - .. change:: - :tags: orm - :tickets: 2030 - - Added a `name` argument to `Query.subquery()`, to allow - a fixed name to be assigned to the alias object. - - .. change:: - :tags: orm - :tickets: 2019 - - A warning is emitted when a joined-table inheriting mapper - has no primary keys on the locally mapped table - (but has pks on the superclass table). - - .. change:: - :tags: orm - :tickets: 2038 - - Fixed bug where "middle" class in a polymorphic hierarchy - would have no 'polymorphic_on' column if it didn't also - specify a 'polymorphic_identity', leading to strange - errors upon refresh, wrong class loaded when querying - from that target. Also emits the correct WHERE criterion - when using single table inheritance. - - .. change:: - :tags: orm - :tickets: 1995 - - Fixed bug where a column with a SQL or server side default - that was excluded from a mapping with include_properties - or exclude_properties would result in UnmappedColumnError. - - .. change:: - :tags: orm - :tickets: 2046 - - A warning is emitted in the unusual case that an - append or similar event on a collection occurs after - the parent object has been dereferenced, which - prevents the parent from being marked as "dirty" - in the session. This will be an exception in 0.7. - - .. change:: - :tags: orm - :tickets: 2098 - - Fixed bug in query.options() whereby a path - applied to a lazyload using string keys could - overlap a same named attribute on the wrong - entity. Note 0.7 has an updated version of this - fix. - - .. change:: - :tags: orm - :tickets: 2063 - - Reworded the exception raised when a flush - is attempted of a subclass that is not polymorphic - against the supertype. - - .. change:: - :tags: orm - :tickets: 2123 - - Some fixes to the state handling regarding - backrefs, typically when autoflush=False, where - the back-referenced collection wouldn't - properly handle add/removes with no net - change. Thanks to Richard Murri for the - test case + patch. - - .. change:: - :tags: orm - :tickets: 2130 - - a "having" clause would be copied from the - inside to the outside query if from_self() - were used.. - - .. change:: - :tags: sql - :tickets: 2028 - - Column.copy(), as used in table.tometadata(), copies the - 'doc' attribute. - - .. change:: - :tags: sql - :tickets: 2023 - - Added some defs to the resultproxy.c extension so that - the extension compiles and runs on Python 2.4. - - .. change:: - :tags: sql - :tickets: 2042 - - The compiler extension now supports overriding the default - compilation of expression._BindParamClause including that - the auto-generated binds within the VALUES/SET clause - of an insert()/update() statement will also use the new - compilation rules. - - .. change:: - :tags: sql - :tickets: 2089 - - Added accessors to ResultProxy "returns_rows", "is_insert" - - .. change:: - :tags: sql - :tickets: 2116 - - The limit/offset keywords to select() as well - as the value passed to select.limit()/offset() - will be coerced to integer. - - .. change:: - :tags: engine - :tickets: 2102 - - Fixed bug in QueuePool, SingletonThreadPool whereby - connections that were discarded via overflow or periodic - cleanup() were not explicitly closed, leaving garbage - collection to the task instead. This generally only - affects non-reference-counting backends like Jython - and PyPy. Thanks to Jaimy Azle for spotting - this. - - .. change:: - :tags: sqlite - :tickets: 2115 - - Fixed bug where reflection of foreign key - created as "REFERENCES " without - col name would fail. - - .. change:: - :tags: postgresql - :tickets: 1083 - - When explicit sequence execution derives the name - of the auto-generated sequence of a SERIAL column, - which currently only occurs if implicit_returning=False, - now accommodates if the table + column name is greater - than 63 characters using the same logic PostgreSQL uses. - - .. change:: - :tags: postgresql - :tickets: 2044 - - Added an additional libpq message to the list of "disconnect" - exceptions, "could not receive data from server" - - .. change:: - :tags: postgresql - :tickets: 2092 - - Added RESERVED_WORDS for postgresql dialect. - - .. change:: - :tags: postgresql - :tickets: 2073 - - Fixed the BIT type to allow a "length" parameter, "varying" - parameter. Reflection also fixed. - - .. change:: - :tags: informix - :tickets: 2092 - - Added RESERVED_WORDS informix dialect. - - .. change:: - :tags: mssql - :tickets: 2071 - - Rewrote the query used to get the definition of a view, - typically when using the Inspector interface, to - use sys.sql_modules instead of the information schema, - thereby allowing views definitions longer than 4000 - characters to be fully returned. - - .. change:: - :tags: mysql - :tickets: 2047 - - oursql dialect accepts the same "ssl" arguments in - create_engine() as that of MySQLdb. - - .. change:: - :tags: firebird - :tickets: 2083 - - The "implicit_returning" flag on create_engine() is - honored if set to False. - - .. change:: - :tags: oracle - :tickets: 2100 - - Using column names that would require quotes - for the column itself or for a name-generated - bind parameter, such as names with special - characters, underscores, non-ascii characters, - now properly translate bind parameter keys when - talking to cx_oracle. - - .. change:: - :tags: oracle - :tickets: 2116 - - Oracle dialect adds use_binds_for_limits=False - create_engine() flag, will render the LIMIT/OFFSET - values inline instead of as binds, reported to - modify the execution plan used by Oracle. - - .. change:: - :tags: ext - :tickets: 2090 - - The horizontal_shard ShardedSession class accepts the common - Session argument "query_cls" as a constructor argument, - to enable further subclassing of ShardedQuery. - - .. change:: - :tags: declarative - :tickets: 2050 - - Added an explicit check for the case that the name - 'metadata' is used for a column attribute on a - declarative class. - - .. change:: - :tags: declarative - :tickets: 2061 - - Fix error message referencing old @classproperty - name to reference @declared_attr - - .. change:: - :tags: declarative - :tickets: 2091 - - Arguments in __mapper_args__ that aren't "hashable" - aren't mistaken for always-hashable, possibly-column - arguments. - - .. change:: - :tags: documentation - :tickets: 2029 - - Documented SQLite DATE/TIME/DATETIME types. - - .. change:: - :tags: examples - :tickets: 2090 - - The Beaker caching example allows a "query_cls" argument - to the query_callable() function. - -.. changelog:: - :version: 0.6.6 - :released: Sat Jan 08 2011 - - .. change:: - :tags: orm - :tickets: - - Fixed bug whereby a non-"mutable" attribute modified event - which occurred on an object that was clean except for - preceding mutable attribute changes would fail to strongly - reference itself in the identity map. This would cause the - object to be garbage collected, losing track of any changes - that weren't previously saved in the "mutable changes" - dictionary. - - .. change:: - :tags: orm - :tickets: 2013 - - Fixed bug whereby "passive_deletes='all'" wasn't passing - the correct symbols to lazy loaders during flush, thereby - causing an unwarranted load. - - .. change:: - :tags: orm - :tickets: 1997 - - Fixed bug which prevented composite mapped - attributes from being used on a mapped select statement.. Note the workings of composite are slated to - change significantly in 0.7. - - .. change:: - :tags: orm - :tickets: 1976 - - active_history flag also added to composite(). - The flag has no effect in 0.6, but is instead - a placeholder flag for forwards compatibility, - as it applies in 0.7 for composites. - - .. change:: - :tags: orm - :tickets: 2002 - - Fixed uow bug whereby expired objects passed to - Session.delete() would not have unloaded references - or collections taken into account when deleting - objects, despite passive_deletes remaining at - its default of False. - - .. change:: - :tags: orm - :tickets: 1987 - - A warning is emitted when version_id_col is specified - on an inheriting mapper when the inherited mapper - already has one, if those column expressions are not - the same. - - .. change:: - :tags: orm - :tickets: 1954 - - "innerjoin" flag doesn't take effect along the chain - of joinedload() joins if a previous join in that chain - is an outer join, thus allowing primary rows without - a referenced child row to be correctly returned - in results. - - .. change:: - :tags: orm - :tickets: 1964 - - Fixed bug regarding "subqueryload" strategy whereby - strategy would fail if the entity was an aliased() - construct. - - .. change:: - :tags: orm - :tickets: 2014 - - Fixed bug regarding "subqueryload" strategy whereby - the join would fail if using a multi-level load - of the form from A->joined-subclass->C - - .. change:: - :tags: orm - :tickets: 1968 - - Fixed indexing of Query objects by -1. It was erroneously - transformed to the empty slice -1:0 that resulted in - IndexError. - - .. change:: - :tags: orm - :tickets: 1971 - - The mapper argument "primary_key" can be passed as a - single column as well as a list or tuple. - The documentation examples that illustrated it as a - scalar value have been changed to lists. - - .. change:: - :tags: orm - :tickets: 1961 - - Added active_history flag to relationship() - and column_property(), forces attribute events to - always load the "old" value, so that it's available to - attributes.get_history(). - - .. change:: - :tags: orm - :tickets: 1977 - - Query.get() will raise if the number of params - in a composite key is too large, as well as too - small. - - .. change:: - :tags: orm - :tickets: 1992 - - Backport of "optimized get" fix from 0.7, - improves the generation of joined-inheritance - "load expired row" behavior. - - .. change:: - :tags: orm - :tickets: - - A little more verbiage to the "primaryjoin" error, - in an unusual condition that the join condition - "works" for viewonly but doesn't work for non-viewonly, - and foreign_keys wasn't used - adds "foreign_keys" to - the suggestion. Also add "foreign_keys" to the - suggestion for the generic "direction" error. - - .. change:: - :tags: sql - :tickets: 1984 - - Fixed operator precedence rules for multiple - chains of a single non-associative operator. - I.e. "x - (y - z)" will compile as "x - (y - z)" - and not "x - y - z". Also works with labels, - i.e. "x - (y - z).label('foo')" - - .. change:: - :tags: sql - :tickets: 1967 - - The 'info' attribute of Column is copied during - Column.copy(), i.e. as occurs when using columns - in declarative mixins. - - .. change:: - :tags: sql - :tickets: - - Added a bind processor for booleans which coerces - to int, for DBAPIs such as pymssql that naively call - str() on values. - - .. change:: - :tags: sql - :tickets: 2000 - - CheckConstraint will copy its 'initially', 'deferrable', - and '_create_rule' attributes within a copy()/tometadata() - - .. change:: - :tags: engine - :tickets: - - The "unicode warning" against non-unicode bind data - is now raised only when the - Unicode type is used explicitly; not when - convert_unicode=True is used on the engine - or String type. - - .. change:: - :tags: engine - :tickets: 1978 - - Fixed memory leak in C version of Decimal result - processor. - - .. change:: - :tags: engine - :tickets: 1871 - - Implemented sequence check capability for the C - version of RowProxy, as well as 2.7 style - "collections.Sequence" registration for RowProxy. - - .. change:: - :tags: engine - :tickets: 1998 - - Threadlocal engine methods rollback(), commit(), - prepare() won't raise if no transaction is in progress; - this was a regression introduced in 0.6. - - .. change:: - :tags: engine - :tickets: 2004 - - Threadlocal engine returns itself upon begin(), - begin_nested(); engine then implements contextmanager - methods to allow the "with" statement. - - .. change:: - :tags: postgresql - :tickets: 1984 - - Single element tuple expressions inside an IN clause - parenthesize correctly, also from - - .. change:: - :tags: postgresql - :tickets: 1955 - - Ensured every numeric, float, int code, scalar + array, - are recognized by psycopg2 and pg8000's "numeric" - base type. - - .. change:: - :tags: postgresql - :tickets: 1956 - - Added as_uuid=True flag to the UUID type, will receive - and return values as Python UUID() objects rather than - strings. Currently, the UUID type is only known to - work with psycopg2. - - .. change:: - :tags: postgresql - :tickets: 1989 - - Fixed bug whereby KeyError would occur with non-ENUM - supported PG versions after a pool dispose+recreate - would occur. - - .. change:: - :tags: mysql - :tickets: 1960 - - Fixed error handling for Jython + zxjdbc, such that - has_table() property works again. Regression from - 0.6.3 (we don't have a Jython buildbot, sorry) - - .. change:: - :tags: sqlite - :tickets: 1851 - - The REFERENCES clause in a CREATE TABLE that includes - a remote schema to another table with the same schema - name now renders the remote name without - the schema clause, as required by SQLite. - - .. change:: - :tags: sqlite - :tickets: - - On the same theme, the REFERENCES clause in a CREATE TABLE - that includes a remote schema to a *different* schema - than that of the parent table doesn't render at all, - as cross-schema references do not appear to be supported. - - .. change:: - :tags: mssql - :tickets: 1770 - - The rewrite of index reflection in was - unfortunately not tested correctly, and returned incorrect - results. This regression is now fixed. - - .. change:: - :tags: oracle - :tickets: 1953 - - The cx_oracle "decimal detection" logic, which takes place - for result set columns with ambiguous numeric characteristics, - now uses the decimal point character determined by the locale/ - NLS_LANG setting, using an on-first-connect detection of - this character. cx_oracle 5.0.3 or greater is also required - when using a non-period-decimal-point NLS_LANG setting.. - - .. change:: - :tags: firebird - :tickets: 2012 - - Firebird numeric type now checks for Decimal explicitly, - lets float() pass right through, thereby allowing - special values such as float('inf'). - - .. change:: - :tags: declarative - :tickets: 1972 - - An error is raised if __table_args__ is not in tuple - or dict format, and is not None. - - .. change:: - :tags: sqlsoup - :tickets: 1975 - - Added "map_to()" method to SqlSoup, which is a "master" - method which accepts explicit arguments for each aspect of - the selectable and mapping, including a base class per - mapping. - - .. change:: - :tags: sqlsoup - :tickets: - - Mapped selectables used with the map(), with_labels(), - join() methods no longer put the given argument into the - internal "cache" dictionary. Particularly since the - join() and select() objects are created in the method - itself this was pretty much a pure memory leaking behavior. - - .. change:: - :tags: examples - :tickets: - - The versioning example now supports detection of changes - in an associated relationship(). - -.. changelog:: - :version: 0.6.5 - :released: Sun Oct 24 2010 - - .. change:: - :tags: orm - :tickets: 1914 - - Added a new "lazyload" option "immediateload". - Issues the usual "lazy" load operation automatically - as the object is populated. The use case - here is when loading objects to be placed in - an offline cache, or otherwise used after - the session isn't available, and straight 'select' - loading, not 'joined' or 'subquery', is desired. - - .. change:: - :tags: orm - :tickets: 1920 - - New Query methods: query.label(name), query.as_scalar(), - return the query's statement as a scalar subquery - with /without label; - query.with_entities(\*ent), replaces the SELECT list of - the query with new entities. - Roughly equivalent to a generative form of query.values() - which accepts mapped entities as well as column - expressions. - - .. change:: - :tags: orm - :tickets: - - Fixed recursion bug which could occur when moving - an object from one reference to another, with - backrefs involved, where the initiating parent - was a subclass (with its own mapper) of the - previous parent. - - .. change:: - :tags: orm - :tickets: 1918 - - Fixed a regression in 0.6.4 which occurred if you - passed an empty list to "include_properties" on - mapper() - - .. change:: - :tags: orm - :tickets: - - Fixed labeling bug in Query whereby the NamedTuple - would mis-apply labels if any of the column - expressions were un-labeled. - - .. change:: - :tags: orm - :tickets: 1925 - - Patched a case where query.join() would adapt the - right side to the right side of the left's join - inappropriately - - .. change:: - :tags: orm - :tickets: - - Query.select_from() has been beefed up to help - ensure that a subsequent call to query.join() - will use the select_from() entity, assuming it's - a mapped entity and not a plain selectable, - as the default "left" side, not the first entity - in the Query object's list of entities. - - .. change:: - :tags: orm - :tickets: - - The exception raised by Session when it is used - subsequent to a subtransaction rollback (which is what - happens when a flush fails in autocommit=False mode) has - now been reworded (this is the "inactive due to a - rollback in a subtransaction" message). In particular, - if the rollback was due to an exception during flush(), - the message states this is the case, and reiterates the - string form of the original exception that occurred - during flush. If the session is closed due to explicit - usage of subtransactions (not very common), the message - just states this is the case. - - .. change:: - :tags: orm - :tickets: - - The exception raised by Mapper when repeated requests to - its initialization are made after initialization already - failed no longer assumes the "hasattr" case, since - there's other scenarios in which this message gets - emitted, and the message also does not compound onto - itself multiple times - you get the same message for - each attempt at usage. The misnomer "compiles" is being - traded out for "initialize". - - .. change:: - :tags: orm - :tickets: 1935 - - Fixed bug in query.update() where 'evaluate' or 'fetch' - expiration would fail if the column expression key was - a class attribute with a different keyname as the - actual column name. - - .. change:: - :tags: orm - :tickets: - - Added an assertion during flush which ensures - that no NULL-holding identity keys were generated - on "newly persistent" objects. - This can occur when user defined code inadvertently - triggers flushes on not-fully-loaded objects. - - .. change:: - :tags: orm - :tickets: 1910 - - lazy loads for relationship attributes now use - the current state, not the "committed" state, - of foreign and primary key attributes - when issuing SQL, if a flush is not in process. - Previously, only the database-committed state would - be used. In particular, this would cause a many-to-one - get()-on-lazyload operation to fail, as autoflush - is not triggered on these loads when the attributes are - determined and the "committed" state may not be - available. - - .. change:: - :tags: orm - :tickets: - - A new flag on relationship(), load_on_pending, allows - the lazy loader to fire off on pending objects without a - flush taking place, as well as a transient object that's - been manually "attached" to the session. Note that this - flag blocks attribute events from taking place when an - object is loaded, so backrefs aren't available until - after a flush. The flag is only intended for very - specific use cases. - - .. change:: - :tags: orm - :tickets: - - Another new flag on relationship(), cascade_backrefs, - disables the "save-update" cascade when the event was - initiated on the "reverse" side of a bidirectional - relationship. This is a cleaner behavior so that - many-to-ones can be set on a transient object without - it getting sucked into the child object's session, - while still allowing the forward collection to - cascade. We *might* default this to False in 0.7. - - .. change:: - :tags: orm - :tickets: - - Slight improvement to the behavior of - "passive_updates=False" when placed only on the - many-to-one side of a relationship; documentation has - been clarified that passive_updates=False should really - be on the one-to-many side. - - .. change:: - :tags: orm - :tickets: - - Placing passive_deletes=True on a many-to-one emits - a warning, since you probably intended to put it on - the one-to-many side. - - .. change:: - :tags: orm - :tickets: - - Fixed bug that would prevent "subqueryload" from - working correctly with single table inheritance - for a relationship from a subclass - the "where - type in (x, y, z)" only gets placed on the inside, - instead of repeatedly. - - .. change:: - :tags: orm - :tickets: - - When using from_self() with single table inheritance, - the "where type in (x, y, z)" is placed on the outside - of the query only, instead of repeatedly. May make - some more adjustments to this. - - .. change:: - :tags: orm - :tickets: 1924 - - scoped_session emits a warning when configure() is - called if a Session is already present (checks only the - current thread) - - .. change:: - :tags: orm - :tickets: 1932 - - reworked the internals of mapper.cascade_iterator() to - cut down method calls by about 9% in some circumstances. - - .. change:: - :tags: sql - :tickets: - - Fixed bug in TypeDecorator whereby the dialect-specific - type was getting pulled in to generate the DDL for a - given type, which didn't always return the correct result. - - .. change:: - :tags: sql - :tickets: - - TypeDecorator can now have a fully constructed type - specified as its "impl", in addition to a type class. - - .. change:: - :tags: sql - :tickets: - - TypeDecorator will now place itself as the resulting - type for a binary expression where the type coercion - rules would normally return its impl type - previously, - a copy of the impl type would be returned which would - have the TypeDecorator embedded into it as the "dialect" - impl, this was probably an unintentional way of achieving - the desired effect. - - .. change:: - :tags: sql - :tickets: - - TypeDecorator.load_dialect_impl() returns "self.impl" by - default, i.e. not the dialect implementation type of - "self.impl". This to support compilation correctly. - Behavior can be user-overridden in exactly the same way - as before to the same effect. - - .. change:: - :tags: sql - :tickets: - - Added type_coerce(expr, type\_) expression element. - Treats the given expression as the given type when evaluating - expressions and processing result rows, but does not - affect the generation of SQL, other than an anonymous - label. - - .. change:: - :tags: sql - :tickets: - - Table.tometadata() now copies Index objects associated - with the Table as well. - - .. change:: - :tags: sql - :tickets: - - Table.tometadata() issues a warning if the given Table - is already present in the target MetaData - the existing - Table object is returned. - - .. change:: - :tags: sql - :tickets: - - An informative error message is raised if a Column - which has not yet been assigned a name, i.e. as in - declarative, is used in a context where it is - exported to the columns collection of an enclosing - select() construct, or if any construct involving - that column is compiled before its name is - assigned. - - .. change:: - :tags: sql - :tickets: 1862 - - as_scalar(), label() can be called on a selectable - which contains a Column that is not yet named. - - .. change:: - :tags: sql - :tickets: 1907 - - Fixed recursion overflow which could occur when operating - with two expressions both of type "NullType", but - not the singleton NULLTYPE instance. - - .. change:: - :tags: declarative - :tickets: 1922 - - @classproperty (soon/now @declared_attr) takes effect for - __mapper_args__, __table_args__, __tablename__ on - a base class that is not a mixin, as well as mixins. - - .. change:: - :tags: declarative - :tickets: 1915 - - @classproperty 's official name/location for usage - with declarative is sqlalchemy.ext.declarative.declared_attr. - Same thing, but moving there since it is more of a - "marker" that's specific to declarative, - not just an attribute technique. - - .. change:: - :tags: declarative - :tickets: 1931, 1930 - - Fixed bug whereby columns on a mixin wouldn't propagate - correctly to a single-table, or joined-table, - inheritance scheme where the attribute name is - different than that of the column.,. - - .. change:: - :tags: declarative - :tickets: - - A mixin can now specify a column that overrides - a column of the same name associated with a superclass. - Thanks to Oystein Haaland. - - .. change:: - :tags: engine - :tickets: - - Fixed a regression in 0.6.4 whereby the change that - allowed cursor errors to be raised consistently broke - the result.lastrowid accessor. Test coverage has - been added for result.lastrowid. Note that lastrowid - is only supported by Pysqlite and some MySQL drivers, - so isn't super-useful in the general case. - - .. change:: - :tags: engine - :tickets: - - the logging message emitted by the engine when - a connection is first used is now "BEGIN (implicit)" - to emphasize that DBAPI has no explicit begin(). - - .. change:: - :tags: engine - :tickets: 1936 - - added "views=True" option to metadata.reflect(), - will add the list of available views to those - being reflected. - - .. change:: - :tags: engine - :tickets: 1899 - - engine_from_config() now accepts 'debug' for - 'echo', 'echo_pool', 'force' for 'convert_unicode', - boolean values for 'use_native_unicode'. - - .. change:: - :tags: postgresql - :tickets: - - Added "as_tuple" flag to ARRAY type, returns results - as tuples instead of lists to allow hashing. - - .. change:: - :tags: postgresql - :tickets: 1933 - - Fixed bug which prevented "domain" built from a - custom type such as "enum" from being reflected. - - .. change:: - :tags: mysql - :tickets: 1940 - - Fixed bug involving reflection of CURRENT_TIMESTAMP - default used with ON UPDATE clause, thanks to - Taavi Burns - - .. change:: - :tags: oracle - :tickets: 1878 - - The implicit_returning argument to create_engine() - is now honored regardless of detected version of - Oracle. Previously, the flag would be forced - to False if server version info was < 10. - - .. change:: - :tags: mssql - :tickets: 1946 - - Fixed reflection bug which did not properly handle - reflection of unknown types. - - .. change:: - :tags: mssql - :tickets: 1943 - - Fixed bug where aliasing of tables with "schema" would - fail to compile properly. - - .. change:: - :tags: mssql - :tickets: 1770 - - Rewrote the reflection of indexes to use sys. - catalogs, so that column names of any configuration - (spaces, embedded commas, etc.) can be reflected. - Note that reflection of indexes requires SQL - Server 2005 or greater. - - .. change:: - :tags: mssql - :tickets: 1952 - - mssql+pymssql dialect now honors the "port" portion - of the URL instead of discarding it. - - .. change:: - :tags: informix - :tickets: 1906 - - *Major* cleanup / modernization of the Informix - dialect for 0.6, courtesy Florian Apolloner. - - .. change:: - :tags: tests - :tickets: - - the NoseSQLAlchemyPlugin has been moved to a - new package "sqlalchemy_nose" which installs - along with "sqlalchemy". This so that the "nosetests" - script works as always but also allows the - --with-coverage option to turn on coverage before - SQLAlchemy modules are imported, allowing coverage - to work correctly. - - .. change:: - :tags: misc - :tickets: 1890 - - CircularDependencyError now has .cycles and .edges - members, which are the set of elements involved in - one or more cycles, and the set of edges as 2-tuples. - -.. changelog:: - :version: 0.6.4 - :released: Tue Sep 07 2010 - - .. change:: - :tags: orm - :tickets: - - The name ConcurrentModificationError has been - changed to StaleDataError, and descriptive - error messages have been revised to reflect - exactly what the issue is. Both names will - remain available for the foreseeable future - for schemes that may be specifying - ConcurrentModificationError in an "except:" - clause. - - .. change:: - :tags: orm - :tickets: 1891 - - Added a mutex to the identity map which mutexes - remove operations against iteration methods, - which now pre-buffer before returning an - iterable. This because asynchronous gc - can remove items via the gc thread at any time. - - .. change:: - :tags: orm - :tickets: - - The Session class is now present in sqlalchemy.orm.*. - We're moving away from the usage of create_session(), - which has non-standard defaults, for those situations - where a one-step Session constructor is desired. Most - users should stick with sessionmaker() for general use, - however. - - .. change:: - :tags: orm - :tickets: - - query.with_parent() now accepts transient objects - and will use the non-persistent values of their pk/fk - attributes in order to formulate the criterion. - Docs are also clarified as to the purpose of with_parent(). - - .. change:: - :tags: orm - :tickets: - - The include_properties and exclude_properties arguments - to mapper() now accept Column objects as members in - addition to strings. This so that same-named Column - objects, such as those within a join(), can be - disambiguated. - - .. change:: - :tags: orm - :tickets: 1896 - - A warning is now emitted if a mapper is created against a - join or other single selectable that includes multiple - columns with the same name in its .c. collection, - and those columns aren't explicitly named as part of - the same or separate attributes (or excluded). - In 0.7 this warning will be an exception. Note that - this warning is not emitted when the combination occurs - as a result of inheritance, so that attributes - still allow being overridden naturally.. In 0.7 this will be improved further. - - .. change:: - :tags: orm - :tickets: 1896 - - The primary_key argument to mapper() can now specify - a series of columns that are only a subset of - the calculated "primary key" columns of the mapped - selectable, without an error being raised. This - helps for situations where a selectable's effective - primary key is simpler than the number of columns - in the selectable that are actually marked as - "primary_key", such as a join against two - tables on their primary key columns. - - .. change:: - :tags: orm - :tickets: - - An object that's been deleted now gets a flag - 'deleted', which prohibits the object from - being re-add()ed to the session, as previously - the object would live in the identity map - silently until its attributes were accessed. - The make_transient() function now resets this - flag along with the "key" flag. - - .. change:: - :tags: orm - :tickets: - - make_transient() can be safely called on an - already transient instance. - - .. change:: - :tags: orm - :tickets: - - a warning is emitted in mapper() if the polymorphic_on - column is not present either in direct or derived - form in the mapped selectable or in the - with_polymorphic selectable, instead of silently - ignoring it. Look for this to become an - exception in 0.7. - - .. change:: - :tags: orm - :tickets: - - Another pass through the series of error messages - emitted when relationship() is configured with - ambiguous arguments. The "foreign_keys" - setting is no longer mentioned, as it is almost - never needed and it is preferable users set up - correct ForeignKey metadata, which is now the - recommendation. If 'foreign_keys' - is used and is incorrect, the message suggests - the attribute is probably unnecessary. Docs - for the attribute are beefed up. This - because all confused relationship() users on the - ML appear to be attempting to use foreign_keys - due to the message, which only confuses them - further since Table metadata is much clearer. - - .. change:: - :tags: orm - :tickets: 1877 - - If the "secondary" table has no ForeignKey metadata - and no foreign_keys is set, even though the - user is passing screwed up information, it is assumed - that primary/secondaryjoin expressions should - consider only and all cols in "secondary" to be - foreign. It's not possible with "secondary" for - the foreign keys to be elsewhere in any case. - A warning is now emitted instead of an error, - and the mapping succeeds. - - .. change:: - :tags: orm - :tickets: 1856 - - Moving an o2m object from one collection to - another, or vice versa changing the referenced - object by an m2o, where the foreign key is also a - member of the primary key, will now be more - carefully checked during flush if the change in - value of the foreign key on the "many" side is the - result of a change in the primary key of the "one" - side, or if the "one" is just a different object. - In one case, a cascade-capable DB would have - cascaded the value already and we need to look at - the "new" PK value to do an UPDATE, in the other we - need to continue looking at the "old". We now look - at the "old", assuming passive_updates=True, - unless we know it was a PK switch that - triggered the change. - - .. change:: - :tags: orm - :tickets: 1857 - - The value of version_id_col can be changed - manually, and this will result in an UPDATE - of the row. Versioned UPDATEs and DELETEs - now use the "committed" value of the - version_id_col in the WHERE clause and - not the pending changed value. The - version generator is also bypassed if - manual changes are present on the attribute. - - .. change:: - :tags: orm - :tickets: - - Repaired the usage of merge() when used with - concrete inheriting mappers. Such mappers frequently - have so-called "concrete" attributes, which are - subclass attributes that "disable" propagation from - the parent - these needed to allow a merge() - operation to pass through without effect. - - .. change:: - :tags: orm - :tickets: 1863 - - Specifying a non-column based argument - for column_mapped_collection, including string, - text() etc., will raise an error message that - specifically asks for a column element, no longer - misleads with incorrect information about - text() or literal(). - - .. change:: - :tags: orm - :tickets: - - Similarly, for relationship(), foreign_keys, - remote_side, order_by - all column-based - expressions are enforced - lists of strings - are explicitly disallowed since this is a - very common error - - .. change:: - :tags: orm - :tickets: 1864 - - Dynamic attributes don't support collection - population - added an assertion for when - set_committed_value() is called, as well as - when joinedload() or subqueryload() options - are applied to a dynamic attribute, instead - of failure / silent failure. - - .. change:: - :tags: orm - :tickets: 1852 - - Fixed bug whereby generating a Query derived - from one which had the same column repeated - with different label names, typically - in some UNION situations, would fail to - propagate the inner columns completely to - the outer query. - - .. change:: - :tags: orm - :tickets: 1881 - - object_session() raises the proper - UnmappedInstanceError when presented with an - unmapped instance. - - .. change:: - :tags: orm - :tickets: - - Applied further memoizations to calculated Mapper - properties, with significant (~90%) runtime mapper.py - call count reduction in heavily polymorphic mapping - configurations. - - .. change:: - :tags: orm - :tickets: - - mapper _get_col_to_prop private method used - by the versioning example is deprecated; - now use mapper.get_property_by_column() which - will remain the public method for this. - - .. change:: - :tags: orm - :tickets: - - the versioning example works correctly now - if versioning on a col that was formerly - NULL. - - .. change:: - :tags: sql - :tickets: - - Calling execute() on an alias() construct is pending - deprecation for 0.7, as it is not itself an - "executable" construct. It currently "proxies" its - inner element and is conditionally "executable" but - this is not the kind of ambiguity we like these days. - - .. change:: - :tags: sql - :tickets: - - The execute() and scalar() methods of ClauseElement - are now moved appropriately to the Executable - subclass. ClauseElement.execute()/ scalar() are still - present and are pending deprecation in 0.7, but note - these would always raise an error anyway if you were - not an Executable (unless you were an alias(), see - previous note). - - .. change:: - :tags: sql - :tickets: - - Added basic math expression coercion for - Numeric->Integer, - so that resulting type is Numeric regardless - of the direction of the expression. - - .. change:: - :tags: sql - :tickets: 1855 - - Changed the scheme used to generate truncated - "auto" index names when using the "index=True" - flag on Column. The truncation only takes - place with the auto-generated name, not one - that is user-defined (an error would be - raised instead), and the truncation scheme - itself is now based on a fragment of an md5 - hash of the identifier name, so that multiple - indexes on columns with similar names still - have unique names. - - .. change:: - :tags: sql - :tickets: 1412 - - The generated index name also is based on - a "max index name length" attribute which is - separate from the "max identifier length" - - this to appease MySQL who has a max length - of 64 for index names, separate from their - overall max length of 255. - - .. change:: - :tags: sql - :tickets: - - the text() construct, if placed in a column - oriented situation, will at least return NULLTYPE - for its type instead of None, allowing it to - be used a little more freely for ad-hoc column - expressions than before. literal_column() - is still the better choice, however. - - .. change:: - :tags: sql - :tickets: - - Added full description of parent table/column, - target table/column in error message raised when - ForeignKey can't resolve target. - - .. change:: - :tags: sql - :tickets: 1865 - - Fixed bug whereby replacing composite foreign key - columns in a reflected table would cause an attempt - to remove the reflected constraint from the table - a second time, raising a KeyError. - - .. change:: - :tags: sql - :tickets: - - the _Label construct, i.e. the one that is produced - whenever you say somecol.label(), now counts itself - in its "proxy_set" unioned with that of its - contained column's proxy set, instead of - directly returning that of the contained column. - This allows column correspondence - operations which depend on the identity of the - _Labels themselves to return the correct result - - .. change:: - :tags: sql - :tickets: 1852 - - fixes ORM bug. - - .. change:: - :tags: engine - :tickets: - - Calling fetchone() or similar on a result that - has already been exhausted, has been closed, - or is not a result-returning result now - raises ResourceClosedError, a subclass of - InvalidRequestError, in all cases, regardless - of backend. Previously, some DBAPIs would - raise ProgrammingError (i.e. pysqlite), others - would return None leading to downstream breakages - (i.e. MySQL-python). - - .. change:: - :tags: engine - :tickets: 1894 - - Fixed bug in Connection whereby if a "disconnect" - event occurred in the "initialize" phase of the - first connection pool connect, an AttributeError - would be raised when the Connection would attempt - to invalidate the DBAPI connection. - - .. change:: - :tags: engine - :tickets: - - Connection, ResultProxy, as well as Session use - ResourceClosedError for all "this - connection/transaction/result is closed" types of - errors. - - .. change:: - :tags: engine - :tickets: - - Connection.invalidate() can be called more than - once and subsequent calls do nothing. - - .. change:: - :tags: declarative - :tickets: - - if @classproperty is used with a regular class-bound - mapper property attribute, it will be called to get the - actual attribute value during initialization. Currently, - there's no advantage to using @classproperty on a column - or relationship attribute of a declarative class that - isn't a mixin - evaluation is at the same time as if - @classproperty weren't used. But here we at least allow - it to function as expected. - - .. change:: - :tags: declarative - :tickets: - - Fixed bug where "Can't add additional column" message - would display the wrong name. - - .. change:: - :tags: postgresql - :tickets: - - Fixed the psycopg2 dialect to use its - set_isolation_level() method instead of relying - upon the base "SET SESSION ISOLATION" command, - as psycopg2 resets the isolation level on each new - transaction otherwise. - - .. change:: - :tags: mssql - :tickets: - - Fixed "default schema" query to work with - pymssql backend. - - .. change:: - :tags: firebird - :tickets: - - Fixed bug whereby a column default would fail to - reflect if the "default" keyword were lower case. - - .. change:: - :tags: oracle - :tickets: 1879 - - Added ROWID type to the Oracle dialect, for those - cases where an explicit CAST might be needed. - - .. change:: - :tags: oracle - :tickets: 1867 - - Oracle reflection of indexes has been tuned so - that indexes which include some or all primary - key columns, but not the same set of columns - as that of the primary key, are reflected. - Indexes which contain the identical columns - as that of the primary key are skipped within - reflection, as the index in that case is assumed - to be the auto-generated primary key index. - Previously, any index with PK columns present - would be skipped. Thanks to Kent Bower - for the patch. - - .. change:: - :tags: oracle - :tickets: 1868 - - Oracle now reflects the names of primary key - constraints - also thanks to Kent Bower. - - .. change:: - :tags: informix - :tickets: 1904 - - Applied patches from to get - basic Informix functionality up again. We - rely upon end-user testing to ensure that - Informix is working to some degree. - - .. change:: - :tags: documentation - :tickets: - - The docs have been reorganized such that the "API - Reference" section is gone - all the docstrings from - there which were public API are moved into the - context of the main doc section that talks about it. - Main docs divided into "SQLAlchemy Core" and - "SQLAlchemy ORM" sections, mapper/relationship docs - have been broken out. Lots of sections rewritten - and/or reorganized. - - .. change:: - :tags: examples - :tickets: - - The beaker_caching example has been reorganized - such that the Session, cache manager, - declarative_base are part of environment, and - custom cache code is portable and now within - "caching_query.py". This allows the example to - be easier to "drop in" to existing projects. - - .. change:: - :tags: examples - :tickets: 1887 - - the history_meta versioning recipe sets "unique=False" - when copying columns, so that the versioning - table handles multiple rows with repeating values. - -.. changelog:: - :version: 0.6.3 - :released: Thu Jul 15 2010 - - .. change:: - :tags: orm - :tickets: 1845 - - Removed errant many-to-many load in unitofwork - which triggered unnecessarily on expired/unloaded - collections. This load now takes place only if - passive_updates is False and the parent primary - key has changed, or if passive_deletes is False - and a delete of the parent has occurred. - - .. change:: - :tags: orm - :tickets: 1853 - - Column-entities (i.e. query(Foo.id)) copy their - state more fully when queries are derived from - themselves + a selectable (i.e. from_self(), - union(), etc.), so that join() and such have the - correct state to work from. - - .. change:: - :tags: orm - :tickets: 1853 - - Fixed bug where Query.join() would fail if - querying a non-ORM column then joining without - an on clause when a FROM clause is already - present, now raises a checked exception the - same way it does when the clause is not - present. - - .. change:: - :tags: orm - :tickets: 1142 - - Improved the check for an "unmapped class", - including the case where the superclass is mapped - but the subclass is not. Any attempts to access - cls._sa_class_manager.mapper now raise - UnmappedClassError(). - - .. change:: - :tags: orm - :tickets: - - Added "column_descriptions" accessor to Query, - returns a list of dictionaries containing - naming/typing information about the entities - the Query will return. Can be helpful for - building GUIs on top of ORM queries. - - .. change:: - :tags: mysql - :tickets: 1848 - - The _extract_error_code() method now works - correctly with each MySQL dialect ( - MySQL-python, OurSQL, MySQL-Connector-Python, - PyODBC). Previously, - the reconnect logic would fail for OperationalError - conditions, however since MySQLdb and OurSQL - have their own reconnect feature, there was no - symptom for these drivers here unless one - watched the logs. - - .. change:: - :tags: oracle - :tickets: 1840 - - More tweaks to cx_oracle Decimal handling. - "Ambiguous" numerics with no decimal place - are coerced to int at the connection handler - level. The advantage here is that ints - come back as ints without SQLA type - objects being involved and without needless - conversion to Decimal first. - - Unfortunately, some exotic subquery cases - can even see different types between - individual result rows, so the Numeric - handler, when instructed to return Decimal, - can't take full advantage of "native decimal" - mode and must run isinstance() on every value - to check if its Decimal already. Reopen of - -.. changelog:: - :version: 0.6.2 - :released: Tue Jul 06 2010 - - .. change:: - :tags: orm - :tickets: - - Query.join() will check for a call of the - form query.join(target, clause_expression), - i.e. missing the tuple, and raise an informative - error message that this is the wrong calling form. - - .. change:: - :tags: orm - :tickets: 1824 - - Fixed bug regarding flushes on self-referential - bi-directional many-to-many relationships, where - two objects made to mutually reference each other - in one flush would fail to insert a row for both - sides. Regression from 0.5. - - .. change:: - :tags: orm - :tickets: - - the post_update feature of relationship() has been - reworked architecturally to integrate more closely - with the new 0.6 unit of work. The motivation - for the change is so that multiple "post update" - calls, each affecting different foreign key - columns of the same row, are executed in a single - UPDATE statement, rather than one UPDATE - statement per column per row. Multiple row - updates are also batched into executemany()s as - possible, while maintaining consistent row ordering. - - .. change:: - :tags: orm - :tickets: - - Query.statement, Query.subquery(), etc. now transfer - the values of bind parameters, i.e. those specified - by query.params(), into the resulting SQL expression. - Previously the values would not be transferred - and bind parameters would come out as None. - - .. change:: - :tags: orm - :tickets: - - Subquery-eager-loading now works with Query objects - which include params(), as well as get() Queries. - - .. change:: - :tags: orm - :tickets: - - Can now call make_transient() on an instance that - is referenced by parent objects via many-to-one, - without the parent's foreign key value getting - temporarily set to None - this was a function - of the "detect primary key switch" flush handler. - It now ignores objects that are no longer - in the "persistent" state, and the parent's - foreign key identifier is left unaffected. - - .. change:: - :tags: orm - :tickets: - - query.order_by() now accepts False, which cancels - any existing order_by() state on the Query, allowing - subsequent generative methods to be called which do - not support ORDER BY. This is not the same as the - already existing feature of passing None, which - suppresses any existing order_by() settings, including - those configured on the mapper. False will make it - as though order_by() was never called, while - None is an active setting. - - .. change:: - :tags: orm - :tickets: - - An instance which is moved to "transient", has - an incomplete or missing set of primary key - attributes, and contains expired attributes, will - raise an InvalidRequestError if an expired attribute - is accessed, instead of getting a recursion overflow. - - .. change:: - :tags: orm - :tickets: - - The make_transient() function is now in the generated - documentation. - - .. change:: - :tags: orm - :tickets: - - make_transient() removes all "loader" callables from - the state being made transient, removing any - "expired" state - all unloaded attributes reset back - to undefined, None/empty on access. - - .. change:: - :tags: sql - :tickets: 1822 - - The warning emitted by the Unicode and String types - with convert_unicode=True no longer embeds the actual - value passed. This so that the Python warning - registry does not continue to grow in size, the warning - is emitted once as per the warning filter settings, - and large string values don't pollute the output. - - .. change:: - :tags: sql - :tickets: - - Fixed bug that would prevent overridden clause - compilation from working for "annotated" expression - elements, which are often generated by the ORM. - - .. change:: - :tags: sql - :tickets: 1400 - - The argument to "ESCAPE" of a LIKE operator or similar - is passed through render_literal_value(), which may - implement escaping of backslashes. - - .. change:: - :tags: sql - :tickets: - - Fixed bug in Enum type which blew away native_enum - flag when used with TypeDecorators or other adaption - scenarios. - - .. change:: - :tags: sql - :tickets: - - Inspector hits bind.connect() when invoked to ensure - initialize has been called. the internal name ".conn" - is changed to ".bind", since that's what it is. - - .. change:: - :tags: sql - :tickets: - - Modified the internals of "column annotation" such that - a custom Column subclass can safely override - _constructor to return Column, for the purposes of - making "configurational" column classes that aren't - involved in proxying, etc. - - .. change:: - :tags: sql - :tickets: 1829 - - Column.copy() takes along the "unique" attribute - among others, fixes regarding declarative - mixins - - .. change:: - :tags: postgresql - :tickets: 1400 - - render_literal_value() is overridden which escapes - backslashes, currently applies to the ESCAPE clause - of LIKE and similar expressions. - Ultimately this will have to detect the value of - "standard_conforming_strings" for full behavior. - - .. change:: - :tags: postgresql - :tickets: 1836 - - Won't generate "CREATE TYPE" / "DROP TYPE" if - using types.Enum on a PG version prior to 8.3 - - the supports_native_enum flag is fully - honored. - - .. change:: - :tags: mysql - :tickets: 1826 - - MySQL dialect doesn't emit CAST() for MySQL version - detected < 4.0.2. This allows the unicode - check on connect to proceed. - - .. change:: - :tags: mysql - :tickets: - - MySQL dialect now detects NO_BACKSLASH_ESCAPES sql - mode, in addition to ANSI_QUOTES. - - .. change:: - :tags: mysql - :tickets: 1400 - - render_literal_value() is overridden which escapes - backslashes, currently applies to the ESCAPE clause - of LIKE and similar expressions. This behavior - is derived from detecting the value of - NO_BACKSLASH_ESCAPES. - - .. change:: - :tags: oracle - :tickets: 1819 - - Fixed ora-8 compatibility flags such that they - don't cache a stale value from before the first - database connection actually occurs. - - .. change:: - :tags: oracle - :tickets: 1840 - - Oracle's "native decimal" metadata begins to return - ambiguous typing information about numerics - when columns are embedded in subqueries as well - as when ROWNUM is consulted with subqueries, as we - do for limit/offset. We've added these ambiguous - conditions to the cx_oracle "convert to Decimal()" - handler, so that we receive numerics as Decimal - in more cases instead of as floats. These are - then converted, if requested, into Integer - or Float, or otherwise kept as the lossless - Decimal. - - .. change:: - :tags: mssql - :tickets: 1825 - - If server_version_info is outside the usual - range of (8, ), (9, ), (10, ), a warning is emitted - which suggests checking that the FreeTDS version - configuration is using 7.0 or 8.0, not 4.2. - - .. change:: - :tags: firebird - :tickets: 1823 - - Fixed incorrect signature in do_execute(), error - introduced in 0.6.1. - - .. change:: - :tags: firebird - :tickets: 1813 - - Firebird dialect adds CHAR, VARCHAR types which - accept a "charset" flag, to support Firebird - "CHARACTER SET" clause. - - .. change:: - :tags: declarative - :tickets: 1805, 1796, 1751 - - Added support for @classproperty to provide - any kind of schema/mapping construct from a - declarative mixin, including columns with foreign - keys, relationships, column_property, deferred. - This solves all such issues on declarative mixins. - An error is raised if any MapperProperty subclass - is specified on a mixin without using @classproperty. - - .. change:: - :tags: declarative - :tickets: 1821 - - a mixin class can now define a column that matches - one which is present on a __table__ defined on a - subclass. It cannot, however, define one that is - not present in the __table__, and the error message - here now works. - - .. change:: - :tags: extension, compiler - :tickets: 1838 - - The 'default' compiler is automatically copied over - when overriding the compilation of a built in - clause construct, so no KeyError is raised if the - user-defined compiler is specific to certain - backends and compilation for a different backend - is invoked. - - .. change:: - :tags: documentation - :tickets: 1820 - - Added documentation for the Inspector. - - .. change:: - :tags: documentation - :tickets: 1830 - - Fixed @memoized_property and @memoized_instancemethod - decorators so that Sphinx documentation picks up - these attributes and methods, such as - ResultProxy.inserted_primary_key. - -.. changelog:: - :version: 0.6.1 - :released: Mon May 31 2010 - - .. change:: - :tags: orm - :tickets: 1782 - - Fixed regression introduced in 0.6.0 involving improper - history accounting on mutable attributes. - - .. change:: - :tags: orm - :tickets: 1807 - - Fixed regression introduced in 0.6.0 unit of work refactor - that broke updates for bi-directional relationship() - with post_update=True. - - .. change:: - :tags: orm - :tickets: 1789 - - session.merge() will not expire attributes on the returned - instance if that instance is "pending". - - .. change:: - :tags: orm - :tickets: 1802 - - fixed __setstate__ method of CollectionAdapter to not - fail during deserialize where parent InstanceState not - yet unserialized. - - .. change:: - :tags: orm - :tickets: 1797 - - Added internal warning in case an instance without a - full PK happened to be expired and then was asked - to refresh. - - .. change:: - :tags: orm - :tickets: - - Added more aggressive caching to the mapper's usage of - UPDATE, INSERT, and DELETE expressions. Assuming the - statement has no per-object SQL expressions attached, - the expression objects are cached by the mapper after - the first create, and their compiled form is stored - persistently in a cache dictionary for the duration of - the related Engine. The cache is an LRUCache for the - rare case that a mapper receives an extremely - high number of different column patterns as UPDATEs. - - .. change:: - :tags: sql - :tickets: 1793 - - expr.in_() now accepts a text() construct as the argument. - Grouping parenthesis are added automatically, i.e. usage - is like `col.in_(text("select id from table"))`. - - .. change:: - :tags: sql - :tickets: - - Columns of _Binary type (i.e. LargeBinary, BLOB, etc.) - will coerce a "basestring" on the right side into a - _Binary as well so that required DBAPI processing - takes place. - - .. change:: - :tags: sql - :tickets: 1801 - - Added table.add_is_dependent_on(othertable), allows manual - placement of dependency rules between two Table objects - for use within create_all(), drop_all(), sorted_tables. - - .. change:: - :tags: sql - :tickets: 1778 - - Fixed bug that prevented implicit RETURNING from functioning - properly with composite primary key that contained zeroes. - - .. change:: - :tags: sql - :tickets: - - Fixed errant space character when generating ADD CONSTRAINT - for a named UNIQUE constraint. - - .. change:: - :tags: sql - :tickets: 1571 - - Fixed "table" argument on constructor of ForeignKeyConstraint - - .. change:: - :tags: sql - :tickets: 1786 - - Fixed bug in connection pool cursor wrapper whereby if a - cursor threw an exception on close(), the logging of the - message would fail. - - .. change:: - :tags: sql - :tickets: - - the _make_proxy() method of ColumnClause and Column now use - self.__class__ to determine the class of object to be returned - instead of hardcoding to ColumnClause/Column, making it slightly - easier to produce specific subclasses of these which work in - alias/subquery situations. - - .. change:: - :tags: sql - :tickets: 1798 - - func.XXX() doesn't inadvertently resolve to non-Function - classes (e.g. fixes func.text()). - - .. change:: - :tags: engines - :tickets: 1781 - - Fixed building the C extensions on Python 2.4. - - .. change:: - :tags: engines - :tickets: - - Pool classes will reuse the same "pool_logging_name" setting - after a dispose() occurs. - - .. change:: - :tags: engines - :tickets: - - Engine gains an "execution_options" argument and - update_execution_options() method, which will apply to - all connections generated by this engine. - - .. change:: - :tags: mysql - :tickets: 1794 - - func.sysdate() emits "SYSDATE()", i.e. with the ending - parenthesis, on MySQL. - - .. change:: - :tags: sqlite - :tickets: 1812 - - Fixed concatenation of constraints when "PRIMARY KEY" - constraint gets moved to column level due to SQLite - AUTOINCREMENT keyword being rendered. - - .. change:: - :tags: oracle - :tickets: 1775 - - Added a check for cx_oracle versions lower than version 5, - in which case the incompatible "output type handler" won't - be used. This will impact decimal accuracy and some - unicode handling issues. - - .. change:: - :tags: oracle - :tickets: 1790 - - Fixed use_ansi=False mode, which was producing broken - WHERE clauses in pretty much all cases. - - .. change:: - :tags: oracle - :tickets: 1808 - - Re-established support for Oracle 8 with cx_oracle, - including that use_ansi is set to False automatically, - NVARCHAR2 and NCLOB are not rendered for Unicode, - "native unicode" check doesn't fail, cx_oracle - "native unicode" mode is disabled, VARCHAR() is emitted - with bytes count instead of char count. - - .. change:: - :tags: oracle - :tickets: 1670 - - oracle_xe 5 doesn't accept a Python unicode object in - its connect string in normal Python 2.x mode - so we coerce - to str() directly. non-ascii characters aren't supported - in connect strings here since we don't know what encoding - we could use. - - .. change:: - :tags: oracle - :tickets: 1815 - - FOR UPDATE is emitted in the syntactically correct position - when limit/offset is used, i.e. the ROWNUM subquery. - However, Oracle can't really handle FOR UPDATE with ORDER BY - or with subqueries, so its still not very usable, but at - least SQLA gets the SQL past the Oracle parser. - - .. change:: - :tags: firebird - :tickets: 1521 - - Added a label to the query used within has_table() and - has_sequence() to work with older versions of Firebird - that don't provide labels for result columns. - - .. change:: - :tags: firebird - :tickets: 1779 - - Added integer coercion to the "type_conv" attribute when - passed via query string, so that it is properly interpreted - by Kinterbasdb. - - .. change:: - :tags: firebird - :tickets: 1646 - - Added 'connection shutdown' to the list of exception strings - which indicate a dropped connection. - - .. change:: - :tags: sqlsoup - :tickets: 1783 - - the SqlSoup constructor accepts a `base` argument which specifies - the base class to use for mapped classes, the default being - `object`. - -.. changelog:: - :version: 0.6.0 - :released: Sun Apr 18 2010 - - .. change:: - :tags: orm - :tickets: 1742, 1081 - - Unit of work internals have been rewritten. Units of work - with large numbers of objects interdependent objects - can now be flushed without recursion overflows - as there is no longer reliance upon recursive calls. The number of internal structures now stays - constant for a particular session state, regardless of - how many relationships are present on mappings. The flow - of events now corresponds to a linear list of steps, - generated by the mappers and relationships based on actual - work to be done, filtered through a single topological sort - for correct ordering. Flush actions are assembled using - far fewer steps and less memory. - - .. change:: - :tags: orm - :tickets: - - Along with the UOW rewrite, this also removes an issue - introduced in 0.6beta3 regarding topological cycle detection - for units of work with long dependency cycles. We now use - an algorithm written by Guido (thanks Guido!). - - .. change:: - :tags: orm - :tickets: 1764 - - one-to-many relationships now maintain a list of positive - parent-child associations within the flush, preventing - previous parents marked as deleted from cascading a - delete or NULL foreign key set on those child objects, - despite the end-user not removing the child from the old - association. - - .. change:: - :tags: orm - :tickets: 1495 - - A collection lazy load will switch off default - eagerloading on the reverse many-to-one side, since - that loading is by definition unnecessary. - - .. change:: - :tags: orm - :tickets: - - Session.refresh() now does an equivalent expire() - on the given instance first, so that the "refresh-expire" - cascade is propagated. Previously, refresh() was - not affected in any way by the presence of "refresh-expire" - cascade. This is a change in behavior versus that - of 0.6beta2, where the "lockmode" flag passed to refresh() - would cause a version check to occur. Since the instance - is first expired, refresh() always upgrades the object - to the most recent version. - - .. change:: - :tags: orm - :tickets: 1754 - - The 'refresh-expire' cascade, when reaching a pending object, - will expunge the object if the cascade also includes - "delete-orphan", or will simply detach it otherwise. - - .. change:: - :tags: orm - :tickets: 1756 - - id(obj) is no longer used internally within topological.py, - as the sorting functions now require hashable objects - only. - - .. change:: - :tags: orm - :tickets: - - The ORM will set the docstring of all generated descriptors - to None by default. This can be overridden using 'doc' - (or if using Sphinx, attribute docstrings work too). - - .. change:: - :tags: orm - :tickets: - - Added kw argument 'doc' to all mapper property callables - as well as Column(). Will assemble the string 'doc' as - the '__doc__' attribute on the descriptor. - - .. change:: - :tags: orm - :tickets: 1761 - - Usage of version_id_col on a backend that supports - cursor.rowcount for execute() but not executemany() now works - when a delete is issued (already worked for saves, since those - don't use executemany()). For a backend that doesn't support - cursor.rowcount at all, a warning is emitted the same - as with saves. - - .. change:: - :tags: orm - :tickets: - - The ORM now short-term caches the "compiled" form of - insert() and update() constructs when flushing lists of - objects of all the same class, thereby avoiding redundant - compilation per individual INSERT/UPDATE within an - individual flush() call. - - .. change:: - :tags: orm - :tickets: - - internal getattr(), setattr(), getcommitted() methods - on ColumnProperty, CompositeProperty, RelationshipProperty - have been underscored (i.e. are private), signature has - changed. - - .. change:: - :tags: engines - :tickets: 1757 - - The C extension now also works with DBAPIs which use custom - sequences as row (and not only tuples). - - .. change:: - :tags: sql - :tickets: 1755 - - Restored some bind-labeling logic from 0.5 which ensures - that tables with column names that overlap another column - of the form "_" won't produce - errors if column._label is used as a bind name during - an UPDATE. Test coverage which wasn't present in 0.5 - has been added. - - .. change:: - :tags: sql - :tickets: 1729 - - somejoin.select(fold_equivalents=True) is no longer - deprecated, and will eventually be rolled into a more - comprehensive version of the feature for. - - .. change:: - :tags: sql - :tickets: 1759 - - the Numeric type raises an *enormous* warning when expected - to convert floats to Decimal from a DBAPI that returns floats. - This includes SQLite, Sybase, MS-SQL. - - .. change:: - :tags: sql - :tickets: - - Fixed an error in expression typing which caused an endless - loop for expressions with two NULL types. - - .. change:: - :tags: sql - :tickets: - - Fixed bug in execution_options() feature whereby the existing - Transaction and other state information from the parent - connection would not be propagated to the sub-connection. - - .. change:: - :tags: sql - :tickets: - - Added new 'compiled_cache' execution option. A dictionary - where Compiled objects will be cached when the Connection - compiles a clause expression into a dialect- and parameter- - specific Compiled object. It is the user's responsibility to - manage the size of this dictionary, which will have keys - corresponding to the dialect, clause element, the column - names within the VALUES or SET clause of an INSERT or UPDATE, - as well as the "batch" mode for an INSERT or UPDATE statement. - - .. change:: - :tags: sql - :tickets: 1769 - - Added get_pk_constraint() to reflection.Inspector, similar - to get_primary_keys() except returns a dict that includes the - name of the constraint, for supported backends (PG so far). - - .. change:: - :tags: sql - :tickets: 1771 - - Table.create() and Table.drop() no longer apply metadata- - level create/drop events. - - .. change:: - :tags: ext - :tickets: - - the compiler extension now allows @compiles decorators - on base classes that extend to child classes, @compiles - decorators on child classes that aren't broken by a - @compiles decorator on the base class. - - .. change:: - :tags: ext - :tickets: - - Declarative will raise an informative error message - if a non-mapped class attribute is referenced in the - string-based relationship() arguments. - - .. change:: - :tags: ext - :tickets: - - Further reworked the "mixin" logic in declarative to - additionally allow __mapper_args__ as a @classproperty - on a mixin, such as to dynamically assign polymorphic_identity. - - .. change:: - :tags: postgresql - :tickets: 1071 - - PostgreSQL now reflects sequence names associated with - SERIAL columns correctly, after the name of the sequence - has been changed. Thanks to Kumar McMillan for the patch. - - .. change:: - :tags: postgresql - :tickets: - - Repaired missing import in psycopg2._PGNumeric type when - unknown numeric is received. - - .. change:: - :tags: postgresql - :tickets: - - psycopg2/pg8000 dialects now aware of REAL[], FLOAT[], - DOUBLE_PRECISION[], NUMERIC[] return types without - raising an exception. - - .. change:: - :tags: postgresql - :tickets: 1769 - - PostgreSQL reflects the name of primary key constraints, - if one exists. - - .. change:: - :tags: oracle - :tickets: - - Now using cx_oracle output converters so that the - DBAPI returns natively the kinds of values we prefer: - - .. change:: - :tags: oracle - :tickets: 1759 - - NUMBER values with positive precision + scale convert - to cx_oracle.STRING and then to Decimal. This - allows perfect precision for the Numeric type when - using cx_oracle. - - .. change:: - :tags: oracle - :tickets: - - STRING/FIXED_CHAR now convert to unicode natively. - SQLAlchemy's String types then don't need to - apply any kind of conversions. - - .. change:: - :tags: firebird - :tickets: - - The functionality of result.rowcount can be disabled on a - per-engine basis by setting 'enable_rowcount=False' - on create_engine(). Normally, cursor.rowcount is called - after any UPDATE or DELETE statement unconditionally, - because the cursor is then closed and Firebird requires - an open cursor in order to get a rowcount. This - call is slightly expensive however so it can be disabled. - To re-enable on a per-execution basis, the - 'enable_rowcount=True' execution option may be used. - - .. change:: - :tags: examples - :tickets: - - Updated attribute_shard.py example to use a more robust - method of searching a Query for binary expressions which - compare columns against literal values. - -.. changelog:: - :version: 0.6beta3 - :released: Sun Mar 28 2010 - - .. change:: - :tags: orm - :tickets: 1675 - - Major feature: Added new "subquery" loading capability to - relationship(). This is an eager loading option which - generates a second SELECT for each collection represented - in a query, across all parents at once. The query - re-issues the original end-user query wrapped in a subquery, - applies joins out to the target collection, and loads - all those collections fully in one result, similar to - "joined" eager loading but using all inner joins and not - re-fetching full parent rows repeatedly (as most DBAPIs seem - to do, even if columns are skipped). Subquery loading is - available at mapper config level using "lazy='subquery'" and - at the query options level using "subqueryload(props..)", - "subqueryload_all(props...)". - - .. change:: - :tags: orm - :tickets: - - To accommodate the fact that there are now two kinds of eager - loading available, the new names for eagerload() and - eagerload_all() are joinedload() and joinedload_all(). The - old names will remain as synonyms for the foreseeable future. - - .. change:: - :tags: orm - :tickets: - - The "lazy" flag on the relationship() function now accepts - a string argument for all kinds of loading: "select", "joined", - "subquery", "noload" and "dynamic", where the default is now - "select". The old values of True/ - False/None still retain their usual meanings and will remain - as synonyms for the foreseeable future. - - .. change:: - :tags: orm - :tickets: 921 - - Added with_hint() method to Query() construct. This calls - directly down to select().with_hint() and also accepts - entities as well as tables and aliases. See with_hint() in the - SQL section below. - - .. change:: - :tags: orm - :tickets: - - Fixed bug in Query whereby calling q.join(prop).from_self(...). - join(prop) would fail to render the second join outside the - subquery, when joining on the same criterion as was on the - inside. - - .. change:: - :tags: orm - :tickets: - - Fixed bug in Query whereby the usage of aliased() constructs - would fail if the underlying table (but not the actual alias) - were referenced inside the subquery generated by - q.from_self() or q.select_from(). - - .. change:: - :tags: orm - :tickets: - - Fixed bug which affected all eagerload() and similar options - such that "remote" eager loads, i.e. eagerloads off of a lazy - load such as query(A).options(eagerload(A.b, B.c)) - wouldn't eagerload anything, but using eagerload("b.c") would - work fine. - - .. change:: - :tags: orm - :tickets: - - Query gains an add_columns(\*columns) method which is a multi- - version of add_column(col). add_column(col) is future - deprecated. - - .. change:: - :tags: orm - :tickets: - - Query.join() will detect if the end result will be - "FROM A JOIN A", and will raise an error if so. - - .. change:: - :tags: orm - :tickets: - - Query.join(Cls.propname, from_joinpoint=True) will check more - carefully that "Cls" is compatible with the current joinpoint, - and act the same way as Query.join("propname", from_joinpoint=True) - in that regard. - - .. change:: - :tags: sql - :tickets: 921 - - Added with_hint() method to select() construct. Specify - a table/alias, hint text, and optional dialect name, and - "hints" will be rendered in the appropriate place in the - statement. Works for Oracle, Sybase, MySQL. - - .. change:: - :tags: sql - :tickets: 1747 - - Fixed bug introduced in 0.6beta2 where column labels would - render inside of column expressions already assigned a label. - - .. change:: - :tags: postgresql - :tickets: 877 - - The psycopg2 dialect will log NOTICE messages via the - "sqlalchemy.dialects.postgresql" logger name. - - .. change:: - :tags: postgresql - :tickets: 997 - - the TIME and TIMESTAMP types are now available from the - postgresql dialect directly, which add the PG-specific - argument 'precision' to both. 'precision' and - 'timezone' are correctly reflected for both TIME and - TIMEZONE types. - - .. change:: - :tags: mysql - :tickets: 1752 - - No longer guessing that TINYINT(1) should be BOOLEAN - when reflecting - TINYINT(1) is returned. Use Boolean/ - BOOLEAN in table definition to get boolean conversion - behavior. - - .. change:: - :tags: oracle - :tickets: 1744 - - The Oracle dialect will issue VARCHAR type definitions - using character counts, i.e. VARCHAR2(50 CHAR), so that - the column is sized in terms of characters and not bytes. - Column reflection of character types will also use - ALL_TAB_COLUMNS.CHAR_LENGTH instead of - ALL_TAB_COLUMNS.DATA_LENGTH. Both of these behaviors take - effect when the server version is 9 or higher - for - version 8, the old behaviors are used. - - .. change:: - :tags: declarative - :tickets: 1746 - - Using a mixin won't break if the mixin implements an - unpredictable __getattribute__(), i.e. Zope interfaces. - - .. change:: - :tags: declarative - :tickets: 1749 - - Using @classdecorator and similar on mixins to define - __tablename__, __table_args__, etc. now works if - the method references attributes on the ultimate - subclass. - - .. change:: - :tags: declarative - :tickets: 1751 - - relationships and columns with foreign keys aren't - allowed on declarative mixins, sorry. - - .. change:: - :tags: ext - :tickets: - - The sqlalchemy.orm.shard module now becomes an extension, - sqlalchemy.ext.horizontal_shard. The old import - works with a deprecation warning. - -.. changelog:: - :version: 0.6beta2 - :released: Sat Mar 20 2010 - - .. change:: - :tags: py3k - :tickets: - - Improved the installation/test setup regarding Python 3, - now that Distribute runs on Py3k. distribute_setup.py - is now included. See README.py3k for Python 3 installation/ - testing instructions. - - .. change:: - :tags: orm - :tickets: 1740 - - The official name for the relation() function is now - relationship(), to eliminate confusion over the relational - algebra term. relation() however will remain available - in equal capacity for the foreseeable future. - - .. change:: - :tags: orm - :tickets: 1692 - - Added "version_id_generator" argument to Mapper, this is a - callable that, given the current value of the "version_id_col", - returns the next version number. Can be used for alternate - versioning schemes such as uuid, timestamps. - - .. change:: - :tags: orm - :tickets: - - added "lockmode" kw argument to Session.refresh(), will - pass through the string value to Query the same as - in with_lockmode(), will also do version check for a - version_id_col-enabled mapping. - - .. change:: - :tags: orm - :tickets: 1188 - - Fixed bug whereby calling query(A).join(A.bs).add_entity(B) - in a joined inheritance scenario would double-add B as a - target and produce an invalid query. - - .. change:: - :tags: orm - :tickets: 1674 - - Fixed bug in session.rollback() which involved not removing - formerly "pending" objects from the session before - re-integrating "deleted" objects, typically occurred with - natural primary keys. If there was a primary key conflict - between them, the attach of the deleted would fail - internally. The formerly "pending" objects are now expunged - first. - - .. change:: - :tags: orm - :tickets: 1719 - - Removed a lot of logging that nobody really cares about, - logging that remains will respond to live changes in the - log level. No significant overhead is added. - - .. change:: - :tags: orm - :tickets: - - Fixed bug in session.merge() which prevented dict-like - collections from merging. - - .. change:: - :tags: orm - :tickets: - - session.merge() works with relations that specifically - don't include "merge" in their cascade options - the target - is ignored completely. - - .. change:: - :tags: orm - :tickets: 1681 - - session.merge() will not expire existing scalar attributes - on an existing target if the target has a value for that - attribute, even if the incoming merged doesn't have - a value for the attribute. This prevents unnecessary loads - on existing items. Will still mark the attr as expired - if the destination doesn't have the attr, though, which - fulfills some contracts of deferred cols. - - .. change:: - :tags: orm - :tickets: 1680 - - The "allow_null_pks" flag is now called "allow_partial_pks", - defaults to True, acts like it did in 0.5 again. Except, - it also is implemented within merge() such that a SELECT - won't be issued for an incoming instance with partially - NULL primary key if the flag is False. - - .. change:: - :tags: orm - :tickets: 1737 - - Fixed bug in 0.6-reworked "many-to-one" optimizations - such that a many-to-one that is against a non-primary key - column on the remote table (i.e. foreign key against a - UNIQUE column) will pull the "old" value in from the - database during a change, since if it's in the session - we will need it for proper history/backref accounting, - and we can't pull from the local identity map on a - non-primary key column. - - .. change:: - :tags: orm - :tickets: 1731 - - fixed internal error which would occur if calling has() - or similar complex expression on a single-table inheritance - relation(). - - .. change:: - :tags: orm - :tickets: 1688 - - query.one() no longer applies LIMIT to the query, this to - ensure that it fully counts all object identities present - in the result, even in the case where joins may conceal - multiple identities for two or more rows. As a bonus, - one() can now also be called with a query that issued - from_statement() to start with since it no longer modifies - the query. - - .. change:: - :tags: orm - :tickets: 1727 - - query.get() now returns None if queried for an identifier - that is present in the identity map with a different class - than the one requested, i.e. when using polymorphic loading. - - .. change:: - :tags: orm - :tickets: 1706 - - A major fix in query.join(), when the "on" clause is an - attribute of an aliased() construct, but there is already - an existing join made out to a compatible target, query properly - joins to the right aliased() construct instead of sticking - onto the right side of the existing join. - - .. change:: - :tags: orm - :tickets: 1362 - - Slight improvement to the fix for to not issue - needless updates of the primary key column during a so-called - "row switch" operation, i.e. add + delete of two objects - with the same PK. - - .. change:: - :tags: orm - :tickets: - - Now uses sqlalchemy.orm.exc.DetachedInstanceError when an - attribute load or refresh action fails due to object - being detached from any Session. UnboundExecutionError - is specific to engines bound to sessions and statements. - - .. change:: - :tags: orm - :tickets: - - Query called in the context of an expression will render - disambiguating labels in all cases. Note that this does - not apply to the existing .statement and .subquery() - accessor/method, which still honors the .with_labels() - setting that defaults to False. - - .. change:: - :tags: orm - :tickets: 1676 - - Query.union() retains disambiguating labels within the - returned statement, thus avoiding various SQL composition - errors which can result from column name conflicts. - - .. change:: - :tags: orm - :tickets: - - Fixed bug in attribute history that inadvertently invoked - __eq__ on mapped instances. - - .. change:: - :tags: orm - :tickets: - - Some internal streamlining of object loading grants a - small speedup for large results, estimates are around - 10-15%. Gave the "state" internals a good solid - cleanup with less complexity, datamembers, - method calls, blank dictionary creates. - - .. change:: - :tags: orm - :tickets: 1689 - - Documentation clarification for query.delete() - - .. change:: - :tags: orm - :tickets: - - Fixed cascade bug in many-to-one relation() when attribute - was set to None, introduced in r6711 (cascade deleted - items into session during add()). - - .. change:: - :tags: orm - :tickets: 1736 - - Calling query.order_by() or query.distinct() before calling - query.select_from(), query.with_polymorphic(), or - query.from_statement() raises an exception now instead of - silently dropping those criterion. - - .. change:: - :tags: orm - :tickets: 1735 - - query.scalar() now raises an exception if more than one - row is returned. All other behavior remains the same. - - .. change:: - :tags: orm - :tickets: 1692 - - Fixed bug which caused "row switch" logic, that is an - INSERT and DELETE replaced by an UPDATE, to fail when - version_id_col was in use. - - .. change:: - :tags: sql - :tickets: 1714 - - join() will now simulate a NATURAL JOIN by default. Meaning, - if the left side is a join, it will attempt to join the right - side to the rightmost side of the left first, and not raise - any exceptions about ambiguous join conditions if successful - even if there are further join targets across the rest of - the left. - - .. change:: - :tags: sql - :tickets: - - The most common result processors conversion function were - moved to the new "processors" module. Dialect authors are - encouraged to use those functions whenever they correspond - to their needs instead of implementing custom ones. - - .. change:: - :tags: sql - :tickets: 1694, 1698 - - SchemaType and subclasses Boolean, Enum are now serializable, - including their ddl listener and other event callables. - - .. change:: - :tags: sql - :tickets: - - Some platforms will now interpret certain literal values - as non-bind parameters, rendered literally into the SQL - statement. This to support strict SQL-92 rules that are - enforced by some platforms including MS-SQL and Sybase. - In this model, bind parameters aren't allowed in the - columns clause of a SELECT, nor are certain ambiguous - expressions like "?=?". When this mode is enabled, the base - compiler will render the binds as inline literals, but only across - strings and numeric values. Other types such as dates - will raise an error, unless the dialect subclass defines - a literal rendering function for those. The bind parameter - must have an embedded literal value already or an error - is raised (i.e. won't work with straight bindparam('x')). - Dialects can also expand upon the areas where binds are not - accepted, such as within argument lists of functions - (which don't work on MS-SQL when native SQL binding is used). - - .. change:: - :tags: sql - :tickets: - - Added "unicode_errors" parameter to String, Unicode, etc. - Behaves like the 'errors' keyword argument to - the standard library's string.decode() functions. This flag - requires that `convert_unicode` is set to `"force"` - otherwise, - SQLAlchemy is not guaranteed to handle the task of unicode - conversion. Note that this flag adds significant performance - overhead to row-fetching operations for backends that already - return unicode objects natively (which most DBAPIs do). This - flag should only be used as an absolute last resort for reading - strings from a column with varied or corrupted encodings, - which only applies to databases that accept invalid encodings - in the first place (i.e. MySQL. *not* PG, Sqlite, etc.) - - .. change:: - :tags: sql - :tickets: - - Added math negation operator support, -x. - - .. change:: - :tags: sql - :tickets: - - FunctionElement subclasses are now directly executable the - same way any func.foo() construct is, with automatic - SELECT being applied when passed to execute(). - - .. change:: - :tags: sql - :tickets: - - The "type" and "bind" keyword arguments of a func.foo() - construct are now local to "func." constructs and are - not part of the FunctionElement base class, allowing - a "type" to be handled in a custom constructor or - class-level variable. - - .. change:: - :tags: sql - :tickets: - - Restored the keys() method to ResultProxy. - - .. change:: - :tags: sql - :tickets: 1647, 1683 - - The type/expression system now does a more complete job - of determining the return type from an expression - as well as the adaptation of the Python operator into - a SQL operator, based on the full left/right/operator - of the given expression. In particular - the date/time/interval system created for PostgreSQL - EXTRACT in has now been generalized into - the type system. The previous behavior which often - occurred of an expression "column + literal" forcing - the type of "literal" to be the same as that of "column" - will now usually not occur - the type of - "literal" is first derived from the Python type of the - literal, assuming standard native Python types + date - types, before falling back to that of the known type - on the other side of the expression. If the - "fallback" type is compatible (i.e. CHAR from String), - the literal side will use that. TypeDecorator - types override this by default to coerce the "literal" - side unconditionally, which can be changed by implementing - the coerce_compared_value() method. Also part of. - - .. change:: - :tags: sql - :tickets: - - Made sqlalchemy.sql.expressions.Executable part of public - API, used for any expression construct that can be sent to - execute(). FunctionElement now inherits Executable so that - it gains execution_options(), which are also propagated - to the select() that's generated within execute(). - Executable in turn subclasses _Generative which marks - any ClauseElement that supports the @_generative - decorator - these may also become "public" for the benefit - of the compiler extension at some point. - - .. change:: - :tags: sql - :tickets: 1579 - - A change to the solution for - an end-user - defined bind parameter name that directly conflicts with - a column-named bind generated directly from the SET or - VALUES clause of an update/insert generates a compile error. - This reduces call counts and eliminates some cases where - undesirable name conflicts could still occur. - - .. change:: - :tags: sql - :tickets: 1705 - - Column() requires a type if it has no foreign keys (this is - not new). An error is now raised if a Column() has no type - and no foreign keys. - - .. change:: - :tags: sql - :tickets: 1717 - - the "scale" argument of the Numeric() type is honored when - coercing a returned floating point value into a string - on its way to Decimal - this allows accuracy to function - on SQLite, MySQL. - - .. change:: - :tags: sql - :tickets: - - the copy() method of Column now copies over uninitialized - "on table attach" events. Helps with the new declarative - "mixin" capability. - - .. change:: - :tags: engines - :tickets: - - Added an optional C extension to speed up the sql layer by - reimplementing RowProxy and the most common result processors. - The actual speedups will depend heavily on your DBAPI and - the mix of datatypes used in your tables, and can vary from - a 30% improvement to more than 200%. It also provides a modest - (~15-20%) indirect improvement to ORM speed for large queries. - Note that it is *not* built/installed by default. - See README for installation instructions. - - .. change:: - :tags: engines - :tickets: - - the execution sequence pulls all rowcount/last inserted ID - info from the cursor before commit() is called on the - DBAPI connection in an "autocommit" scenario. This helps - mxodbc with rowcount and is probably a good idea overall. - - .. change:: - :tags: engines - :tickets: 1719 - - Opened up logging a bit such that isEnabledFor() is called - more often, so that changes to the log level for engine/pool - will be reflected on next connect. This adds a small - amount of method call overhead. It's negligible and will make - life a lot easier for all those situations when logging - just happens to be configured after create_engine() is called. - - .. change:: - :tags: engines - :tickets: - - The assert_unicode flag is deprecated. SQLAlchemy will raise - a warning in all cases where it is asked to encode a non-unicode - Python string, as well as when a Unicode or UnicodeType type - is explicitly passed a bytestring. The String type will do nothing - for DBAPIs that already accept Python unicode objects. - - .. change:: - :tags: engines - :tickets: - - Bind parameters are sent as a tuple instead of a list. Some - backend drivers will not accept bind parameters as a list. - - .. change:: - :tags: engines - :tickets: - - threadlocal engine wasn't properly closing the connection - upon close() - fixed that. - - .. change:: - :tags: engines - :tickets: - - Transaction object doesn't rollback or commit if it isn't - "active", allows more accurate nesting of begin/rollback/commit. - - .. change:: - :tags: engines - :tickets: - - Python unicode objects as binds result in the Unicode type, - not string, thus eliminating a certain class of unicode errors - on drivers that don't support unicode binds. - - .. change:: - :tags: engines - :tickets: 1555 - - Added "logging_name" argument to create_engine(), Pool() constructor - as well as "pool_logging_name" argument to create_engine() which - filters down to that of Pool. Issues the given string name - within the "name" field of logging messages instead of the default - hex identifier string. - - .. change:: - :tags: engines - :tickets: - - The visit_pool() method of Dialect is removed, and replaced with - on_connect(). This method returns a callable which receives - the raw DBAPI connection after each one is created. The callable - is assembled into a first_connect/connect pool listener by the - connection strategy if non-None. Provides a simpler interface - for dialects. - - .. change:: - :tags: engines - :tickets: 1728 - - StaticPool now initializes, disposes and recreates without - opening a new connection - the connection is only opened when - first requested. dispose() also works on AssertionPool now. - - .. change:: - :tags: ticket: 1673, metadata - :tickets: - - Added the ability to strip schema information when using - "tometadata" by passing "schema=None" as an argument. If schema - is not specified then the table's schema is retained. - - .. change:: - :tags: declarative - :tickets: - - DeclarativeMeta exclusively uses cls.__dict__ (not dict\_) - as the source of class information; _as_declarative exclusively - uses the dict\_ passed to it as the source of class information - (which when using DeclarativeMeta is cls.__dict__). This should - in theory make it easier for custom metaclasses to modify - the state passed into _as_declarative. - - .. change:: - :tags: declarative - :tickets: 1707 - - declarative now accepts mixin classes directly, as a means - to provide common functional and column-based elements on - all subclasses, as well as a means to propagate a fixed - set of __table_args__ or __mapper_args__ to subclasses. - For custom combinations of __table_args__/__mapper_args__ from - an inherited mixin to local, descriptors can now be used. - New details are all up in the Declarative documentation. - Thanks to Chris Withers for putting up with my strife - on this. - - .. change:: - :tags: declarative - :tickets: 1393 - - the __mapper_args__ dict is copied when propagating to a subclass, - and is taken straight off the class __dict__ to avoid any - propagation from the parent. mapper inheritance already - propagates the things you want from the parent mapper. - - .. change:: - :tags: declarative - :tickets: 1732 - - An exception is raised when a single-table subclass specifies - a column that is already present on the base class. - - .. change:: - :tags: mysql - :tickets: 1655 - - Fixed reflection bug whereby when COLLATE was present, - nullable flag and server defaults would not be reflected. - - .. change:: - :tags: mysql - :tickets: - - Fixed reflection of TINYINT(1) "boolean" columns defined with - integer flags like UNSIGNED. - - .. change:: - :tags: mysql - :tickets: 1668 - - Further fixes for the mysql-connector dialect. - - .. change:: - :tags: mysql - :tickets: 1496 - - Composite PK table on InnoDB where the "autoincrement" column - isn't first will emit an explicit "KEY" phrase within - CREATE TABLE thereby avoiding errors. - - .. change:: - :tags: mysql - :tickets: 1634 - - Added reflection/create table support for a wide range - of MySQL keywords. - - .. change:: - :tags: mysql - :tickets: 1580 - - Fixed import error which could occur reflecting tables on - a Windows host - - .. change:: - :tags: mssql - :tickets: - - Re-established support for the pymssql dialect. - - .. change:: - :tags: mssql - :tickets: - - Various fixes for implicit returning, reflection, - etc. - the MS-SQL dialects aren't quite complete - in 0.6 yet (but are close) - - .. change:: - :tags: mssql - :tickets: 1710 - - Added basic support for mxODBC. - - .. change:: - :tags: mssql - :tickets: - - Removed the text_as_varchar option. - - .. change:: - :tags: oracle - :tickets: - - "out" parameters require a type that is supported by - cx_oracle. An error will be raised if no cx_oracle - type can be found. - - .. change:: - :tags: oracle - :tickets: - - Oracle 'DATE' now does not perform any result processing, - as the DATE type in Oracle stores full date+time objects, - that's what you'll get. Note that the generic types.Date - type *will* still call value.date() on incoming values, - however. When reflecting a table, the reflected type - will be 'DATE'. - - .. change:: - :tags: oracle - :tickets: 1670 - - Added preliminary support for Oracle's WITH_UNICODE - mode. At the very least this establishes initial - support for cx_Oracle with Python 3. When WITH_UNICODE - mode is used in Python 2.xx, a large and scary warning - is emitted asking that the user seriously consider - the usage of this difficult mode of operation. - - .. change:: - :tags: oracle - :tickets: 1712 - - The except_() method now renders as MINUS on Oracle, - which is more or less equivalent on that platform. - - .. change:: - :tags: oracle - :tickets: 651 - - Added support for rendering and reflecting - TIMESTAMP WITH TIME ZONE, i.e. TIMESTAMP(timezone=True). - - .. change:: - :tags: oracle - :tickets: - - Oracle INTERVAL type can now be reflected. - - .. change:: - :tags: sqlite - :tickets: 1685 - - Added "native_datetime=True" flag to create_engine(). - This will cause the DATE and TIMESTAMP types to skip - all bind parameter and result row processing, under - the assumption that PARSE_DECLTYPES has been enabled - on the connection. Note that this is not entirely - compatible with the "func.current_date()", which - will be returned as a string. - - .. change:: - :tags: sybase - :tickets: - - Implemented a preliminary working dialect for Sybase, - with sub-implementations for Python-Sybase as well - as Pyodbc. Handles table - creates/drops and basic round trip functionality. - Does not yet include reflection or comprehensive - support of unicode/special expressions/etc. - - .. change:: - :tags: examples - :tickets: - - Changed the beaker cache example a bit to have a separate - RelationCache option for lazyload caching. This object - does a lookup among any number of potential attributes - more efficiently by grouping several into a common structure. - Both FromCache and RelationCache are simpler individually. - - .. change:: - :tags: documentation - :tickets: 1700 - - Major cleanup work in the docs to link class, function, and - method names into the API docs. - -.. changelog:: - :version: 0.6beta1 - :released: Wed Feb 03 2010 - - .. change:: - :tags: release, major - :tickets: - - For the full set of feature descriptions, see - https://docs.sqlalchemy.org/en/latest/changelog/migration_06.html . - This document is a work in progress. - - .. change:: - :tags: release, major - :tickets: - - All bug fixes and feature enhancements from the most - recent 0.5 version and below are also included within 0.6. - - .. change:: - :tags: release, major - :tickets: - - Platforms targeted now include Python 2.4/2.5/2.6, Python - 3.1, Jython2.5. - - .. change:: - :tags: orm - :tickets: - - Changes to query.update() and query.delete(): - - the 'expire' option on query.update() has been renamed to - 'fetch', thus matching that of query.delete(). - 'expire' is deprecated and issues a warning. - - - query.update() and query.delete() both default to - 'evaluate' for the synchronize strategy. - - - the 'synchronize' strategy for update() and delete() - raises an error on failure. There is no implicit fallback - onto "fetch". Failure of evaluation is based on the - structure of criteria, so success/failure is deterministic - based on code structure. - - .. change:: - :tags: orm - :tickets: 1186, 1492, 1544 - - Enhancements on many-to-one relations: - - many-to-one relations now fire off a lazyload in fewer - cases, including in most cases will not fetch the "old" - value when a new one is replaced. - - - many-to-one relation to a joined-table subclass now uses - get() for a simple load (known as the "use_get" - condition), i.e. Related->Sub(Base), without the need to - redefine the primaryjoin condition in terms of the base - table. - - - specifying a foreign key with a declarative column, i.e. - ForeignKey(MyRelatedClass.id) doesn't break the "use_get" - condition from taking place - - - relation(), eagerload(), and eagerload_all() now feature - an option called "innerjoin". Specify `True` or `False` to - control whether an eager join is constructed as an INNER - or OUTER join. Default is `False` as always. The mapper - options will override whichever setting is specified on - relation(). Should generally be set for many-to-one, not - nullable foreign key relations to allow improved join - performance. - - - the behavior of eagerloading such that the main query is - wrapped in a subquery when LIMIT/OFFSET are present now - makes an exception for the case when all eager loads are - many-to-one joins. In those cases, the eager joins are - against the parent table directly along with the - limit/offset without the extra overhead of a subquery, - since a many-to-one join does not add rows to the result. - - .. change:: - :tags: orm - :tickets: - - Enhancements / Changes on Session.merge(): - - .. change:: - :tags: orm - :tickets: - - the "dont_load=True" flag on Session.merge() is deprecated - and is now "load=False". - - .. change:: - :tags: orm - :tickets: - - Session.merge() is performance optimized, using half the - call counts for "load=False" mode compared to 0.5 and - significantly fewer SQL queries in the case of collections - for "load=True" mode. - - .. change:: - :tags: orm - :tickets: - - merge() will not issue a needless merge of attributes if the - given instance is the same instance which is already present. - - .. change:: - :tags: orm - :tickets: - - merge() now also merges the "options" associated with a given - state, i.e. those passed through query.options() which follow - along with an instance, such as options to eagerly- or - lazyily- load various attributes. This is essential for - the construction of highly integrated caching schemes. This - is a subtle behavioral change vs. 0.5. - - .. change:: - :tags: orm - :tickets: - - A bug was fixed regarding the serialization of the "loader - path" present on an instance's state, which is also necessary - when combining the usage of merge() with serialized state - and associated options that should be preserved. - - .. change:: - :tags: orm - :tickets: - - The all new merge() is showcased in a new comprehensive - example of how to integrate Beaker with SQLAlchemy. See - the notes in the "examples" note below. - - .. change:: - :tags: orm - :tickets: 1362 - - Primary key values can now be changed on a joined-table inheritance - object, and ON UPDATE CASCADE will be taken into account when - the flush happens. Set the new "passive_updates" flag to False - on mapper() when using SQLite or MySQL/MyISAM. - - .. change:: - :tags: orm - :tickets: 1671 - - flush() now detects when a primary key column was updated by - an ON UPDATE CASCADE operation from another primary key, and - can then locate the row for a subsequent UPDATE on the new PK - value. This occurs when a relation() is there to establish - the relationship as well as passive_updates=True. - - .. change:: - :tags: orm - :tickets: - - the "save-update" cascade will now cascade the pending *removed* - values from a scalar or collection attribute into the new session - during an add() operation. This so that the flush() operation - will also delete or modify rows of those disconnected items. - - .. change:: - :tags: orm - :tickets: 1531 - - Using a "dynamic" loader with a "secondary" table now produces - a query where the "secondary" table is *not* aliased. This - allows the secondary Table object to be used in the "order_by" - attribute of the relation(), and also allows it to be used - in filter criterion against the dynamic relation. - - .. change:: - :tags: orm - :tickets: 1643 - - relation() with uselist=False will emit a warning when - an eager or lazy load locates more than one valid value for - the row. This may be due to primaryjoin/secondaryjoin - conditions which aren't appropriate for an eager LEFT OUTER - JOIN or for other conditions. - - .. change:: - :tags: orm - :tickets: 1633 - - an explicit check occurs when a synonym() is used with - map_column=True, when a ColumnProperty (deferred or otherwise) - exists separately in the properties dictionary sent to mapper - with the same keyname. Instead of silently replacing - the existing property (and possible options on that property), - an error is raised. - - .. change:: - :tags: orm - :tickets: - - a "dynamic" loader sets up its query criterion at construction - time so that the actual query is returned from non-cloning - accessors like "statement". - - .. change:: - :tags: orm - :tickets: - - the "named tuple" objects returned when iterating a - Query() are now pickleable. - - .. change:: - :tags: orm - :tickets: 1542 - - mapping to a select() construct now requires that you - make an alias() out of it distinctly. This to eliminate - confusion over such issues as - - .. change:: - :tags: orm - :tickets: 1537 - - query.join() has been reworked to provide more consistent - behavior and more flexibility (includes) - - .. change:: - :tags: orm - :tickets: - - query.select_from() accepts multiple clauses to produce - multiple comma separated entries within the FROM clause. - Useful when selecting from multiple-homed join() clauses. - - .. change:: - :tags: orm - :tickets: - - query.select_from() also accepts mapped classes, aliased() - constructs, and mappers as arguments. In particular this - helps when querying from multiple joined-table classes to ensure - the full join gets rendered. - - .. change:: - :tags: orm - :tickets: 1135 - - query.get() can be used with a mapping to an outer join - where one or more of the primary key values are None. - - .. change:: - :tags: orm - :tickets: 1568 - - query.from_self(), query.union(), others which do a - "SELECT * from (SELECT...)" type of nesting will do - a better job translating column expressions within the subquery - to the columns clause of the outer query. This is - potentially backwards incompatible with 0.5, in that this - may break queries with literal expressions that do not have labels - applied (i.e. literal('foo'), etc.) - - .. change:: - :tags: orm - :tickets: 1622 - - relation primaryjoin and secondaryjoin now check that they - are column-expressions, not just clause elements. this prohibits - things like FROM expressions being placed there directly. - - .. change:: - :tags: orm - :tickets: 1415 - - `expression.null()` is fully understood the same way - None is when comparing an object/collection-referencing - attribute within query.filter(), filter_by(), etc. - - .. change:: - :tags: orm - :tickets: 1052 - - added "make_transient()" helper function which transforms a - persistent/ detached instance into a transient one (i.e. - deletes the instance_key and removes from any session.) - - .. change:: - :tags: orm - :tickets: 1339 - - the allow_null_pks flag on mapper() is deprecated, and - the feature is turned "on" by default. This means that - a row which has a non-null value for any of its primary key - columns will be considered an identity. The need for this - scenario typically only occurs when mapping to an outer join. - - .. change:: - :tags: orm - :tickets: - - the mechanics of "backref" have been fully merged into the - finer grained "back_populates" system, and take place entirely - within the _generate_backref() method of RelationProperty. This - makes the initialization procedure of RelationProperty - simpler and allows easier propagation of settings (such as from - subclasses of RelationProperty) into the reverse reference. - The internal BackRef() is gone and backref() returns a plain - tuple that is understood by RelationProperty. - - .. change:: - :tags: orm - :tickets: 1569 - - The version_id_col feature on mapper() will raise a warning when - used with dialects that don't support "rowcount" adequately. - - .. change:: - :tags: orm - :tickets: - - added "execution_options()" to Query, to so options can be - passed to the resulting statement. Currently only - Select-statements have these options, and the only option - used is "stream_results", and the only dialect which knows - "stream_results" is psycopg2. - - .. change:: - :tags: orm - :tickets: - - Query.yield_per() will set the "stream_results" statement - option automatically. - - .. change:: - :tags: orm - :tickets: - - Deprecated or removed: - * 'allow_null_pks' flag on mapper() is deprecated. It does - nothing now and the setting is "on" in all cases. - * 'transactional' flag on sessionmaker() and others is - removed. Use 'autocommit=True' to indicate 'transactional=False'. - * 'polymorphic_fetch' argument on mapper() is removed. - Loading can be controlled using the 'with_polymorphic' - option. - * 'select_table' argument on mapper() is removed. Use - 'with_polymorphic=("*", )' for this - functionality. - * 'proxy' argument on synonym() is removed. This flag - did nothing throughout 0.5, as the "proxy generation" - behavior is now automatic. - * Passing a single list of elements to eagerload(), - eagerload_all(), contains_eager(), lazyload(), - defer(), and undefer() instead of multiple positional - \*args is deprecated. - * Passing a single list of elements to query.order_by(), - query.group_by(), query.join(), or query.outerjoin() - instead of multiple positional \*args is deprecated. - * query.iterate_instances() is removed. Use query.instances(). - * Query.query_from_parent() is removed. Use the - sqlalchemy.orm.with_parent() function to produce a - "parent" clause, or alternatively query.with_parent(). - * query._from_self() is removed, use query.from_self() - instead. - * the "comparator" argument to composite() is removed. - Use "comparator_factory". - * RelationProperty._get_join() is removed. - * the 'echo_uow' flag on Session is removed. Use - logging on the "sqlalchemy.orm.unitofwork" name. - * session.clear() is removed. use session.expunge_all(). - * session.save(), session.update(), session.save_or_update() - are removed. Use session.add() and session.add_all(). - * the "objects" flag on session.flush() remains deprecated. - * the "dont_load=True" flag on session.merge() is deprecated - in favor of "load=False". - * ScopedSession.mapper remains deprecated. See the - usage recipe at - https://www.sqlalchemy.org/trac/wiki/UsageRecipes/SessionAwareMapper - * passing an InstanceState (internal SQLAlchemy state object) to - attributes.init_collection() or attributes.get_history() is - deprecated. These functions are public API and normally - expect a regular mapped object instance. - * the 'engine' parameter to declarative_base() is removed. - Use the 'bind' keyword argument. - - .. change:: - :tags: sql - :tickets: - - the "autocommit" flag on select() and text() as well - as select().autocommit() are deprecated - now call - .execution_options(autocommit=True) on either of those - constructs, also available directly on Connection and orm.Query. - - .. change:: - :tags: sql - :tickets: - - the autoincrement flag on column now indicates the column - which should be linked to cursor.lastrowid, if that method - is used. See the API docs for details. - - .. change:: - :tags: sql - :tickets: 1566 - - an executemany() now requires that all bound parameter - sets require that all keys are present which are - present in the first bound parameter set. The structure - and behavior of an insert/update statement is very much - determined by the first parameter set, including which - defaults are going to fire off, and a minimum of - guesswork is performed with all the rest so that performance - is not impacted. For this reason defaults would otherwise - silently "fail" for missing parameters, so this is now guarded - against. - - .. change:: - :tags: sql - :tickets: - - returning() support is native to insert(), update(), - delete(). Implementations of varying levels of - functionality exist for PostgreSQL, Firebird, MSSQL and - Oracle. returning() can be called explicitly with column - expressions which are then returned in the resultset, - usually via fetchone() or first(). - - insert() constructs will also use RETURNING implicitly to - get newly generated primary key values, if the database - version in use supports it (a version number check is - performed). This occurs if no end-user returning() was - specified. - - .. change:: - :tags: sql - :tickets: 1665 - - union(), intersect(), except() and other "compound" types - of statements have more consistent behavior w.r.t. - parenthesizing. Each compound element embedded within - another will now be grouped with parenthesis - previously, - the first compound element in the list would not be grouped, - as SQLite doesn't like a statement to start with - parenthesis. However, PostgreSQL in particular has - precedence rules regarding INTERSECT, and it is - more consistent for parenthesis to be applied equally - to all sub-elements. So now, the workaround for SQLite - is also what the workaround for PG was previously - - when nesting compound elements, the first one usually needs - ".alias().select()" called on it to wrap it inside - of a subquery. - - .. change:: - :tags: sql - :tickets: 1579 - - insert() and update() constructs can now embed bindparam() - objects using names that match the keys of columns. These - bind parameters will circumvent the usual route to those - keys showing up in the VALUES or SET clause of the generated - SQL. - - .. change:: - :tags: sql - :tickets: 1524 - - the Binary type now returns data as a Python string - (or a "bytes" type in Python 3), instead of the built- - in "buffer" type. This allows symmetric round trips - of binary data. - - .. change:: - :tags: sql - :tickets: - - Added a tuple_() construct, allows sets of expressions - to be compared to another set, typically with IN against - composite primary keys or similar. Also accepts an - IN with multiple columns. The "scalar select can - have only one column" error message is removed - will - rely upon the database to report problems with - col mismatch. - - .. change:: - :tags: sql - :tickets: - - User-defined "default" and "onupdate" callables which - accept a context should now call upon - "context.current_parameters" to get at the dictionary - of bind parameters currently being processed. This - dict is available in the same way regardless of - single-execute or executemany-style statement execution. - - .. change:: - :tags: sql - :tickets: 1428 - - multi-part schema names, i.e. with dots such as - "dbo.master", are now rendered in select() labels - with underscores for dots, i.e. "dbo_master_table_column". - This is a "friendly" label that behaves better - in result sets. - - .. change:: - :tags: sql - :tickets: - - removed needless "counter" behavior with select() - labelnames that match a column name in the table, - i.e. generates "tablename_id" for "id", instead of - "tablename_id_1" in an attempt to avoid naming - conflicts, when the table has a column actually - named "tablename_id" - this is because - the labeling logic is always applied to all columns - so a naming conflict will never occur. - - .. change:: - :tags: sql - :tickets: 1628 - - calling expr.in_([]), i.e. with an empty list, emits a warning - before issuing the usual "expr != expr" clause. The - "expr != expr" can be very expensive, and it's preferred - that the user not issue in_() if the list is empty, - instead simply not querying, or modifying the criterion - as appropriate for more complex situations. - - .. change:: - :tags: sql - :tickets: - - Added "execution_options()" to select()/text(), which set the - default options for the Connection. See the note in "engines". - - .. change:: - :tags: sql - :tickets: 1131 - - Deprecated or removed: - * "scalar" flag on select() is removed, use - select.as_scalar(). - * "shortname" attribute on bindparam() is removed. - * postgres_returning, firebird_returning flags on - insert(), update(), delete() are deprecated, use - the new returning() method. - * fold_equivalents flag on join is deprecated (will remain - until is implemented) - - .. change:: - :tags: engines - :tickets: 443 - - transaction isolation level may be specified with - create_engine(... isolation_level="..."); available on - postgresql and sqlite. - - .. change:: - :tags: engines - :tickets: - - Connection has execution_options(), generative method - which accepts keywords that affect how the statement - is executed w.r.t. the DBAPI. Currently supports - "stream_results", causes psycopg2 to use a server - side cursor for that statement, as well as - "autocommit", which is the new location for the "autocommit" - option from select() and text(). select() and - text() also have .execution_options() as well as - ORM Query(). - - .. change:: - :tags: engines - :tickets: 1630 - - fixed the import for entrypoint-driven dialects to - not rely upon silly tb_info trick to determine import - error status. - - .. change:: - :tags: engines - :tickets: - - added first() method to ResultProxy, returns first row and - closes result set immediately. - - .. change:: - :tags: engines - :tickets: - - RowProxy objects are now pickleable, i.e. the object returned - by result.fetchone(), result.fetchall() etc. - - .. change:: - :tags: engines - :tickets: - - RowProxy no longer has a close() method, as the row no longer - maintains a reference to the parent. Call close() on - the parent ResultProxy instead, or use autoclose. - - .. change:: - :tags: engines - :tickets: 1586 - - ResultProxy internals have been overhauled to greatly reduce - method call counts when fetching columns. Can provide a large - speed improvement (up to more than 100%) when fetching large - result sets. The improvement is larger when fetching columns - that have no type-level processing applied and when using - results as tuples (instead of as dictionaries). Many - thanks to Elixir's Gaëtan de Menten for this dramatic - improvement ! - - .. change:: - :tags: engines - :tickets: - - Databases which rely upon postfetch of "last inserted id" - to get at a generated sequence value (i.e. MySQL, MS-SQL) - now work correctly when there is a composite primary key - where the "autoincrement" column is not the first primary - key column in the table. - - .. change:: - :tags: engines - :tickets: - - the last_inserted_ids() method has been renamed to the - descriptor "inserted_primary_key". - - .. change:: - :tags: engines - :tickets: 1554 - - setting echo=False on create_engine() now sets the loglevel - to WARN instead of NOTSET. This so that logging can be - disabled for a particular engine even if logging - for "sqlalchemy.engine" is enabled overall. Note that the - default setting of "echo" is `None`. - - .. change:: - :tags: engines - :tickets: - - ConnectionProxy now has wrapper methods for all transaction - lifecycle events, including begin(), rollback(), commit() - begin_nested(), begin_prepared(), prepare(), release_savepoint(), - etc. - - .. change:: - :tags: engines - :tickets: - - Connection pool logging now uses both INFO and DEBUG - log levels for logging. INFO is for major events such - as invalidated connections, DEBUG for all the acquire/return - logging. `echo_pool` can be False, None, True or "debug" - the same way as `echo` works. - - .. change:: - :tags: engines - :tickets: 1621 - - All pyodbc-dialects now support extra pyodbc-specific - kw arguments 'ansi', 'unicode_results', 'autocommit'. - - .. change:: - :tags: engines - :tickets: - - the "threadlocal" engine has been rewritten and simplified - and now supports SAVEPOINT operations. - - .. change:: - :tags: engines - :tickets: - - deprecated or removed - * result.last_inserted_ids() is deprecated. Use - result.inserted_primary_key - * dialect.get_default_schema_name(connection) is now - public via dialect.default_schema_name. - * the "connection" argument from engine.transaction() and - engine.run_callable() is removed - Connection itself - now has those methods. All four methods accept - \*args and \**kwargs which are passed to the given callable, - as well as the operating connection. - - .. change:: - :tags: schema - :tickets: 1541 - - the `__contains__()` method of `MetaData` now accepts - strings or `Table` objects as arguments. If given - a `Table`, the argument is converted to `table.key` first, - i.e. "[schemaname.]" - - .. change:: - :tags: schema - :tickets: - - deprecated MetaData.connect() and - ThreadLocalMetaData.connect() have been removed - send - the "bind" attribute to bind a metadata. - - .. change:: - :tags: schema - :tickets: - - deprecated metadata.table_iterator() method removed (use - sorted_tables) - - .. change:: - :tags: schema - :tickets: - - deprecated PassiveDefault - use DefaultClause. - - .. change:: - :tags: schema - :tickets: - - the "metadata" argument is removed from DefaultGenerator - and subclasses, but remains locally present on Sequence, - which is a standalone construct in DDL. - - .. change:: - :tags: schema - :tickets: - - Removed public mutability from Index and Constraint - objects: - - * ForeignKeyConstraint.append_element() - * Index.append_column() - * UniqueConstraint.append_column() - * PrimaryKeyConstraint.add() - * PrimaryKeyConstraint.remove() - - These should be constructed declaratively (i.e. in one - construction). - - .. change:: - :tags: schema - :tickets: 1545 - - The "start" and "increment" attributes on Sequence now - generate "START WITH" and "INCREMENT BY" by default, - on Oracle and PostgreSQL. Firebird doesn't support - these keywords right now. - - .. change:: - :tags: schema - :tickets: - - UniqueConstraint, Index, PrimaryKeyConstraint all accept - lists of column names or column objects as arguments. - - .. change:: - :tags: schema - :tickets: - - Other removed things: - - Table.key (no idea what this was for) - - Table.primary_key is not assignable - use - table.append_constraint(PrimaryKeyConstraint(...)) - - Column.bind (get via column.table.bind) - - Column.metadata (get via column.table.metadata) - - Column.sequence (use column.default) - - ForeignKey(constraint=some_parent) (is now private _constraint) - - .. change:: - :tags: schema - :tickets: - - The use_alter flag on ForeignKey is now a shortcut option - for operations that can be hand-constructed using the - DDL() event system. A side effect of this refactor is - that ForeignKeyConstraint objects with use_alter=True - will *not* be emitted on SQLite, which does not support - ALTER for foreign keys. - - .. change:: - :tags: schema - :tickets: 1605 - - ForeignKey and ForeignKeyConstraint objects now correctly - copy() all their public keyword arguments. - - .. change:: - :tags: reflection/inspection - :tickets: - - Table reflection has been expanded and generalized into - a new API called "sqlalchemy.engine.reflection.Inspector". - The Inspector object provides fine-grained information about - a wide variety of schema information, with room for expansion, - including table names, column names, view definitions, sequences, - indexes, etc. - - .. change:: - :tags: reflection/inspection - :tickets: - - Views are now reflectable as ordinary Table objects. The same - Table constructor is used, with the caveat that "effective" - primary and foreign key constraints aren't part of the reflection - results; these have to be specified explicitly if desired. - - .. change:: - :tags: reflection/inspection - :tickets: - - The existing autoload=True system now uses Inspector underneath - so that each dialect need only return "raw" data about tables - and other objects - Inspector is the single place that information - is compiled into Table objects so that consistency is at a maximum. - - .. change:: - :tags: ddl - :tickets: - - the DDL system has been greatly expanded. the DDL() class - now extends the more generic DDLElement(), which forms the basis - of many new constructs: - - - CreateTable() - - DropTable() - - AddConstraint() - - DropConstraint() - - CreateIndex() - - DropIndex() - - CreateSequence() - - DropSequence() - - These support "on" and "execute-at()" just like plain DDL() - does. User-defined DDLElement subclasses can be created and - linked to a compiler using the sqlalchemy.ext.compiler extension. - - .. change:: - :tags: ddl - :tickets: - - The signature of the "on" callable passed to DDL() and - DDLElement() is revised as follows: - - ddl - the DDLElement object itself - event - the string event name. - target - previously "schema_item", the Table or MetaData object triggering the event. - connection - the Connection object in use for the operation. - \**kw - keyword arguments. In the case of MetaData before/after - create/drop, the list of Table objects for which - CREATE/DROP DDL is to be issued is passed as the kw - argument "tables". This is necessary for metadata-level - DDL that is dependent on the presence of specific tables. - - The "schema_item" attribute of DDL has been renamed to - "target". - - .. change:: - :tags: dialect, refactor - :tickets: - - Dialect modules are now broken into database dialects - plus DBAPI implementations. Connect URLs are now - preferred to be specified using dialect+driver://..., - i.e. "mysql+mysqldb://scott:tiger@localhost/test". See - the 0.6 documentation for examples. - - .. change:: - :tags: dialect, refactor - :tickets: - - the setuptools entrypoint for external dialects is now - called "sqlalchemy.dialects". - - .. change:: - :tags: dialect, refactor - :tickets: - - the "owner" keyword argument is removed from Table. Use - "schema" to represent any namespaces to be prepended to - the table name. - - .. change:: - :tags: dialect, refactor - :tickets: - - server_version_info becomes a static attribute. - - .. change:: - :tags: dialect, refactor - :tickets: - - dialects receive an initialize() event on initial - connection to determine connection properties. - - .. change:: - :tags: dialect, refactor - :tickets: - - dialects receive a visit_pool event have an opportunity - to establish pool listeners. - - .. change:: - :tags: dialect, refactor - :tickets: - - cached TypeEngine classes are cached per-dialect class - instead of per-dialect. - - .. change:: - :tags: dialect, refactor - :tickets: - - new UserDefinedType should be used as a base class for - new types, which preserves the 0.5 behavior of - get_col_spec(). - - .. change:: - :tags: dialect, refactor - :tickets: - - The result_processor() method of all type classes now - accepts a second argument "coltype", which is the DBAPI - type argument from cursor.description. This argument - can help some types decide on the most efficient processing - of result values. - - .. change:: - :tags: dialect, refactor - :tickets: - - Deprecated Dialect.get_params() removed. - - .. change:: - :tags: dialect, refactor - :tickets: - - Dialect.get_rowcount() has been renamed to a descriptor - "rowcount", and calls cursor.rowcount directly. Dialects - which need to hardwire a rowcount in for certain calls - should override the method to provide different behavior. - - .. change:: - :tags: dialect, refactor - :tickets: 1566 - - DefaultRunner and subclasses have been removed. The job - of this object has been simplified and moved into - ExecutionContext. Dialects which support sequences should - add a `fire_sequence()` method to their execution context - implementation. - - .. change:: - :tags: dialect, refactor - :tickets: - - Functions and operators generated by the compiler now use - (almost) regular dispatch functions of the form - "visit_" and "visit__fn" to provide - customed processing. This replaces the need to copy the - "functions" and "operators" dictionaries in compiler - subclasses with straightforward visitor methods, and also - allows compiler subclasses complete control over - rendering, as the full _Function or _BinaryExpression - object is passed in. - - .. change:: - :tags: postgresql - :tickets: - - New dialects: pg8000, zxjdbc, and pypostgresql - on py3k. - - .. change:: - :tags: postgresql - :tickets: - - The "postgres" dialect is now named "postgresql" ! - Connection strings look like: - - postgresql://scott:tiger@localhost/test - postgresql+pg8000://scott:tiger@localhost/test - - The "postgres" name remains for backwards compatibility - in the following ways: - - - There is a "postgres.py" dummy dialect which - allows old URLs to work, i.e. - postgres://scott:tiger@localhost/test - - - The "postgres" name can be imported from the old - "databases" module, i.e. "from - sqlalchemy.databases import postgres" as well as - "dialects", "from sqlalchemy.dialects.postgres - import base as pg", will send a deprecation - warning. - - - Special expression arguments are now named - "postgresql_returning" and "postgresql_where", but - the older "postgres_returning" and - "postgres_where" names still work with a - deprecation warning. - - .. change:: - :tags: postgresql - :tickets: - - "postgresql_where" now accepts SQL expressions which - can also include literals, which will be quoted as needed. - - .. change:: - :tags: postgresql - :tickets: - - The psycopg2 dialect now uses psycopg2's "unicode extension" - on all new connections, which allows all String/Text/etc. - types to skip the need to post-process bytestrings into - unicode (an expensive step due to its volume). Other - dialects which return unicode natively (pg8000, zxjdbc) - also skip unicode post-processing. - - .. change:: - :tags: postgresql - :tickets: 1511 - - Added new ENUM type, which exists as a schema-level - construct and extends the generic Enum type. Automatically - associates itself with tables and their parent metadata - to issue the appropriate CREATE TYPE/DROP TYPE - commands as needed, supports unicode labels, supports - reflection. - - .. change:: - :tags: postgresql - :tickets: - - INTERVAL supports an optional "precision" argument - corresponding to the argument that PG accepts. - - .. change:: - :tags: postgresql - :tickets: - - using new dialect.initialize() feature to set up - version-dependent behavior. - - .. change:: - :tags: postgresql - :tickets: 1279 - - somewhat better support for % signs in table/column names; - psycopg2 can't handle a bind parameter name of - %(foobar)s however and SQLA doesn't want to add overhead - just to treat that one non-existent use case. - - .. change:: - :tags: postgresql - :tickets: 1516 - - Inserting NULL into a primary key + foreign key column - will allow the "not null constraint" error to raise, - not an attempt to execute a nonexistent "col_id_seq" - sequence. - - .. change:: - :tags: postgresql - :tickets: - - autoincrement SELECT statements, i.e. those which - select from a procedure that modifies rows, now work - with server-side cursor mode (the named cursor isn't - used for such statements.) - - .. change:: - :tags: postgresql - :tickets: 1636 - - postgresql dialect can properly detect pg "devel" version - strings, i.e. "8.5devel" - - .. change:: - :tags: postgresql - :tickets: 1619 - - The psycopg2 now respects the statement option - "stream_results". This option overrides the connection setting - "server_side_cursors". If true, server side cursors will be - used for the statement. If false, they will not be used, even - if "server_side_cursors" is true on the - connection. - - .. change:: - :tags: mysql - :tickets: - - New dialects: oursql, a new native dialect, - MySQL Connector/Python, a native Python port of MySQLdb, - and of course zxjdbc on Jython. - - .. change:: - :tags: mysql - :tickets: - - VARCHAR/NVARCHAR will not render without a length, raises - an error before passing to MySQL. Doesn't impact - CAST since VARCHAR is not allowed in MySQL CAST anyway, - the dialect renders CHAR/NCHAR in those cases. - - .. change:: - :tags: mysql - :tickets: - - all the _detect_XXX() functions now run once underneath - dialect.initialize() - - .. change:: - :tags: mysql - :tickets: 1279 - - somewhat better support for % signs in table/column names; - MySQLdb can't handle % signs in SQL when executemany() is used, - and SQLA doesn't want to add overhead just to treat that one - non-existent use case. - - .. change:: - :tags: mysql - :tickets: - - the BINARY and MSBinary types now generate "BINARY" in all - cases. Omitting the "length" parameter will generate - "BINARY" with no length. Use BLOB to generate an unlengthed - binary column. - - .. change:: - :tags: mysql - :tickets: - - the "quoting='quoted'" argument to MSEnum/ENUM is deprecated. - It's best to rely upon the automatic quoting. - - .. change:: - :tags: mysql - :tickets: - - ENUM now subclasses the new generic Enum type, and also handles - unicode values implicitly, if the given labelnames are unicode - objects. - - .. change:: - :tags: mysql - :tickets: 1539 - - a column of type TIMESTAMP now defaults to NULL if - "nullable=False" is not passed to Column(), and no default - is present. This is now consistent with all other types, - and in the case of TIMESTAMP explicitly renders "NULL" - due to MySQL's "switching" of default nullability - for TIMESTAMP columns. - - .. change:: - :tags: oracle - :tickets: - - unit tests pass 100% with cx_oracle ! - - .. change:: - :tags: oracle - :tickets: - - support for cx_Oracle's "native unicode" mode which does - not require NLS_LANG to be set. Use the latest 5.0.2 or - later of cx_oracle. - - .. change:: - :tags: oracle - :tickets: - - an NCLOB type is added to the base types. - - .. change:: - :tags: oracle - :tickets: - - use_ansi=False won't leak into the FROM/WHERE clause of - a statement that's selecting from a subquery that also - uses JOIN/OUTERJOIN. - - .. change:: - :tags: oracle - :tickets: 1467 - - added native INTERVAL type to the dialect. This supports - only the DAY TO SECOND interval type so far due to lack - of support in cx_oracle for YEAR TO MONTH. - - .. change:: - :tags: oracle - :tickets: - - usage of the CHAR type results in cx_oracle's - FIXED_CHAR dbapi type being bound to statements. - - .. change:: - :tags: oracle - :tickets: 885 - - the Oracle dialect now features NUMBER which intends - to act justlike Oracle's NUMBER type. It is the primary - numeric type returned by table reflection and attempts - to return Decimal()/float/int based on the precision/scale - parameters. - - .. change:: - :tags: oracle - :tickets: - - func.char_length is a generic function for LENGTH - - .. change:: - :tags: oracle - :tickets: - - ForeignKey() which includes onupdate= will emit a - warning, not emit ON UPDATE CASCADE which is unsupported - by oracle - - .. change:: - :tags: oracle - :tickets: - - the keys() method of RowProxy() now returns the result - column names *normalized* to be SQLAlchemy case - insensitive names. This means they will be lower case for - case insensitive names, whereas the DBAPI would normally - return them as UPPERCASE names. This allows row keys() to - be compatible with further SQLAlchemy operations. - - .. change:: - :tags: oracle - :tickets: - - using new dialect.initialize() feature to set up - version-dependent behavior. - - .. change:: - :tags: oracle - :tickets: 1125 - - using types.BigInteger with Oracle will generate - NUMBER(19) - - .. change:: - :tags: oracle - :tickets: - - "case sensitivity" feature will detect an all-lowercase - case-sensitive column name during reflect and add - "quote=True" to the generated Column, so that proper - quoting is maintained. - - .. change:: - :tags: firebird - :tickets: - - the keys() method of RowProxy() now returns the result - column names *normalized* to be SQLAlchemy case - insensitive names. This means they will be lower case for - case insensitive names, whereas the DBAPI would normally - return them as UPPERCASE names. This allows row keys() to - be compatible with further SQLAlchemy operations. - - .. change:: - :tags: firebird - :tickets: - - using new dialect.initialize() feature to set up - version-dependent behavior. - - .. change:: - :tags: firebird - :tickets: - - "case sensitivity" feature will detect an all-lowercase - case-sensitive column name during reflect and add - "quote=True" to the generated Column, so that proper - quoting is maintained. - - .. change:: - :tags: mssql - :tickets: - - MSSQL + Pyodbc + FreeTDS now works for the most part, - with possible exceptions regarding binary data as well as - unicode schema identifiers. - - .. change:: - :tags: mssql - :tickets: - - the "has_window_funcs" flag is removed. LIMIT/OFFSET - usage will use ROW NUMBER as always, and if on an older - version of SQL Server, the operation fails. The behavior - is exactly the same except the error is raised by SQL - server instead of the dialect, and no flag setting is - required to enable it. - - .. change:: - :tags: mssql - :tickets: - - the "auto_identity_insert" flag is removed. This feature - always takes effect when an INSERT statement overrides a - column that is known to have a sequence on it. As with - "has_window_funcs", if the underlying driver doesn't - support this, then you can't do this operation in any - case, so there's no point in having a flag. - - .. change:: - :tags: mssql - :tickets: - - using new dialect.initialize() feature to set up - version-dependent behavior. - - .. change:: - :tags: mssql - :tickets: - - removed references to sequence which is no longer used. - implicit identities in mssql work the same as implicit - sequences on any other dialects. Explicit sequences are - enabled through the use of "default=Sequence()". See - the MSSQL dialect documentation for more information. - - .. change:: - :tags: sqlite - :tickets: - - DATE, TIME and DATETIME types can now take optional storage_format - and regexp argument. storage_format can be used to store those types - using a custom string format. regexp allows to use a custom regular - expression to match string values from the database. - - .. change:: - :tags: sqlite - :tickets: - - Time and DateTime types now use by a default a stricter regular - expression to match strings from the database. Use the regexp - argument if you are using data stored in a legacy format. - - .. change:: - :tags: sqlite - :tickets: - - __legacy_microseconds__ on SQLite Time and DateTime types is not - supported anymore. You should use the storage_format argument - instead. - - .. change:: - :tags: sqlite - :tickets: - - Date, Time and DateTime types are now stricter in what they accept as - bind parameters: Date type only accepts date objects (and datetime - ones, because they inherit from date), Time only accepts time - objects, and DateTime only accepts date and datetime objects. - - .. change:: - :tags: sqlite - :tickets: 1016 - - Table() supports a keyword argument "sqlite_autoincrement", which - applies the SQLite keyword "AUTOINCREMENT" to the single integer - primary key column when generating DDL. Will prevent generation of - a separate PRIMARY KEY constraint. - - .. change:: - :tags: types - :tickets: - - The construction of types within dialects has been totally - overhauled. Dialects now define publicly available types - as UPPERCASE names exclusively, and internal implementation - types using underscore identifiers (i.e. are private). - The system by which types are expressed in SQL and DDL - has been moved to the compiler system. This has the - effect that there are much fewer type objects within - most dialects. A detailed document on this architecture - for dialect authors is in - lib/sqlalchemy/dialects/type_migration_guidelines.txt . - - .. change:: - :tags: types - :tickets: - - Types no longer make any guesses as to default - parameters. In particular, Numeric, Float, NUMERIC, - FLOAT, DECIMAL don't generate any length or scale unless - specified. - - .. change:: - :tags: types - :tickets: 1664 - - types.Binary is renamed to types.LargeBinary, it only - produces BLOB, BYTEA, or a similar "long binary" type. - New base BINARY and VARBINARY - types have been added to access these MySQL/MS-SQL specific - types in an agnostic way. - - .. change:: - :tags: types - :tickets: - - String/Text/Unicode types now skip the unicode() check - on each result column value if the dialect has - detected the DBAPI as returning Python unicode objects - natively. This check is issued on first connect - using "SELECT CAST 'some text' AS VARCHAR(10)" or - equivalent, then checking if the returned object - is a Python unicode. This allows vast performance - increases for native-unicode DBAPIs, including - pysqlite/sqlite3, psycopg2, and pg8000. - - .. change:: - :tags: types - :tickets: - - Most types result processors have been checked for possible speed - improvements. Specifically, the following generic types have been - optimized, resulting in varying speed improvements: - Unicode, PickleType, Interval, TypeDecorator, Binary. - Also the following dbapi-specific implementations have been improved: - Time, Date and DateTime on Sqlite, ARRAY on PostgreSQL, - Time on MySQL, Numeric(as_decimal=False) on MySQL, oursql and - pypostgresql, DateTime on cx_oracle and LOB-based types on cx_oracle. - - .. change:: - :tags: types - :tickets: - - Reflection of types now returns the exact UPPERCASE - type within types.py, or the UPPERCASE type within - the dialect itself if the type is not a standard SQL - type. This means reflection now returns more accurate - information about reflected types. - - .. change:: - :tags: types - :tickets: 1511, 1109 - - Added a new Enum generic type. Enum is a schema-aware object - to support databases which require specific DDL in order to - use enum or equivalent; in the case of PG it handles the - details of `CREATE TYPE`, and on other databases without - native enum support will by generate VARCHAR + an inline CHECK - constraint to enforce the enum. - - .. change:: - :tags: types - :tickets: 1467 - - The Interval type includes a "native" flag which controls - if native INTERVAL types (postgresql + oracle) are selected - if available, or not. "day_precision" and "second_precision" - arguments are also added which propagate as appropriately - to these native types. Related to. - - .. change:: - :tags: types - :tickets: 1589 - - The Boolean type, when used on a backend that doesn't - have native boolean support, will generate a CHECK - constraint "col IN (0, 1)" along with the int/smallint- - based column type. This can be switched off if - desired with create_constraint=False. - Note that MySQL has no native boolean *or* CHECK constraint - support so this feature isn't available on that platform. - - .. change:: - :tags: types - :tickets: - - PickleType now uses == for comparison of values when - mutable=True, unless the "comparator" argument with a - comparison function is specified to the type. Objects - being pickled will be compared based on identity (which - defeats the purpose of mutable=True) if __eq__() is not - overridden or a comparison function is not provided. - - .. change:: - :tags: types - :tickets: - - The default "precision" and "scale" arguments of Numeric - and Float have been removed and now default to None. - NUMERIC and FLOAT will be rendered with no numeric - arguments by default unless these values are provided. - - .. change:: - :tags: types - :tickets: - - AbstractType.get_search_list() is removed - the games - that was used for are no longer necessary. - - .. change:: - :tags: types - :tickets: 1125 - - Added a generic BigInteger type, compiles to - BIGINT or NUMBER(19). - - .. change:: - :tags: types - :tickets: - - sqlsoup has been overhauled to explicitly support an 0.5 style - session, using autocommit=False, autoflush=True. Default - behavior of SQLSoup now requires the usual usage of commit() - and rollback(), which have been added to its interface. An - explicit Session or scoped_session can be passed to the - constructor, allowing these arguments to be overridden. - - .. change:: - :tags: types - :tickets: - - sqlsoup db..update() and delete() now call - query(cls).update() and delete(), respectively. - - .. change:: - :tags: types - :tickets: - - sqlsoup now has execute() and connection(), which call upon - the Session methods of those names, ensuring that the bind is - in terms of the SqlSoup object's bind. - - .. change:: - :tags: types - :tickets: - - sqlsoup objects no longer have the 'query' attribute - it's - not needed for sqlsoup's usage paradigm and it gets in the - way of a column that is actually named 'query'. - - .. change:: - :tags: types - :tickets: 1259 - - The signature of the proxy_factory callable passed to - association_proxy is now (lazy_collection, creator, - value_attr, association_proxy), adding a fourth argument - that is the parent AssociationProxy argument. Allows - serializability and subclassing of the built in collections. - - .. change:: - :tags: types - :tickets: 1372 - - association_proxy now has basic comparator methods .any(), - .has(), .contains(), ==, !=, thanks to Scott Torborg. diff --git a/doc/build/changelog/changelog_07.rst b/doc/build/changelog/changelog_07.rst deleted file mode 100644 index 300985f0215..00000000000 --- a/doc/build/changelog/changelog_07.rst +++ /dev/null @@ -1,4699 +0,0 @@ -============= -0.7 Changelog -============= - - -.. changelog:: - :version: 0.7.11 - :released: - - .. change:: - :tags: bug, engine - :tickets: 2851 - :versions: 0.8.3, 0.9.0b1 - - The regexp used by the :func:`~sqlalchemy.engine.url.make_url` function now parses - ipv6 addresses, e.g. surrounded by brackets. - - .. change:: - :tags: bug, orm - :tickets: 2807 - :versions: 0.8.3, 0.9.0b1 - - Fixed bug where list instrumentation would fail to represent a - setslice of ``[0:0]`` correctly, which in particular could occur - when using ``insert(0, item)`` with the association proxy. Due - to some quirk in Python collections, the issue was much more likely - with Python 3 rather than 2. - - .. change:: - :tags: bug, sql - :tickets: 2801 - :versions: 0.8.3, 0.9.0b1 - - Fixed regression dating back to 0.7.9 whereby the name of a CTE might - not be properly quoted if it was referred to in multiple FROM clauses. - - .. change:: - :tags: mysql, bug - :tickets: 2791 - :versions: 0.8.3, 0.9.0b1 - - Updates to MySQL reserved words for versions 5.5, 5.6, courtesy - Hanno Schlichting. - - .. change:: - :tags: sql, bug, cte - :tickets: 2783 - :versions: 0.8.3, 0.9.0b1 - - Fixed bug in common table expression system where if the CTE were - used only as an ``alias()`` construct, it would not render using the - WITH keyword. - - .. change:: - :tags: bug, sql - :tickets: 2784 - :versions: 0.8.3, 0.9.0b1 - - Fixed bug in :class:`.CheckConstraint` DDL where the "quote" flag from a - :class:`_schema.Column` object would not be propagated. - - .. change:: - :tags: bug, orm - :tickets: 2699 - :versions: 0.8.1 - - Fixed bug when a query of the form: - ``query(SubClass).options(subqueryload(Baseclass.attrname))``, - where ``SubClass`` is a joined inh of ``BaseClass``, - would fail to apply the ``JOIN`` inside the subquery - on the attribute load, producing a cartesian product. - The populated results still tended to be correct as additional - rows are just ignored, so this issue may be present as a - performance degradation in applications that are - otherwise working correctly. - - .. change:: - :tags: bug, orm - :tickets: 2689 - :versions: 0.8.1 - - Fixed bug in unit of work whereby a joined-inheritance - subclass could insert the row for the "sub" table - before the parent table, if the two tables had no - ForeignKey constraints set up between them. - - .. change:: - :tags: feature, postgresql - :tickets: 2676 - :versions: 0.8.0 - - Added support for PostgreSQL's traditional SUBSTRING - function syntax, renders as "SUBSTRING(x FROM y FOR z)" - when regular ``func.substring()`` is used. - Courtesy Gunnlaugur Þór Briem. - - .. change:: - :tags: bug, tests - :tickets: 2669 - :pullreq: 41 - - Fixed an import of "logging" in test_execute which was not - working on some linux platforms. - - .. change:: - :tags: bug, orm - :tickets: 2674 - - Improved the error message emitted when a "backref loop" is detected, - that is when an attribute event triggers a bidirectional - assignment between two other attributes with no end. - This condition can occur not just when an object of the wrong - type is assigned, but also when an attribute is mis-configured - to backref into an existing backref pair. - - .. change:: - :tags: bug, orm - :tickets: 2674 - - A warning is emitted when a MapperProperty is assigned to a mapper - that replaces an existing property, if the properties in question - aren't plain column-based properties. Replacement of relationship - properties is rarely (ever?) what is intended and usually refers to a - mapper mis-configuration. This will also warn if a backref configures - itself on top of an existing one in an inheritance relationship - (which is an error in 0.8). - -.. changelog:: - :version: 0.7.10 - :released: Thu Feb 7 2013 - - .. change:: - :tags: engine, bug - :tickets: 2604 - :versions: 0.8.0b2 - - Fixed :meth:`_schema.MetaData.reflect` to correctly use - the given :class:`_engine.Connection`, if given, without - opening a second connection from that connection's - :class:`_engine.Engine`. - - .. change:: - :tags: mssql, bug - :tickets:2607 - :versions: 0.8.0b2 - - Fixed bug whereby using "key" with Column - in conjunction with "schema" for the owning - Table would fail to locate result rows due - to the MSSQL dialect's "schema rendering" - logic's failure to take .key into account. - - .. change:: - :tags: sql, mysql, gae - :tickets: 2649 - - Added a conditional import to the ``gaerdbms`` dialect which attempts - to import rdbms_apiproxy vs. rdbms_googleapi to work - on both dev and production platforms. Also now honors the - ``instance`` attribute. Courtesy Sean Lynch. Also backported - enhancements to allow username/password as well as - fixing error code interpretation from 0.8. - - .. change:: - :tags: sql, bug - :tickets: 2594, 2584 - - Backported adjustment to ``__repr__`` for - :class:`.TypeDecorator` to 0.7, allows :class:`.PickleType` - to produce a clean ``repr()`` to help with Alembic. - - .. change:: - :tags: sql, bug - :tickets: 2643 - - Fixed bug where :meth:`_schema.Table.tometadata` would fail if a - :class:`_schema.Column` had both a foreign key as well as an - alternate ".key" name for the column. - - .. change:: - :tags: mssql, bug - :tickets: 2638 - - Added a Py3K conditional around unnecessary .decode() - call in mssql information schema, fixes reflection - in Py3k. - - .. change:: - :tags: orm, bug - :tickets: 2650 - - Fixed potential memory leak which could occur if an - arbitrary number of :class:`.sessionmaker` objects - were created. The anonymous subclass created by - the sessionmaker, when dereferenced, would not be garbage - collected due to remaining class-level references from the - event package. This issue also applies to any custom system - that made use of ad-hoc subclasses in conjunction with - an event dispatcher. - - .. change:: - :tags: orm, bug - :tickets: 2640 - - :meth:`_query.Query.merge_result` can now load rows from an outer join - where an entity may be ``None`` without throwing an error. - - .. change:: - :tags: sqlite, bug - :tickets: 2568 - :versions: 0.8.0b2 - - More adjustment to this SQLite related issue which was released in - 0.7.9, to intercept legacy SQLite quoting characters when reflecting - foreign keys. In addition to intercepting double quotes, other - quoting characters such as brackets, backticks, and single quotes - are now also intercepted. - - .. change:: - :tags: sql, bug - :tickets: 2631 - :versions: 0.8.0b2 - - Fixed bug where using server_onupdate= - without passing the "for_update=True" flag would apply the default - object to the server_default, blowing away whatever was there. - The explicit for_update=True argument shouldn't be needed with this usage - (especially since the documentation shows an example without it being - used) so it is now arranged internally using a copy of the given default - object, if the flag isn't set to what corresponds to that argument. - - .. change:: - :tags: oracle, bug - :tickets: 2620 - - The Oracle LONG type, while an unbounded text type, does not appear - to use the cx_Oracle.LOB type when result rows are returned, - so the dialect has been repaired to exclude LONG from - having cx_Oracle.LOB filtering applied. - - .. change:: - :tags: oracle, bug - :tickets: 2611 - - Repaired the usage of ``.prepare()`` in conjunction with - cx_Oracle so that a return value of ``False`` will result - in no call to ``connection.commit()``, hence avoiding - "no transaction" errors. Two-phase transactions have - now been shown to work in a rudimental fashion with - SQLAlchemy and cx_oracle, however are subject to caveats - observed with the driver; check the documentation - for details. - - .. change:: - :tags: orm, bug - :tickets: 2624 - - The :class:`.MutableComposite` type did not allow for the - :meth:`.MutableBase.coerce` method to be used, even though - the code seemed to indicate this intent, so this now works - and a brief example is added. As a side-effect, - the mechanics of this event handler have been changed so that - new :class:`.MutableComposite` types no longer add per-type - global event handlers. Also in 0.8.0b2. - - .. change:: - :tags: orm, bug - :tickets: 2583 - - Fixed Session accounting bug whereby replacing - a deleted object in the identity map with another - object of the same primary key would raise a - "conflicting state" error on rollback(), - if the replaced primary key were established either - via non-unitofwork-established INSERT statement - or by primary key switch of another instance. - - .. change:: - :tags: oracle, bug - :tickets: 2561 - - changed the list of cx_oracle types that are - excluded from the setinputsizes() step to only include - STRING and UNICODE; CLOB and NCLOB are removed. This - is to work around cx_oracle behavior which is broken - for the executemany() call. In 0.8, this same change - is applied however it is also configurable via the - exclude_setinputsizes argument. - - .. change:: - :tags: feature, mysql - :tickets: 2523 - - Added "raise_on_warnings" flag to OurSQL - dialect. - - .. change:: - :tags: feature, mysql - :tickets: 2554 - - Added "read_timeout" flag to MySQLdb - dialect. - -.. changelog:: - :version: 0.7.9 - :released: Mon Oct 01 2012 - - .. change:: - :tags: orm, bug - :tickets: - - Fixed bug mostly local to new - AbstractConcreteBase helper where the "type" - attribute from the superclass would not - be overridden on the subclass to produce the - "reserved for base" error message, instead placing - a do-nothing attribute there. This was inconsistent - vs. using ConcreteBase as well as all the behavior - of classical concrete mappings, where the "type" - column from the polymorphic base would be explicitly - disabled on subclasses, unless overridden - explicitly. - - .. change:: - :tags: orm, bug - :tickets: - - A warning is emitted when lazy='dynamic' - is combined with uselist=False. This is an - exception raise in 0.8. - - .. change:: - :tags: orm, bug - :tickets: - - Fixed bug whereby user error in related-object - assignment could cause recursion overflow if the - assignment triggered a backref of the same name - as a bi-directional attribute on the incorrect - class to the same target. An informative - error is raised now. - - .. change:: - :tags: orm, bug - :tickets: 2539 - - Fixed bug where incorrect type information - would be passed when the ORM would bind the - "version" column, when using the "version" feature. - Tests courtesy Daniel Miller. - - .. change:: - :tags: orm, bug - :tickets: 2566 - - Extra logic has been added to the "flush" - that occurs within Session.commit(), such that the - extra state added by an after_flush() or - after_flush_postexec() hook is also flushed in a - subsequent flush, before the "commit" completes. - Subsequent calls to flush() will continue until - the after_flush hooks stop adding new state. - An "overflow" counter of 100 is also in place, - in the event of a broken after_flush() hook - adding new content each time. - - .. change:: - :tags: bug, sql - :tickets: 2571 - - Fixed the DropIndex construct to support - an Index associated with a Table in a remote - schema. - - .. change:: - :tags: bug, sql - :tickets: 2574 - - Fixed bug in over() construct whereby - passing an empty list for either partition_by - or order_by, as opposed to None, would fail - to generate correctly. - Courtesy Gunnlaugur Þór Briem. - - .. change:: - :tags: bug, sql - :tickets: 2521 - - Fixed CTE bug whereby positional - bound parameters present in the CTEs themselves - would corrupt the overall ordering of - bound parameters. This primarily - affected SQL Server as the platform with - positional binds + CTE support. - - .. change:: - :tags: bug, sql - :tickets: - - Fixed more un-intuitivenesses in CTEs - which prevented referring to a CTE in a union - of itself without it being aliased. - CTEs now render uniquely - on name, rendering the outermost CTE of a given - name only - all other references are rendered - just as the name. This even includes other - CTE/SELECTs that refer to different versions - of the same CTE object, such as a SELECT - or a UNION ALL of that SELECT. We are - somewhat loosening the usual link between object - identity and lexical identity in this case. - A true name conflict between two unrelated - CTEs now raises an error. - - .. change:: - :tags: bug, sql - :tickets: 2512 - - quoting is applied to the column names - inside the WITH RECURSIVE clause of a - common table expression according to the - quoting rules for the originating Column. - - .. change:: - :tags: bug, sql - :tickets: 2518 - - Fixed regression introduced in 0.7.6 - whereby the FROM list of a SELECT statement - could be incorrect in certain "clone+replace" - scenarios. - - .. change:: - :tags: bug, sql - :tickets: 2552 - - Fixed bug whereby usage of a UNION - or similar inside of an embedded subquery - would interfere with result-column targeting, - in the case that a result-column had the same - ultimate name as a name inside the embedded - UNION. - - .. change:: - :tags: bug, sql - :tickets: 2558 - - Fixed a regression since 0.6 regarding - result-row targeting. It should be possible - to use a select() statement with string - based columns in it, that is - select(['id', 'name']).select_from('mytable'), - and have this statement be targetable by - Column objects with those names; this is the - mechanism by which - query(MyClass).from_statement(some_statement) - works. At some point the specific case of - using select(['id']), which is equivalent to - select([literal_column('id')]), stopped working - here, so this has been re-instated and of - course tested. - - .. change:: - :tags: bug, sql - :tickets: 2544 - - Added missing operators is_(), isnot() - to the ColumnOperators base, so that these long-available - operators are present as methods like all - the other operators. - - .. change:: - :tags: engine, bug - :tickets: 2522 - - Fixed bug whereby - a disconnect detect + dispose that occurs - when the QueuePool has threads waiting - for connections would leave those - threads waiting for the duration of - the timeout on the old pool (or indefinitely - if timeout was disabled). The fix - now notifies those waiters with a special - exception case and has them move onto - the new pool. - - .. change:: - :tags: engine, feature - :tickets: 2516 - - Dramatic improvement in memory - usage of the event system; instance-level - collections are no longer created for a - particular type of event until - instance-level listeners are established - for that event. - - .. change:: - :tags: engine, bug - :tickets: 2529 - - Added gaerdbms import to mysql/__init__.py, - the absence of which was preventing the new - GAE dialect from being loaded. - - .. change:: - :tags: engine, bug - :tickets: 2553 - - Fixed cextension bug whereby the - "ambiguous column error" would fail to - function properly if the given index were - a Column object and not a string. - Note there are still some column-targeting - issues here which are fixed in 0.8. - - .. change:: - :tags: engine, bug - :tickets: - - Fixed the repr() of Enum to include - the "name" and "native_enum" flags. Helps - Alembic autogenerate. - - .. change:: - :tags: sqlite, bug - :tickets: 2568 - - Adjusted a very old bugfix which attempted - to work around a SQLite issue that itself was - "fixed" as of sqlite 3.6.14, regarding quotes - surrounding a table name when using - the "foreign_key_list" pragma. The fix has been - adjusted to not interfere with quotes that - are *actually in the name* of a column or table, - to as much a degree as possible; sqlite still - doesn't return the correct result for foreign_key_list() - if the target table actually has quotes surrounding - its name, as *part* of its name (i.e. """mytable"""). - - .. change:: - :tags: sqlite, bug - :tickets: 2265 - - Adjusted column default reflection code to - convert non-string values to string, to accommodate - old SQLite versions that don't deliver - default info as a string. - - .. change:: - :tags: sqlite, feature - :tickets: - - Added support for the localtimestamp() - SQL function implemented in SQLite, courtesy - Richard Mitchell. - - .. change:: - :tags: postgresql, bug - :tickets: 2531 - - Columns in reflected primary key constraint - are now returned in the order in which the constraint - itself defines them, rather than how the table - orders them. Courtesy Gunnlaugur Þór Briem.. - - .. change:: - :tags: postgresql, bug - :tickets: 2570 - - Added 'terminating connection' to the list - of messages we use to detect a disconnect with PG, which - appears to be present in some versions when the server - is restarted. - - .. change:: - :tags: bug, mysql - :tickets: - - Updated mysqlconnector interface to use - updated "client flag" and "charset" APIs, - courtesy David McNelis. - - .. change:: - :tags: mssql, bug - :tickets: 2538 - - Fixed compiler bug whereby using a correlated - subquery within an ORDER BY would fail to render correctly - if the statement also used LIMIT/OFFSET, due to mis-rendering - within the ROW_NUMBER() OVER clause. Fix courtesy - sayap - - .. change:: - :tags: mssql, bug - :tickets: 2545 - - Fixed compiler bug whereby a given - select() would be modified if it had an "offset" - attribute, causing the construct to not compile - correctly a second time. - - .. change:: - :tags: mssql, bug - :tickets: - - Fixed bug where reflection of primary key constraint - would double up columns if the same constraint/table - existed in multiple schemas. - -.. changelog:: - :version: 0.7.8 - :released: Sat Jun 16 2012 - - .. change:: - :tags: orm, bug - :tickets: 2480 - - Fixed bug whereby subqueryload() from - a polymorphic mapping to a target would incur - a new invocation of the query for each - distinct class encountered in the polymorphic - result. - - .. change:: - :tags: orm, bug - :tickets: 2491, 1892 - - Fixed bug in declarative - whereby the precedence of columns - in a joined-table, composite - column (typically for id) would fail to - be correct if the columns contained - names distinct from their attribute - names. This would cause things like - primaryjoin conditions made against the - entity attributes to be incorrect. Related - to as this was supposed - to be part of that, this is. - - .. change:: - :tags: orm, feature - :tickets: - - The 'objects' argument to - flush() is no longer deprecated, as some - valid use cases have been identified. - - .. change:: - :tags: orm, bug - :tickets: 2508 - - Fixed identity_key() function which - was not accepting a scalar argument - for the identity. . - - .. change:: - :tags: orm, bug - :tickets: 2497 - - Fixed bug whereby populate_existing - option would not propagate to subquery - eager loaders. . - - .. change:: - :tags: bug, sql - :tickets: 2499 - - added BIGINT to types.__all__, - BIGINT, BINARY, VARBINARY to sqlalchemy - module namespace, plus test to ensure - this breakage doesn't occur again. - - .. change:: - :tags: bug, sql - :tickets: 2490 - - Repaired common table expression - rendering to function correctly when the - SELECT statement contains UNION or other - compound expressions, courtesy btbuilder. - - .. change:: - :tags: bug, sql - :tickets: 2482 - - Fixed bug whereby append_column() - wouldn't function correctly on a cloned - select() construct, courtesy - Gunnlaugur Þór Briem. - - .. change:: - :tags: engine, bug - :tickets: 2489 - - Fixed memory leak in C version of - result proxy whereby DBAPIs which don't deliver - pure Python tuples for result rows would - fail to decrement refcounts correctly. - The most prominently affected DBAPI - is pyodbc. - - .. change:: - :tags: engine, bug - :tickets: 2503 - - Fixed bug affecting Py3K whereby - string positional parameters passed to - engine/connection execute() would fail to be - interpreted correctly, due to __iter__ - being present on Py3K string.. - - .. change:: - :tags: postgresql, bug - :tickets: 2510 - - removed unnecessary table clause when - reflecting enums,. Courtesy - Gunnlaugur Þór Briem. - - .. change:: - :tags: oracle, bug - :tickets: 2483 - - Added ROWID to oracle.*. - - .. change:: - :tags: feature, mysql - :tickets: 2484 - - Added a new dialect for Google App - Engine. Courtesy Richie Foreman. - -.. changelog:: - :version: 0.7.7 - :released: Sat May 05 2012 - - .. change:: - :tags: orm, bug - :tickets: 2477 - - Fixed issue in unit of work - whereby setting a non-None self-referential - many-to-one relationship to None - would fail to persist the change if the - former value was not already loaded.. - - .. change:: - :tags: orm, feature - :tickets: 2443 - - Added prefix_with() method - to Query, calls upon select().prefix_with() - to allow placement of MySQL SELECT - directives in statements. Courtesy - Diana Clarke - - .. change:: - :tags: orm, bug - :tickets: 2409 - - Fixed bug in 0.7.6 introduced by whereby column_mapped_collection - used against columns that were mapped as - joins or other indirect selectables - would fail to function. - - .. change:: - :tags: orm, feature - :tickets: - - Added new flag to @validates - include_removes. When True, collection - remove and attribute del events - will also be sent to the validation function, - which accepts an additional argument - "is_remove" when this flag is used. - - .. change:: - :tags: orm, bug - :tickets: 2449 - - Fixed bug whereby polymorphic_on - column that's not otherwise mapped on the - class would be incorrectly included - in a merge() operation, raising an error. - - .. change:: - :tags: orm, bug - :tickets: 2453 - - Fixed bug in expression annotation - mechanics which could lead to incorrect - rendering of SELECT statements with aliases - and joins, particularly when using - column_property(). - - .. change:: - :tags: orm, bug - :tickets: 2454 - - Fixed bug which would prevent - OrderingList from being pickleable. Courtesy Jeff Dairiki - - .. change:: - :tags: orm, bug - :tickets: - - Fixed bug in relationship comparisons - whereby calling unimplemented methods like - SomeClass.somerelationship.like() would - produce a recursion overflow, instead - of NotImplementedError. - - .. change:: - :tags: bug, sql - :tickets: - - Removed warning when Index is created - with no columns; while this might not be what - the user intended, it is a valid use case - as an Index could be a placeholder for just an - index of a certain name. - - .. change:: - :tags: feature, sql - :tickets: - - Added new connection event - dbapi_error(). Is called for all DBAPI-level - errors passing the original DBAPI exception - before SQLAlchemy modifies the state - of the cursor. - - .. change:: - :tags: bug, sql - :tickets: - - If conn.begin() fails when calling - "with engine.begin()", the newly acquired - Connection is closed explicitly before - propagating the exception onward normally. - - .. change:: - :tags: bug, sql - :tickets: 2474 - - Add BINARY, VARBINARY to types.__all__. - - .. change:: - :tags: mssql, feature - :tickets: - - Added interim create_engine flag - supports_unicode_binds to PyODBC dialect, - to force whether or not the dialect - passes Python unicode literals to PyODBC - or not. - - .. change:: - :tags: mssql, bug - :tickets: - - Repaired the use_scope_identity - create_engine() flag when using the pyodbc - dialect. Previously this flag would be - ignored if set to False. When set to False, - you'll get "SELECT @@identity" after each - INSERT to get at the last inserted ID, - for those tables which have "implicit_returning" - set to False. - - .. change:: - :tags: mssql, bug - :tickets: 2468 - - UPDATE..FROM syntax with SQL Server - requires that the updated table be present - in the FROM clause when an alias of that - table is also present in the FROM clause. - The updated table is now always present - in the FROM, when FROM is present - in the first place. Courtesy sayap. - - .. change:: - :tags: postgresql, feature - :tickets: 2445 - - Added new for_update/with_lockmode() - options for PostgreSQL: for_update="read"/ - with_lockmode("read"), - for_update="read_nowait"/ - with_lockmode("read_nowait"). - These emit "FOR SHARE" and "FOR SHARE NOWAIT", - respectively. Courtesy Diana Clarke - - .. change:: - :tags: postgresql, bug - :tickets: 2473 - - removed unnecessary table clause - when reflecting domains. - - .. change:: - :tags: bug, mysql - :tickets: 2460 - - Fixed bug whereby column name inside - of "KEY" clause for autoincrement composite - column with InnoDB would double quote a - name that's a reserved word. Courtesy Jeff - Dairiki. - - .. change:: - :tags: bug, mysql - :tickets: - - Fixed bug whereby get_view_names() for - "information_schema" schema would fail - to retrieve views marked as "SYSTEM VIEW". - courtesy Matthew Turland. - - .. change:: - :tags: bug, mysql - :tickets: 2467 - - Fixed bug whereby if cast() is used - on a SQL expression whose type is not supported - by cast() and therefore CAST isn't rendered by - the dialect, the order of evaluation could change - if the casted expression required that it be - grouped; grouping is now applied to those - expressions. - - .. change:: - :tags: sqlite, feature - :tickets: 2475 - - Added SQLite execution option - "sqlite_raw_colnames=True", will bypass - attempts to remove "." from column names - returned by SQLite cursor.description. - - .. change:: - :tags: sqlite, bug - :tickets: 2525 - - When the primary key column of a Table - is replaced, such as via extend_existing, - the "auto increment" column used by insert() - constructs is reset. Previously it would - remain referring to the previous primary - key column. - -.. changelog:: - :version: 0.7.6 - :released: Wed Mar 14 2012 - - .. change:: - :tags: orm, bug - :tickets: 2424 - - Fixed event registration bug - which would primarily show up as - events not being registered with - sessionmaker() instances created - after the event was associated - with the Session class. - - .. change:: - :tags: orm, bug - :tickets: 2425 - - Fixed bug whereby a primaryjoin - condition with a "literal" in it would - raise an error on compile with certain - kinds of deeply nested expressions - which also needed to render the same - bound parameter name more than once. - - .. change:: - :tags: orm, feature - :tickets: - - Added "no_autoflush" context - manager to Session, used with with: - will temporarily disable autoflush. - - .. change:: - :tags: orm, feature - :tickets: 1859 - - Added cte() method to Query, - invokes common table expression support - from the Core (see below). - - .. change:: - :tags: orm, bug - :tickets: 2403 - - Removed the check for number of - rows affected when doing a multi-delete - against mapped objects. If an ON DELETE - CASCADE exists between two rows, we can't - get an accurate rowcount from the DBAPI; - this particular count is not supported - on most DBAPIs in any case, MySQLdb - is the notable case where it is. - - .. change:: - :tags: orm, bug - :tickets: 2409 - - Fixed bug whereby objects using - attribute_mapped_collection or - column_mapped_collection could not be - pickled. - - .. change:: - :tags: orm, bug - :tickets: 2406 - - Fixed bug whereby MappedCollection - would not get the appropriate collection - instrumentation if it were only used - in a custom subclass that used - @collection.internally_instrumented. - - .. change:: - :tags: orm, bug - :tickets: 2419 - - Fixed bug whereby SQL adaption mechanics - would fail in a very nested scenario involving - joined-inheritance, joinedload(), limit(), and a - derived function in the columns clause. - - .. change:: - :tags: orm, bug - :tickets: 2417 - - Fixed the repr() for CascadeOptions to - include refresh-expire. Also reworked - CascadeOptions to be a . - - .. change:: - :tags: orm, feature - :tickets: 2400 - - Added the ability to query for - Table-bound column names when using - query(sometable).filter_by(colname=value). - - .. change:: - :tags: orm, bug - :tickets: - - Improved the "declarative reflection" - example to support single-table inheritance, - multiple calls to prepare(), tables that - are present in alternate schemas, - establishing only a subset of classes - as reflected. - - .. change:: - :tags: orm, bug - :tickets: 2390 - - Scaled back the test applied within - flush() to check for UPDATE against partially - NULL PK within one table to only actually - happen if there's really an UPDATE to occur. - - .. change:: - :tags: orm, bug - :tickets: 2352 - - Fixed bug whereby if a method name - conflicted with a column name, a - TypeError would be raised when the mapper - tried to inspect the __get__() method - on the method object. - - .. change:: - :tags: bug, sql - :tickets: 2427 - - Fixed memory leak in core which would - occur when C extensions were used with - particular types of result fetches, - in particular when orm query.count() - were called. - - .. change:: - :tags: bug, sql - :tickets: 2398 - - Fixed issue whereby attribute-based - column access on a row would raise - AttributeError with non-C version, - NoSuchColumnError with C version. Now - raises AttributeError in both cases. - - .. change:: - :tags: feature, sql - :tickets: 1859 - - Added support for SQL standard - common table expressions (CTE), allowing - SELECT objects as the CTE source (DML - not yet supported). This is invoked via - the cte() method on any select() construct. - - .. change:: - :tags: bug, sql - :tickets: 2392 - - Added support for using the .key - of a Column as a string identifier in a - result set row. The .key is currently - listed as an "alternate" name for a column, - and is superseded by the name of a column - which has that key value as its regular name. - For the next major release - of SQLAlchemy we may reverse this precedence - so that .key takes precedence, but this - is not decided on yet. - - .. change:: - :tags: bug, sql - :tickets: 2413 - - A warning is emitted when a not-present - column is stated in the values() clause - of an insert() or update() construct. - Will move to an exception in 0.8. - - .. change:: - :tags: bug, sql - :tickets: 2396 - - A significant change to how labeling - is applied to columns in SELECT statements - allows "truncated" labels, that is label names - that are generated in Python which exceed - the maximum identifier length (note this is - configurable via label_length on create_engine()), - to be properly referenced when rendered inside - of a subquery, as well as to be present - in a result set row using their original - in-Python names. - - .. change:: - :tags: bug, sql - :tickets: 2402 - - Fixed bug in new "autoload_replace" flag - which would fail to preserve the primary - key constraint of the reflected table. - - .. change:: - :tags: bug, sql - :tickets: 2380 - - Index will raise when arguments passed - cannot be interpreted as columns or expressions. - Will warn when Index is created - with no columns at all. - - .. change:: - :tags: engine, feature - :tickets: 2407 - - Added "no_parameters=True" execution - option for connections. If no parameters - are present, will pass the statement - as cursor.execute(statement), thereby invoking - the DBAPIs behavior when no parameter collection - is present; for psycopg2 and mysql-python, this - means not interpreting % signs in the string. - This only occurs with this option, and not - just if the param list is blank, as otherwise - this would produce inconsistent behavior - of SQL expressions that normally escape percent - signs (and while compiling, can't know ahead of - time if parameters will be present in - some cases). - - .. change:: - :tags: engine, bug - :tickets: - - Added execution_options() call to - MockConnection (i.e., that used with - strategy="mock") which acts as a pass through - for arguments. - - .. change:: - :tags: engine, feature - :tickets: 2378 - - Added pool_reset_on_return argument - to create_engine, allows control over - "connection return" behavior. Also added - new arguments 'rollback', 'commit', None - to pool.reset_on_return to allow more control - over connection return activity. - - .. change:: - :tags: engine, feature - :tickets: - - Added some decent context managers - to Engine, Connection:: - - with engine.begin() as conn: - # - ... - - and:: - - with engine.connect() as conn: - # - ... - - Both close out the connection when done, - commit or rollback transaction with errors - on engine.begin(). - - .. change:: - :tags: sqlite, bug - :tickets: 2432 - - Fixed bug in C extensions whereby - string format would not be applied to a - Numeric value returned as integer; this - affected primarily SQLite which does - not maintain numeric scale settings. - - .. change:: - :tags: mssql, feature - :tickets: 2430 - - Added support for MSSQL INSERT, - UPDATE, and DELETE table hints, using - new with_hint() method on UpdateBase. - - .. change:: - :tags: feature, mysql - :tickets: 2386 - - Added support for MySQL index and - primary key constraint types - (i.e. USING) via new mysql_using parameter - to Index and PrimaryKeyConstraint, - courtesy Diana Clarke. - - .. change:: - :tags: feature, mysql - :tickets: 2394 - - Added support for the "isolation_level" - parameter to all MySQL dialects. Thanks - to mu_mind for the patch here. - - .. change:: - :tags: oracle, feature - :tickets: 2399 - - Added a new create_engine() flag - coerce_to_decimal=False, disables the precision - numeric handling which can add lots of overhead - by converting all numeric values to - Decimal. - - .. change:: - :tags: oracle, bug - :tickets: 2401 - - Added missing compilation support for - LONG - - .. change:: - :tags: oracle, bug - :tickets: 2435 - - Added 'LEVEL' to the list of reserved - words for Oracle. - - .. change:: - :tags: examples, bug - :tickets: - - Altered _params_from_query() function - in Beaker example to pull bindparams from the - fully compiled statement, as a quick means - to get everything including subqueries in the - columns clause, etc. - -.. changelog:: - :version: 0.7.5 - :released: Sat Jan 28 2012 - - .. change:: - :tags: orm, bug - :tickets: 2389 - - Fixed issue where modified session state - established after a failed flush would be committed - as part of the subsequent transaction that - begins automatically after manual call - to rollback(). The state of the session is - checked within rollback(), and if new state - is present, a warning is emitted and - restore_snapshot() is called a second time, - discarding those changes. - - .. change:: - :tags: orm, bug - :tickets: 2345 - - Fixed regression from 0.7.4 whereby - using an already instrumented column from a - superclass as "polymorphic_on" failed to resolve - the underlying Column. - - .. change:: - :tags: orm, bug - :tickets: 2370 - - Raise an exception if xyzload_all() is - used inappropriately with two non-connected - relationships. - - .. change:: - :tags: orm, feature - :tickets: - - Added "class_registry" argument to - declarative_base(). Allows two or more declarative - bases to share the same registry of class names. - - .. change:: - :tags: orm, feature - :tickets: - - query.filter() accepts multiple - criteria which will join via AND, i.e. - query.filter(x==y, z>q, ...) - - .. change:: - :tags: orm, feature - :tickets: 2351 - - Added new capability to relationship - loader options to allow "default" loader strategies. - Pass '*' to any of joinedload(), lazyload(), - subqueryload(), or noload() and that becomes the - loader strategy used for all relationships, - except for those explicitly stated in the - Query. Thanks to up-and-coming contributor - Kent Bower for an exhaustive and well - written test suite ! - - .. change:: - :tags: orm, bug - :tickets: 2367 - - Fixed bug whereby event.listen(SomeClass) - forced an entirely unnecessary compile of the - mapper, making events very hard to set up - at module import time (nobody noticed this ??) - - .. change:: - :tags: orm, bug - :tickets: - - Fixed bug whereby hybrid_property didn't - work as a kw arg in any(), has(). - - .. change:: - :tags: orm - :tickets: - - Fixed regression from 0.6 whereby if - "load_on_pending" relationship() flag were used - where a non-"get()" lazy clause needed to be - emitted on a pending object, it would fail - to load. - - .. change:: - :tags: orm, bug - :tickets: 2371 - - ensure pickleability of all ORM exceptions - for multiprocessing compatibility. - - .. change:: - :tags: orm, bug - :tickets: 2353 - - implemented standard "can't set attribute" / - "can't delete attribute" AttributeError when - setattr/delattr used on a hybrid that doesn't - define fset or fdel. - - .. change:: - :tags: orm, bug - :tickets: 2362 - - Fixed bug where unpickled object didn't - have enough of its state set up to work - correctly within the unpickle() event established - by the mutable object extension, if the object - needed ORM attribute access within - __eq__() or similar. - - .. change:: - :tags: orm, bug - :tickets: 2374 - - Fixed bug where "merge" cascade could - mis-interpret an unloaded attribute, if the - load_on_pending flag were used with - relationship(). Thanks to Kent Bower - for tests. - - .. change:: - :tags: orm, feature - :tickets: 2356 - - New declarative reflection example - added, illustrates how best to mix table reflection - with declarative as well as uses some new features - from. - - .. change:: - :tags: feature, sql - :tickets: 2356 - - New reflection feature "autoload_replace"; - when set to False on Table, the Table can be autoloaded - without existing columns being replaced. Allows - more flexible chains of Table construction/reflection - to be constructed, including that it helps with - combining Declarative with table reflection. - See the new example on the wiki. - - .. change:: - :tags: bug, sql - :tickets: 2356 - - Improved the API for add_column() such that - if the same column is added to its own table, - an error is not raised and the constraints - don't get doubled up. Also helps with some - reflection/declarative patterns. - - .. change:: - :tags: feature, sql - :tickets: - - Added "false()" and "true()" expression - constructs to sqlalchemy.sql namespace, though - not part of __all__ as of yet. - - .. change:: - :tags: feature, sql - :tickets: 2361 - - Dialect-specific compilers now raise - CompileError for all type/statement compilation - issues, instead of InvalidRequestError or ArgumentError. - The DDL for CREATE TABLE will re-raise - CompileError to include table/column information - for the problematic column. - - .. change:: - :tags: bug, sql - :tickets: 2381 - - Fixed issue where the "required" exception - would not be raised for bindparam() with required=True, - if the statement were given no parameters at all. - - .. change:: - :tags: engine, bug - :tickets: 2371 - - Added __reduce__ to StatementError, - DBAPIError, column errors so that exceptions - are pickleable, as when using multiprocessing. - However, not - all DBAPIs support this yet, such as - psycopg2. - - .. change:: - :tags: engine, bug - :tickets: 2382 - - Improved error messages when a non-string - or invalid string is passed to any of the - date/time processors used by SQLite, including - C and Python versions. - - .. change:: - :tags: engine, bug - :tickets: 2377 - - Fixed bug whereby a table-bound Column - object named "_" which matched a column - labeled as "_" could match - inappropriately when targeting in a result - set row. - - .. change:: - :tags: engine, bug - :tickets: 2384 - - Fixed bug in "mock" strategy whereby - correct DDL visit method wasn't called, resulting - in "CREATE/DROP SEQUENCE" statements being - duplicated - - .. change:: - :tags: sqlite, bug - :tickets: 2364 - - the "name" of an FK constraint in SQLite - is reflected as "None", not "0" or other - integer value. - SQLite does not appear to support constraint - naming in any case. - - .. change:: - :tags: sqlite, bug - :tickets: 2368 - - sql.false() and sql.true() compile to - 0 and 1, respectively in sqlite - - .. change:: - :tags: sqlite, bug - :tickets: - - removed an erroneous "raise" in the - SQLite dialect when getting table names - and view names, where logic is in place - to fall back to an older version of - SQLite that doesn't have the - "sqlite_temp_master" table. - - .. change:: - :tags: bug, mysql - :tickets: 2376 - - fixed regexp that filters out warnings - for non-reflected "PARTITION" directives, - thanks to George Reilly - - .. change:: - :tags: mssql, bug - :tickets: 2340 - - Adjusted the regexp used in the - mssql.TIME type to ensure only six digits - are received for the "microseconds" portion - of the value, which is expected by - Python's datetime.time(). Note that - support for sending microseconds doesn't - seem to be possible yet with pyodbc - at least. - - .. change:: - :tags: mssql, bug - :tickets: 2347 - - Dropped the "30 char" limit on pymssql, - based on reports that it's doing things - better these days. pymssql hasn't been - well tested and as the DBAPI is in flux - it's still not clear what the status - is on this driver and how SQLAlchemy's - implementation should adapt. - - .. change:: - :tags: oracle, bug - :tickets: 2388 - - Added ORA-03135 to the never ending - list of oracle "connection lost" errors - - .. change:: - :tags: core, bug - :tickets: 2379 - - Changed LRUCache, used by the mapper - to cache INSERT/UPDATE/DELETE statements, - to use an incrementing counter instead - of a timestamp to track entries, for greater - reliability versus using time.time(), which - can cause test failures on some platforms. - - .. change:: - :tags: core, bug - :tickets: 2383 - - Added a boolean check for the "finalize" - function within the pool connection proxy's - weakref callback before calling it, so that a - warning isn't emitted that this function is None - when the application is exiting and gc has - removed the function from the module before the - weakref callback was invoked. - - .. change:: - :tags: bug, py3k - :tickets: 2348 - - Fixed inappropriate usage of util.py3k - flag and renamed it to util.py3k_warning, since - this flag is intended to detect the -3 flag - series of import restrictions only. - - .. change:: - :tags: examples, feature - :tickets: 2313 - - Simplified the versioning example - a bit to use a declarative mixin as well - as an event listener, instead of a metaclass + - SessionExtension. - - .. change:: - :tags: examples, bug - :tickets: 2346 - - Fixed large_collection.py to close the - session before dropping tables. - -.. changelog:: - :version: 0.7.4 - :released: Fri Dec 09 2011 - - .. change:: - :tags: orm, bug - :tickets: 2315 - - Fixed backref behavior when "popping" the - value off of a many-to-one in response to - a removal from a stale one-to-many - the operation - is skipped, since the many-to-one has since - been updated. - - .. change:: - :tags: orm, bug - :tickets: 2264 - - After some years of not doing this, added - more granularity to the "is X a parent of Y" - functionality, which is used when determining - if the FK on "Y" needs to be "nulled out" as well - as if "Y" should be deleted with delete-orphan - cascade. The test now takes into account the - Python identity of the parent as well its identity - key, to see if the last known parent of Y is - definitely X. If a decision - can't be made, a StaleDataError is raised. The - conditions where this error is raised are fairly - rare, requiring that the previous parent was - garbage collected, and previously - could very well inappropriately update/delete - a record that's since moved onto a new parent, - though there may be some cases where - "silent success" occurred previously that will now - raise in the face of ambiguity. - Expiring "Y" resets the "parent" tracker, meaning - X.remove(Y) could then end up deleting Y even - if X is stale, but this is the same behavior - as before; it's advised to expire X also in that - case. - - .. change:: - :tags: orm, bug - :tickets: 2310 - - fixed inappropriate evaluation of user-mapped - object in a boolean context within query.get(). Also in 0.6.9. - - .. change:: - :tags: orm, bug - :tickets: 2304 - - Added missing comma to PASSIVE_RETURN_NEVER_SET - symbol - - .. change:: - :tags: orm, bug - :tickets: 1776 - - Cls.column.collate("some collation") now - works. Also in 0.6.9 - - .. change:: - :tags: orm, bug - :tickets: 2309 - - the value of a composite attribute is now - expired after an insert or update operation, instead - of regenerated in place. This ensures that a - column value which is expired within a flush - will be loaded first, before the composite - is regenerated using that value. - - .. change:: - :tags: orm, bug - :tickets: 2309, 2308 - - The fix in also emits the - "refresh" event when the composite value is - loaded on access, even if all column - values were already present, as is appropriate. - This fixes the "mutable" extension which relies - upon the "load" event to ensure the _parents - dictionary is up to date, fixes. - Thanks to Scott Torborg for the test case here. - - .. change:: - :tags: orm, bug - :tickets: 2312 - - Fixed bug whereby a subclass of a subclass - using concrete inheritance in conjunction with - the new ConcreteBase or AbstractConcreteBase - would fail to apply the subclasses deeper than - one level to the "polymorphic loader" of each - base - - .. change:: - :tags: orm, bug - :tickets: 2312 - - Fixed bug whereby a subclass of a subclass - using the new AbstractConcreteBase would fail - to acquire the correct "base_mapper" attribute - when the "base" mapper was generated, thereby - causing failures later on. - - .. change:: - :tags: orm, bug - :tickets: 2316 - - Fixed bug whereby column_property() created - against ORM-level column could be treated as - a distinct entity when producing certain - kinds of joined-inh joins. - - .. change:: - :tags: orm, bug - :tickets: 2297 - - Fixed the error formatting raised when - a tuple is inadvertently passed to session.query(). Also in 0.6.9. - - .. change:: - :tags: orm, bug - :tickets: 2328 - - Calls to query.join() to a single-table - inheritance subclass are now tracked, and - are used to eliminate the additional WHERE.. - IN criterion normally tacked on with single - table inheritance, since the join should - accommodate it. This allows OUTER JOIN - to a single table subclass to produce - the correct results, and overall will produce - fewer WHERE criterion when dealing with - single table inheritance joins. - - .. change:: - :tags: orm, bug - :tickets: 2339 - - __table_args__ can now be passed as - an empty tuple as well as an empty dict.. Thanks to Fayaz Yusuf Khan - for the patch. - - .. change:: - :tags: orm, bug - :tickets: 2325 - - Updated warning message when setting - delete-orphan without delete to no longer - refer to 0.6, as we never got around to - upgrading this to an exception. Ideally - this might be better as an exception but - it's not critical either way. - - .. change:: - :tags: orm, feature - :tickets: 2345, 2238 - - polymorphic_on now accepts many - new kinds of values: - - * standalone expressions that aren't - otherwise mapped - * column_property() objects - * string names of any column_property() - or attribute name of a mapped Column - - The docs include an example using - the case() construct, which is likely to be - a common constructed used here. and part of - - Standalone expressions in polymorphic_on - propagate to single-table inheritance - subclasses so that they are used in the - WHERE /JOIN clause to limit rows to that - subclass as is the usual behavior. - - .. change:: - :tags: orm, feature - :tickets: 2301 - - IdentitySet supports the - operator - as the same as difference(), handy when dealing - with Session.dirty etc. - - .. change:: - :tags: orm, feature - :tickets: - - Added new value for Column autoincrement - called "ignore_fk", can be used to force autoincrement - on a column that's still part of a ForeignKeyConstraint. - New example in the relationship docs illustrates - its use. - - .. change:: - :tags: orm, bug - :tickets: - - Fixed bug in get_history() when referring - to a composite attribute that has no value; - added coverage for get_history() regarding - composites which is otherwise just a userland - function. - - .. change:: - :tags: bug, sql - :tickets: 2316, 2261 - - related to, made some - adjustments to the change from - regarding the "from" list on a select(). The - _froms collection is no longer memoized, as this - simplifies various use cases and removes the - need for a "warning" if a column is attached - to a table after it was already used in an - expression - the select() construct will now - always produce the correct expression. - There's probably no real-world - performance hit here; select() objects are - almost always made ad-hoc, and systems that - wish to optimize the re-use of a select() - would be using the "compiled_cache" feature. - A hit which would occur when calling select.bind - has been reduced, but the vast majority - of users shouldn't be using "bound metadata" - anyway :). - - .. change:: - :tags: feature, sql - :tickets: 2166, 1944 - - The update() construct can now accommodate - multiple tables in the WHERE clause, which will - render an "UPDATE..FROM" construct, recognized by - PostgreSQL and MSSQL. When compiled on MySQL, - will instead generate "UPDATE t1, t2, ..". MySQL - additionally can render against multiple tables in the - SET clause, if Column objects are used as keys - in the "values" parameter or generative method. - - .. change:: - :tags: feature, sql - :tickets: 77 - - Added accessor to types called "python_type", - returns the rudimentary Python type object - for a particular TypeEngine instance, if known, - else raises NotImplementedError. - - .. change:: - :tags: bug, sql - :tickets: 2261, 2319 - - further tweak to the fix from, - so that generative methods work a bit better - off of cloned (this is almost a non-use case though). - In particular this allows with_only_columns() - to behave more consistently. Added additional - documentation to with_only_columns() to clarify - expected behavior, which changed as a result - of. - - .. change:: - :tags: engine, bug - :tickets: 2317 - - Fixed bug whereby transaction.rollback() - would throw an error on an invalidated - connection if the transaction were a - two-phase or savepoint transaction. - For plain transactions, rollback() is a no-op - if the connection is invalidated, so while - it wasn't 100% clear if it should be a no-op, - at least now the interface is consistent. - - .. change:: - :tags: feature, schema - :tickets: - - Added new support for remote "schemas": - - .. change:: - :tags: schema - :tickets: - - MetaData() accepts "schema" and "quote_schema" - arguments, which will be applied to the same-named - arguments of a Table - or Sequence which leaves these at their default - of ``None``. - - .. change:: - :tags: schema - :tickets: - - Sequence accepts "quote_schema" argument - - .. change:: - :tags: schema - :tickets: - - tometadata() for Table will use the "schema" - of the incoming MetaData for the new Table - if the schema argument is explicitly "None" - - .. change:: - :tags: schema - :tickets: - - Added CreateSchema and DropSchema DDL - constructs - these accept just the string - name of a schema and a "quote" flag. - - .. change:: - :tags: schema - :tickets: - - When using default "schema" with MetaData, - ForeignKey will also assume the "default" schema - when locating remote table. This allows the "schema" - argument on MetaData to be applied to any - set of Table objects that otherwise don't have - a "schema". - - .. change:: - :tags: schema - :tickets: 1679 - - a "has_schema" method has been implemented - on dialect, but only works on PostgreSQL so far. - Courtesy Manlio Perillo. - - .. change:: - :tags: feature, schema - :tickets: 1410 - - The "extend_existing" flag on Table - now allows for the reflection process to take - effect for a Table object that's already been - defined; when autoload=True and extend_existing=True - are both set, the full set of columns will be - reflected from the Table which will then - *overwrite* those columns already present, - rather than no activity occurring. Columns that - are present directly in the autoload run - will be used as always, however. - - .. change:: - :tags: bug, schema - :tickets: - - Fixed bug whereby TypeDecorator would - return a stale value for _type_affinity, when - using a TypeDecorator that "switches" types, - like the CHAR/UUID type. - - .. change:: - :tags: bug, schema - :tickets: - - Fixed bug whereby "order_by='foreign_key'" - option to Inspector.get_table_names - wasn't implementing the sort properly, replaced - with the existing sort algorithm - - .. change:: - :tags: bug, schema - :tickets: 2305 - - the "name" of a column-level CHECK constraint, - if present, is now rendered in the CREATE TABLE - statement using "CONSTRAINT CHECK ". - - .. change:: - :tags: pyodbc, bug - :tickets: 2318 - - pyodbc-based dialects now parse the - pyodbc accurately as far as observed - pyodbc strings, including such gems - as "py3-3.0.1-beta4" - - .. change:: - :tags: postgresql, bug - :tickets: 2311 - - PostgreSQL dialect memoizes that an ENUM of a - particular name was processed - during a create/drop sequence. This allows - a create/drop sequence to work without any - calls to "checkfirst", and also means with - "checkfirst" turned on it only needs to - check for the ENUM once. - - .. change:: - :tags: postgresql, feature - :tickets: - - Added create_type constructor argument - to pg.ENUM. When False, no CREATE/DROP or - checking for the type will be performed as part - of a table create/drop event; only the - create()/drop)() methods called directly - will do this. Helps with Alembic "offline" - scripts. - - .. change:: - :tags: mssql, feature - :tickets: 822 - - lifted the restriction on SAVEPOINT - for SQL Server. All tests pass using it, - it's not known if there are deeper issues - however. - - .. change:: - :tags: mssql, bug - :tickets: 2336 - - repaired the with_hint() feature which - wasn't implemented correctly on MSSQL - - usually used for the "WITH (NOLOCK)" hint - (which you shouldn't be using anyway ! - use snapshot isolation instead :) ) - - .. change:: - :tags: mssql, bug - :tickets: 2318 - - use new pyodbc version detection for - _need_decimal_fix option. - - .. change:: - :tags: mssql, bug - :tickets: 2343 - - don't cast "table name" as NVARCHAR - on SQL Server 2000. Still mostly in the dark - what incantations are needed to make PyODBC - work fully with FreeTDS 0.91 here, however. - - .. change:: - :tags: mssql, bug - :tickets: 2269 - - Decode incoming values when retrieving - list of index names and the names of columns - within those indexes. - - .. change:: - :tags: bug, mysql - :tickets: - - Unicode adjustments allow latest pymysql - (post 0.4) to pass 100% on Python 2. - - .. change:: - :tags: ext, feature - :tickets: - - Added an example to the hybrid docs - of a "transformer" - a hybrid that returns a - query-transforming callable in combination - with a custom comparator. Uses a new method - on Query called with_transformation(). The use - case here is fairly experimental, but only - adds one line of code to Query. - - .. change:: - :tags: ext, bug - :tickets: - - the @compiles decorator raises an - informative error message when no "default" - compilation handler is present, rather - than KeyError. - - .. change:: - :tags: examples, bug - :tickets: - - Fixed bug in history_meta.py example where - the "unique" flag was not removed from a - single-table-inheritance subclass which - generates columns to put up onto the base. - -.. changelog:: - :version: 0.7.3 - :released: Sun Oct 16 2011 - - .. change:: - :tags: general - :tickets: 2279 - - Adjusted the "importlater" mechanism, which is - used internally to resolve import cycles, - such that the usage of __import__ is completed - when the import of sqlalchemy or sqlalchemy.orm - is done, thereby avoiding any usage of __import__ - after the application starts new threads, - fixes. Also in 0.6.9. - - .. change:: - :tags: orm - :tickets: 2298 - - Improved query.join() such that the "left" side - can more flexibly be a non-ORM selectable, - such as a subquery. A selectable placed - in select_from() will now be used as the left - side, favored over implicit usage - of a mapped entity. - If the join still fails based on lack of - foreign keys, the error message includes - this detail. Thanks to brianrhude - on IRC for the test case. - - .. change:: - :tags: orm - :tickets: 2241 - - Added after_soft_rollback() Session event. This - event fires unconditionally whenever rollback() - is called, regardless of if an actual DBAPI - level rollback occurred. This event - is specifically designed to allow operations - with the Session to proceed after a rollback - when the Session.is_active is True. - - .. change:: - :tags: orm - :tickets: - - added "adapt_on_names" boolean flag to orm.aliased() - construct. Allows an aliased() construct - to link the ORM entity to a selectable that contains - aggregates or other derived forms of a particular - attribute, provided the name is the same as that - of the entity mapped column. - - .. change:: - :tags: orm - :tickets: - - Added new flag expire_on_flush=False to column_property(), - marks those properties that would otherwise be considered - to be "readonly", i.e. derived from SQL expressions, - to retain their value after a flush has occurred, including - if the parent object itself was involved in an update. - - .. change:: - :tags: orm - :tickets: 2237 - - Enhanced the instrumentation in the ORM to support - Py3K's new argument style of "required kw arguments", - i.e. fn(a, b, \*, c, d), fn(a, b, \*args, c, d). - Argument signatures of mapped object's __init__ - method will be preserved, including required kw rules. - - .. change:: - :tags: orm - :tickets: 2282 - - Fixed bug in unit of work whereby detection of - "cycles" among classes in highly interlinked patterns - would not produce a deterministic - result; thereby sometimes missing some nodes that - should be considered cycles and causing further - issues down the road. Note this bug is in 0.6 - also; not backported at the moment. - - .. change:: - :tags: orm - :tickets: - - Fixed a variety of synonym()-related regressions - from 0.6: - - * making a synonym against a synonym now works. - * synonyms made against a relationship() can - be passed to query.join(), options sent - to query.options(), passed by name - to query.with_parent(). - - .. change:: - :tags: orm - :tickets: 2287 - - Fixed bug whereby mapper.order_by attribute would - be ignored in the "inner" query within a - subquery eager load. . - Also in 0.6.9. - - .. change:: - :tags: orm - :tickets: 2267 - - Identity map .discard() uses dict.pop(,None) - internally instead of "del" to avoid KeyError/warning - during a non-determinate gc teardown - - .. change:: - :tags: orm - :tickets: 2253 - - Fixed regression in new composite rewrite where - deferred=True option failed due to missing - import - - .. change:: - :tags: orm - :tickets: 2248 - - Reinstated "comparator_factory" argument to - composite(), removed when 0.7 was released. - - .. change:: - :tags: orm - :tickets: 2247 - - Fixed bug in query.join() which would occur - in a complex multiple-overlapping path scenario, - where the same table could be joined to - twice. Thanks *much* to Dave Vitek - for the excellent fix here. - - .. change:: - :tags: orm - :tickets: - - Query will convert an OFFSET of zero when - slicing into None, so that needless OFFSET - clauses are not invoked. - - .. change:: - :tags: orm - :tickets: - - Repaired edge case where mapper would fail - to fully update internal state when a relationship - on a new mapper would establish a backref on the - first mapper. - - .. change:: - :tags: orm - :tickets: 2260 - - Fixed bug whereby if __eq__() was - redefined, a relationship many-to-one lazyload - would hit the __eq__() and fail. - Does not apply to 0.6.9. - - .. change:: - :tags: orm - :tickets: 2196 - - Calling class_mapper() and passing in an object - that is not a "type" (i.e. a class that could - potentially be mapped) now raises an informative - ArgumentError, rather than UnmappedClassError. - - .. change:: - :tags: orm - :tickets: - - New event hook, MapperEvents.after_configured(). - Called after a configure() step has completed and - mappers were in fact affected. Theoretically this - event is called once per application, unless new mappings - are constructed after existing ones have been used - already. - - .. change:: - :tags: orm - :tickets: 2281 - - When an open Session is garbage collected, the objects - within it which remain are considered detached again - when they are add()-ed to a new Session. - This is accomplished by an extra check that the previous - "session_key" doesn't actually exist among the pool - of Sessions. - - .. change:: - :tags: orm - :tickets: 2239 - - New declarative features: - - * __declare_last__() method, establishes an event - listener for the class method that will be called - when mappers are completed with the final "configure" - step. - * __abstract__ flag. The class will not be mapped - at all when this flag is present on the class. - * New helper classes ConcreteBase, AbstractConcreteBase. - Allow concrete mappings using declarative which automatically - set up the "polymorphic_union" when the "configure" - mapper step is invoked. - * The mapper itself has semi-private methods that allow - the "with_polymorphic" selectable to be assigned - to the mapper after it has already been configured. - - .. change:: - :tags: orm - :tickets: 2283 - - Declarative will warn when a subclass' base uses - @declared_attr for a regular column - this attribute - does not propagate to subclasses. - - .. change:: - :tags: orm - :tickets: 2280 - - The integer "id" used to link a mapped instance with - its owning Session is now generated by a sequence - generation function rather than id(Session), to - eliminate the possibility of recycled id() values - causing an incorrect result, no need to check that - object actually in the session. - - .. change:: - :tags: orm - :tickets: 2257 - - Behavioral improvement: empty - conjunctions such as and_() and or_() will be - flattened in the context of an enclosing conjunction, - i.e. and_(x, or_()) will produce 'X' and not 'X AND - ()'.. - - .. change:: - :tags: orm - :tickets: 2261 - - Fixed bug regarding calculation of "from" list - for a select() element. The "from" calc is now - delayed, so that if the construct uses a Column - object that is not yet attached to a Table, - but is later associated with a Table, it generates - SQL using the table as a FROM. This change - impacted fairly deeply the mechanics of how - the FROM list as well as the "correlates" collection - is calculated, as some "clause adaption" schemes - (these are used very heavily in the ORM) - were relying upon the fact that the "froms" - collection would typically be cached before the - adaption completed. The rework allows it - such that the "froms" collection can be cleared - and re-generated at any time. - - .. change:: - :tags: orm - :tickets: 2270 - - Fixed bug whereby with_only_columns() method of - Select would fail if a selectable were passed.. Also in 0.6.9. - - .. change:: - :tags: schema - :tickets: 2284 - - Modified Column.copy() to use _constructor(), - which defaults to self.__class__, in order to - create the new object. This allows easier support - of subclassing Column. - - .. change:: - :tags: schema - :tickets: 2223 - - Added a slightly nicer __repr__() to SchemaItem - classes. Note the repr here can't fully support - the "repr is the constructor" idea since schema - items can be very deeply nested/cyclical, have - late initialization of some things, etc. - - .. change:: - :tags: engine - :tickets: 2254 - - The recreate() method in all pool classes uses - self.__class__ to get at the type of pool - to produce, in the case of subclassing. Note - there's no usual need to subclass pools. - - .. change:: - :tags: engine - :tickets: 2243 - - Improvement to multi-param statement logging, - long lists of bound parameter sets will be - compressed with an informative indicator - of the compression taking place. Exception - messages use the same improved formatting. - - .. change:: - :tags: engine - :tickets: - - Added optional "sa_pool_key" argument to - pool.manage(dbapi).connect() so that serialization - of args is not necessary. - - .. change:: - :tags: engine - :tickets: 2286 - - The entry point resolution supported by - create_engine() now supports resolution of - individual DBAPI drivers on top of a built-in - or entry point-resolved dialect, using the - standard '+' notation - it's converted to - a '.' before being resolved as an entry - point. - - .. change:: - :tags: engine - :tickets: 2299 - - Added an exception catch + warning for the - "return unicode detection" step within connect, - allows databases that crash on NVARCHAR to - continue initializing, assuming no NVARCHAR - type implemented. - - .. change:: - :tags: types - :tickets: 2258 - - Extra keyword arguments to the base Float - type beyond "precision" and "asdecimal" are ignored; - added a deprecation warning here and additional - docs, related to - - .. change:: - :tags: sqlite - :tickets: - - Ensured that the same ValueError is raised for - illegal date/time/datetime string parsed from - the database regardless of whether C - extensions are in use or not. - - .. change:: - :tags: postgresql - :tickets: 2290 - - Added "postgresql_using" argument to Index(), produces - USING clause to specify index implementation for - PG. . Thanks to Ryan P. Kelly for - the patch. - - .. change:: - :tags: postgresql - :tickets: 1839 - - Added client_encoding parameter to create_engine() - when the postgresql+psycopg2 dialect is used; - calls the psycopg2 set_client_encoding() method - with the value upon connect. - - .. change:: - :tags: postgresql - :tickets: 2291, 2141 - - Fixed bug related to whereby the - same modified index behavior in PG 9 affected - primary key reflection on a renamed column.. Also in 0.6.9. - - .. change:: - :tags: postgresql - :tickets: 2256 - - Reflection functions for Table, Sequence no longer - case insensitive. Names can be differ only in case - and will be correctly distinguished. - - .. change:: - :tags: postgresql - :tickets: - - Use an atomic counter as the "random number" - source for server side cursor names; - conflicts have been reported in rare cases. - - .. change:: - :tags: postgresql - :tickets: 2249 - - Narrowed the assumption made when reflecting - a foreign-key referenced table with schema in - the current search path; an explicit schema will - be applied to the referenced table only if - it actually matches that of the referencing table, - which also has an explicit schema. Previously - it was assumed that "current" schema was synonymous - with the full search_path. - - .. change:: - :tags: mysql - :tickets: 2225 - - a CREATE TABLE will put the COLLATE option - after CHARSET, which appears to be part of - MySQL's arbitrary rules regarding if it will actually - work or not. Also in 0.6.9. - - .. change:: - :tags: mysql - :tickets: 2293 - - Added mysql_length parameter to Index construct, - specifies "length" for indexes. - - .. change:: - :tags: mssql - :tickets: 2273 - - Changes to attempt support of FreeTDS 0.91 with - Pyodbc. This includes that string binds are sent as - Python unicode objects when FreeTDS 0.91 is detected, - and a CAST(? AS NVARCHAR) is used when we detect - for a table. However, I'd continue - to characterize Pyodbc + FreeTDS 0.91 behavior as - pretty crappy, there are still many queries such - as used in reflection which cause a core dump on - Linux, and it is not really usable at all - on OSX, MemoryErrors abound and just plain broken - unicode support. - - .. change:: - :tags: mssql - :tickets: 2277 - - The behavior of =/!= when comparing a scalar select - to a value will no longer produce IN/NOT IN as of 0.8; - this behavior is a little too heavy handed (use ``in_()`` if - you want to emit IN) and now emits a deprecation warning. - To get the 0.8 behavior immediately and remove the warning, - a compiler recipe is given at - https://www.sqlalchemy.org/docs/07/dialects/mssql.html#scalar-select-comparisons - to override the behavior of visit_binary(). - - .. change:: - :tags: mssql - :tickets: 2222 - - "0" is accepted as an argument for limit() which - will produce "TOP 0". - - .. change:: - :tags: oracle - :tickets: 2272 - - Fixed ReturningResultProxy for zxjdbc dialect.. Regression from 0.6. - - .. change:: - :tags: oracle - :tickets: 2252 - - The String type now generates VARCHAR2 on Oracle - which is recommended as the default VARCHAR. - Added an explicit VARCHAR2 and NVARCHAR2 to the Oracle - dialect as well. Using NVARCHAR still generates - "NVARCHAR2" - there is no "NVARCHAR" on Oracle - - this remains a slight breakage of the "uppercase types - always give exactly that" policy. VARCHAR still - generates "VARCHAR", keeping with the policy. If - Oracle were to ever define "VARCHAR" as something - different as they claim (IMHO this will never happen), - the type would be available. - - .. change:: - :tags: ext - :tickets: 2262 - - SQLSoup will not be included in version 0.8 - of SQLAlchemy; while useful, we would like to - keep SQLAlchemy itself focused on one ORM - usage paradigm. SQLSoup will hopefully - soon be superseded by a third party - project. - - .. change:: - :tags: ext - :tickets: 2236 - - Added local_attr, remote_attr, attr accessors - to AssociationProxy, providing quick access - to the proxied attributes at the class - level. - - .. change:: - :tags: ext - :tickets: 2275 - - Changed the update() method on association proxy - dictionary to use a duck typing approach, i.e. - checks for "keys", to discern between update({}) - and update((a, b)). Previously, passing a - dictionary that had tuples as keys would be misinterpreted - as a sequence. - - .. change:: - :tags: examples - :tickets: 2266 - - Adjusted dictlike-polymorphic.py example - to apply the CAST such that it works on - PG, other databases. - Also in 0.6.9. - -.. changelog:: - :version: 0.7.2 - :released: Sun Jul 31 2011 - - .. change:: - :tags: orm - :tickets: 2213 - - Feature enhancement: joined and subquery - loading will now traverse already-present related - objects and collections in search of unpopulated - attributes throughout the scope of the eager load - being defined, so that the eager loading that is - specified via mappings or query options - unconditionally takes place for the full depth, - populating whatever is not already populated. - Previously, this traversal would stop if a related - object or collection were already present leading - to inconsistent behavior (though would save on - loads/cycles for an already-loaded graph). For a - subqueryload, this means that the additional - SELECT statements emitted by subqueryload will - invoke unconditionally, no matter how much of the - existing graph is already present (hence the - controversy). The previous behavior of "stopping" - is still in effect when a query is the result of - an attribute-initiated lazyload, as otherwise an - "N+1" style of collection iteration can become - needlessly expensive when the same related object - is encountered repeatedly. There's also an - as-yet-not-public generative Query method - _with_invoke_all_eagers() - which selects old/new behavior - - .. change:: - :tags: orm - :tickets: 2195 - - A rework of "replacement traversal" within - the ORM as it alters selectables to be against - aliases of things (i.e. clause adaption) includes - a fix for multiply-nested any()/has() constructs - against a joined table structure. - - .. change:: - :tags: orm - :tickets: 2234 - - Fixed bug where query.join() + aliased=True - from a joined-inh structure to itself on - relationship() with join condition on the child - table would convert the lead entity into the - joined one inappropriately. - Also in 0.6.9. - - .. change:: - :tags: orm - :tickets: 2205 - - Fixed regression from 0.6 where Session.add() - against an object which contained None in a - collection would raise an internal exception. - Reverted this to 0.6's behavior which is to - accept the None but obviously nothing is - persisted. Ideally, collections with None - present or on append() should at least emit a - warning, which is being considered for 0.8. - - .. change:: - :tags: orm - :tickets: 2191 - - Load of a deferred() attribute on an object - where row can't be located raises - ObjectDeletedError instead of failing later - on; improved the message in ObjectDeletedError - to include other conditions besides a simple - "delete". - - .. change:: - :tags: orm - :tickets: 2224 - - Fixed regression from 0.6 where a get history - operation on some relationship() based attributes - would fail when a lazyload would emit; this could - trigger within a flush() under certain conditions. Thanks to the user who submitted - the great test for this. - - .. change:: - :tags: orm - :tickets: 2228 - - Fixed bug apparent only in Python 3 whereby - sorting of persistent + pending objects during - flush would produce an illegal comparison, - if the persistent object primary key - is not a single integer. - Also in 0.6.9 - - .. change:: - :tags: orm - :tickets: 2197 - - Fixed bug whereby the source clause - used by query.join() would be inconsistent - if against a column expression that combined - multiple entities together. - Also in 0.6.9 - - .. change:: - :tags: orm - :tickets: 2215 - - Fixed bug whereby if a mapped class - redefined __hash__() or __eq__() to something - non-standard, which is a supported use case - as SQLA should never consult these, - the methods would be consulted if the class - was part of a "composite" (i.e. non-single-entity) - result set. - Also in 0.6.9. - - .. change:: - :tags: orm - :tickets: 2240 - - Added public attribute ".validators" to - Mapper, an immutable dictionary view of - all attributes that have been decorated - with the @validates decorator. courtesy Stefano Fontanelli - - .. change:: - :tags: orm - :tickets: 2188 - - Fixed subtle bug that caused SQL to blow - up if: column_property() against subquery + - joinedload + LIMIT + order by the column - property() occurred. . - Also in 0.6.9 - - .. change:: - :tags: orm - :tickets: 2207 - - The join condition produced by with_parent - as well as when using a "dynamic" relationship - against a parent will generate unique - bindparams, rather than incorrectly repeating - the same bindparam. . - Also in 0.6.9. - - .. change:: - :tags: orm - :tickets: - - Added the same "columns-only" check to - mapper.polymorphic_on as used when - receiving user arguments to - relationship.order_by, foreign_keys, - remote_side, etc. - - .. change:: - :tags: orm - :tickets: 2190 - - Fixed bug whereby comparison of column - expression to a Query() would not call - as_scalar() on the underlying SELECT - statement to produce a scalar subquery, - in the way that occurs if you called - it on Query().subquery(). - - .. change:: - :tags: orm - :tickets: 2194 - - Fixed declarative bug where a class inheriting - from a superclass of the same name would fail - due to an unnecessary lookup of the name - in the _decl_class_registry. - - .. change:: - :tags: orm - :tickets: 2199 - - Repaired the "no statement condition" - assertion in Query which would attempt - to raise if a generative method were called - after from_statement() were called.. Also in 0.6.9. - - .. change:: - :tags: sql - :tickets: 2188 - - Fixed two subtle bugs involving column - correspondence in a selectable, - one with the same labeled subquery repeated, the other - when the label has been "grouped" and - loses itself. Affects. - - .. change:: - :tags: schema - :tickets: 2187 - - New feature: with_variant() method on - all types. Produces an instance of Variant(), - a special TypeDecorator which will select - the usage of a different type based on the - dialect in use. - - .. change:: - :tags: schema - :tickets: - - Added an informative error message when - ForeignKeyConstraint refers to a column name in - the parent that is not found. Also in 0.6.9. - - .. change:: - :tags: schema - :tickets: 2206 - - Fixed bug whereby adaptation of old append_ddl_listener() - function was passing unexpected \**kw through - to the Table event. Table gets no kws, the MetaData - event in 0.6 would get "tables=somecollection", - this behavior is preserved. - - .. change:: - :tags: schema - :tickets: - - Fixed bug where "autoincrement" detection on - Table would fail if the type had no "affinity" - value, in particular this would occur when using - the UUID example on the site that uses TypeEngine - as the "impl". - - .. change:: - :tags: schema - :tickets: 2209 - - Added an improved repr() to TypeEngine objects - that will only display constructor args which - are positional or kwargs that deviate - from the default. - - .. change:: - :tags: engine - :tickets: - - Context manager provided by Connection.begin() - will issue rollback() if the commit() fails, - not just if an exception occurs. - - .. change:: - :tags: engine - :tickets: 1682 - - Use urllib.parse_qsl() in Python 2.6 and above, - no deprecation warning about cgi.parse_qsl() - - .. change:: - :tags: engine - :tickets: - - Added mixin class sqlalchemy.ext.DontWrapMixin. - User-defined exceptions of this type are never - wrapped in StatementException when they - occur in the context of a statement - execution. - - .. change:: - :tags: engine - :tickets: - - StatementException wrapping will display the - original exception class in the message. - - .. change:: - :tags: engine - :tickets: 2201 - - Failures on connect which raise dbapi.Error - will forward the error to dialect.is_disconnect() - and set the "connection_invalidated" flag if - the dialect knows this to be a potentially - "retryable" condition. Only Oracle ORA-01033 - implemented for now. - - .. change:: - :tags: sqlite - :tickets: 2189 - - SQLite dialect no longer strips quotes - off of reflected default value, allowing - a round trip CREATE TABLE to work. - This is consistent with other dialects - that also maintain the exact form of - the default. - - .. change:: - :tags: postgresql - :tickets: 2198 - - Added new "postgresql_ops" argument to - Index, allows specification of PostgreSQL - operator classes for indexed columns. Courtesy Filip Zyzniewski. - - .. change:: - :tags: mysql - :tickets: 2186 - - Fixed OurSQL dialect to use ansi-neutral - quote symbol "'" for XA commands instead - of '"'. . Also in 0.6.9. - - .. change:: - :tags: mssql - :tickets: - - Adjusted the pyodbc dialect such that bound - values are passed as bytes and not unicode - if the "Easysoft" unix drivers are detected. - This is the same behavior as occurs with - FreeTDS. Easysoft appears to segfault - if Python unicodes are passed under - certain circumstances. - - .. change:: - :tags: oracle - :tickets: 2200 - - Added ORA-00028 to disconnect codes, use - cx_oracle _Error.code to get at the code,. Also in 0.6.9. - - .. change:: - :tags: oracle - :tickets: 2201 - - Added ORA-01033 to disconnect codes, which - can be caught during a connection - event. - - .. change:: - :tags: oracle - :tickets: 2220 - - repaired the oracle.RAW type which did not - generate the correct DDL. - Also in 0.6.9. - - .. change:: - :tags: oracle - :tickets: 2212 - - added CURRENT to reserved word list. Also in 0.6.9. - - .. change:: - :tags: oracle - :tickets: - - Fixed bug in the mutable extension whereby - if the same type were used twice in one - mapping, the attributes beyond the first - would not get instrumented. - - .. change:: - :tags: oracle - :tickets: - - Fixed bug in the mutable extension whereby - if None or a non-corresponding type were set, - an error would be raised. None is now accepted - which assigns None to all attributes, - illegal values raise ValueError. - - .. change:: - :tags: examples - :tickets: - - Repaired the examples/versioning test runner - to not rely upon SQLAlchemy test libs, - nosetests must be run from within - examples/versioning to get around setup.cfg - breaking it. - - .. change:: - :tags: examples - :tickets: - - Tweak to examples/versioning to pick the - correct foreign key in a multi-level - inheritance situation. - - .. change:: - :tags: examples - :tickets: - - Fixed the attribute shard example to check - for bind param callable correctly in 0.7 - style. - -.. changelog:: - :version: 0.7.1 - :released: Sun Jun 05 2011 - - .. change:: - :tags: general - :tickets: 2184 - - Added a workaround for Python bug 7511 where - failure of C extension build does not - raise an appropriate exception on Windows 64 - bit + VC express - - .. change:: - :tags: orm - :tickets: 1912 - - "delete-orphan" cascade is now allowed on - self-referential relationships - this since - SQLA 0.7 no longer enforces "parent with no - child" at the ORM level; this check is left - up to foreign key nullability. - Related to - - .. change:: - :tags: orm - :tickets: 2180 - - Repaired new "mutable" extension to propagate - events to subclasses correctly; don't - create multiple event listeners for - subclasses either. - - .. change:: - :tags: orm - :tickets: 2170 - - Modify the text of the message which occurs - when the "identity" key isn't detected on - flush, to include the common cause that - the Column isn't set up to detect - auto-increment correctly;. - Also in 0.6.8. - - .. change:: - :tags: orm - :tickets: 2182 - - Fixed bug where transaction-level "deleted" - collection wouldn't be cleared of expunged - states, raising an error if they later - became transient. - Also in 0.6.8. - - .. change:: - :tags: sql - :tickets: - - Fixed bug whereby metadata.reflect(bind) - would close a Connection passed as a - bind argument. Regression from 0.6. - - .. change:: - :tags: sql - :tickets: - - Streamlined the process by which a Select - determines what's in its '.c' collection. - Behaves identically, except that a - raw ClauseList() passed to select([]) - (which is not a documented case anyway) will - now be expanded into its individual column - elements instead of being ignored. - - .. change:: - :tags: engine - :tickets: - - Deprecate schema/SQL-oriented methods on - Connection/Engine that were never well known - and are redundant: reflecttable(), create(), - drop(), text(), engine.func - - .. change:: - :tags: engine - :tickets: 2178 - - Adjusted the __contains__() method of - a RowProxy result row such that no exception - throw is generated internally; - NoSuchColumnError() also will generate its - message regardless of whether or not the column - construct can be coerced to a string.. Also in 0.6.8. - - .. change:: - :tags: sqlite - :tickets: 2173 - - Accept None from cursor.fetchone() when - "PRAGMA read_uncommitted" is called to determine - current isolation mode at connect time and - default to SERIALIZABLE; this to support SQLite - versions pre-3.3.0 that did not have this - feature. - - .. change:: - :tags: postgresql - :tickets: 2175 - - Some unit test fixes regarding numeric arrays, - MATCH operator. A potential floating-point - inaccuracy issue was fixed, and certain tests - of the MATCH operator only execute within an - EN-oriented locale for now. . - Also in 0.6.8. - - .. change:: - :tags: mysql - :tickets: - - Unit tests pass 100% on MySQL installed - on windows. - - .. change:: - :tags: mysql - :tickets: 2181 - - Removed the "adjust casing" step that would - fail when reflecting a table on MySQL - on windows with a mixed case name. After some - experimenting with a windows MySQL server, it's - been determined that this step wasn't really - helping the situation much; MySQL does not return - FK names with proper casing on non-windows - platforms either, and removing the step at - least allows the reflection to act more like - it does on other OSes. A warning here - has been considered but its difficult to - determine under what conditions such a warning - can be raised, so punted on that for now - - added some docs instead. - - .. change:: - :tags: mysql - :tickets: - - supports_sane_rowcount will be set to False - if using MySQLdb and the DBAPI doesn't provide - the constants.CLIENT module. - -.. changelog:: - :version: 0.7.0 - :released: Fri May 20 2011 - - .. change:: - :tags: - :tickets: - - This section documents those changes from 0.7b4 - to 0.7.0. For an overview of what's new in - SQLAlchemy 0.7, see - https://docs.sqlalchemy.org/en/latest/changelog/migration_07.html - - .. change:: - :tags: orm - :tickets: 2069 - - Fixed regression introduced in 0.7b4 (!) whereby - query.options(someoption("nonexistent name")) would - fail to raise an error. Also added additional - error catching for cases where the option would - try to build off a column-based element, further - fixed up some of the error messages tailored - in - - .. change:: - :tags: orm - :tickets: 2162 - - query.count() emits "count(*)" instead of - "count(1)". - - .. change:: - :tags: orm - :tickets: 2155 - - Fine tuning of Query clause adaptation when - from_self(), union(), or other "select from - myself" operation, such that plain SQL expression - elements added to filter(), order_by() etc. - which are present in the nested "from myself" - query *will* be adapted in the same way an ORM - expression element will, since these - elements are otherwise not easily accessible. - - .. change:: - :tags: orm - :tickets: 2149 - - Fixed bug where determination of "self referential" - relationship would fail with no workaround - for joined-inh subclass related to itself, - or joined-inh subclass related to a subclass - of that with no cols in the sub-sub class - in the join condition. - Also in 0.6.8. - - .. change:: - :tags: orm - :tickets: 2153 - - mapper() will ignore non-configured foreign keys - to unrelated tables when determining inherit - condition between parent and child class, - but will raise as usual for unresolved - columns and table names regarding the inherited - table. This is an enhanced generalization of - behavior that was already applied to declarative - previously. 0.6.8 has a more - conservative version of this which doesn't - fundamentally alter how join conditions - are determined. - - .. change:: - :tags: orm - :tickets: 2144 - - It is an error to call query.get() when the - given entity is not a single, full class - entity or mapper (i.e. a column). This is - a deprecation warning in 0.6.8. - - .. change:: - :tags: orm - :tickets: 2148 - - Fixed a potential KeyError which under some - circumstances could occur with the identity - map, part of - - .. change:: - :tags: orm - :tickets: - - added Query.with_session() method, switches - Query to use a different session. - - .. change:: - :tags: orm - :tickets: 2131 - - horizontal shard query should use execution - options per connection as per - - .. change:: - :tags: orm - :tickets: 2151 - - a non_primary mapper will inherit the _identity_class - of the primary mapper. This so that a non_primary - established against a class that's normally in an - inheritance mapping will produce results that are - identity-map compatible with that of the primary - mapper (also in 0.6.8) - - .. change:: - :tags: orm - :tickets: 2163 - - Fixed the error message emitted for "can't - execute syncrule for destination column 'q'; - mapper 'X' does not map this column" to - reference the correct mapper. . - Also in 0.6.8. - - .. change:: - :tags: orm - :tickets: 1502 - - polymorphic_union() gets a "cast_nulls" option, - disables the usage of CAST when it renders - the labeled NULL columns. - - .. change:: - :tags: orm - :tickets: - - polymorphic_union() renders the columns in their - original table order, as according to the first - table/selectable in the list of polymorphic - unions in which they appear. (which is itself - an unordered mapping unless you pass an OrderedDict). - - .. change:: - :tags: orm - :tickets: 2171 - - Fixed bug whereby mapper mapped to an anonymous - alias would fail if logging were used, due to - unescaped % sign in the alias name. - Also in 0.6.8. - - .. change:: - :tags: sql - :tickets: 2167 - - Fixed bug whereby nesting a label of a select() - with another label in it would produce incorrect - exported columns. Among other things this would - break an ORM column_property() mapping against - another column_property(). . - Also in 0.6.8 - - .. change:: - :tags: sql - :tickets: - - Changed the handling in determination of join - conditions such that foreign key errors are - only considered between the two given tables. - That is, t1.join(t2) will report FK errors - that involve 't1' or 't2', but anything - involving 't3' will be skipped. This affects - join(), as well as ORM relationship and - inherit condition logic. - - .. change:: - :tags: sql - :tickets: - - Some improvements to error handling inside - of the execute procedure to ensure auto-close - connections are really closed when very - unusual DBAPI errors occur. - - .. change:: - :tags: sql - :tickets: - - metadata.reflect() and reflection.Inspector() - had some reliance on GC to close connections - which were internally procured, fixed this. - - .. change:: - :tags: sql - :tickets: 2140 - - Added explicit check for when Column .name - is assigned as blank string - - .. change:: - :tags: sql - :tickets: 2147 - - Fixed bug whereby if FetchedValue was passed - to column server_onupdate, it would not - have its parent "column" assigned, added - test coverage for all column default assignment - patterns. also in 0.6.8 - - .. change:: - :tags: postgresql - :tickets: - - Fixed the psycopg2_version parsing in the - psycopg2 dialect. - - .. change:: - :tags: postgresql - :tickets: 2141 - - Fixed bug affecting PG 9 whereby index reflection - would fail if against a column whose name - had changed. . Also in 0.6.8. - - .. change:: - :tags: mssql - :tickets: 2169 - - Fixed bug in MSSQL dialect whereby the aliasing - applied to a schema-qualified table would leak - into enclosing select statements. - Also in 0.6.8. - - .. change:: - :tags: documentation - :tickets: 2152 - - Removed the usage of the "collections.MutableMapping" - abc from the ext.mutable docs as it was being used - incorrectly and makes the example more difficult - to understand in any case. - - .. change:: - :tags: examples - :tickets: - - removed the ancient "polymorphic association" - examples and replaced with an updated set of - examples that use declarative mixins, - "generic_associations". Each presents an alternative - table layout. - - .. change:: - :tags: ext - :tickets: 2143 - - Fixed bugs in sqlalchemy.ext.mutable extension where - `None` was not appropriately handled, replacement - events were not appropriately handled. - -.. changelog:: - :version: 0.7.0b4 - :released: Sun Apr 17 2011 - - .. change:: - :tags: general - :tickets: - - Changes to the format of CHANGES, this file. - The format changes have been applied to - the 0.7 releases. - - .. change:: - :tags: general - :tickets: - - The "-declarative" changes will now be listed - directly under the "-orm" section, as these - are closely related. - - .. change:: - :tags: general - :tickets: - - The 0.5 series changes have been moved to - the file CHANGES_PRE_06 which replaces - CHANGES_PRE_05. - - .. change:: - :tags: general - :tickets: - - The changelog for 0.6.7 and subsequent within - the 0.6 series is now listed only in the - CHANGES file within the 0.6 branch. - In the 0.7 CHANGES file (i.e. this file), all the - 0.6 changes are listed inline within the 0.7 - section in which they were also applied - (since all 0.6 changes are in 0.7 as well). - Changes that apply to an 0.6 version here - are noted as are if any differences in - implementation/behavior are present. - - .. change:: - :tags: orm - :tickets: 2122 - - Some fixes to "evaluate" and "fetch" evaluation - when query.update(), query.delete() are called. - The retrieval of records is done after autoflush - in all cases, and before update/delete is - emitted, guarding against unflushed data present - as well as expired objects failing during - the evaluation. - - .. change:: - :tags: orm - :tickets: 2063 - - Reworded the exception raised when a flush - is attempted of a subclass that is not polymorphic - against the supertype. - - .. change:: - :tags: orm - :tickets: - - Still more wording adjustments when a query option - can't find the target entity. Explain that the - path must be from one of the root entities. - - .. change:: - :tags: orm - :tickets: 2123 - - Some fixes to the state handling regarding - backrefs, typically when autoflush=False, where - the back-referenced collection wouldn't - properly handle add/removes with no net - change. Thanks to Richard Murri for the - test case + patch. - (also in 0.6.7). - - .. change:: - :tags: orm - :tickets: 2127 - - Added checks inside the UOW to detect the unusual - condition of being asked to UPDATE or DELETE - on a primary key value that contains NULL - in it. - - .. change:: - :tags: orm - :tickets: 2127 - - Some refinements to attribute history. More - changes are pending possibly in 0.8, but - for now history has been modified such that - scalar history doesn't have a "side effect" - of populating None for a non-present value. - This allows a slightly better ability to - distinguish between a None set and no actual - change, affects as well. - - .. change:: - :tags: orm - :tickets: 2130 - - a "having" clause would be copied from the - inside to the outside query if from_self() - were used; in particular this would break - an 0.7 style count() query. - (also in 0.6.7) - - .. change:: - :tags: orm - :tickets: 2131 - - the Query.execution_options() method now passes - those options to the Connection rather than - the SELECT statement, so that all available - options including isolation level and - compiled cache may be used. - - .. change:: - :tags: sql - :tickets: 2131 - - The "compiled_cache" execution option now raises - an error when passed to a SELECT statement - rather than a Connection. Previously it was - being ignored entirely. We may look into - having this option work on a per-statement - level at some point. - - .. change:: - :tags: sql - :tickets: - - Restored the "catchall" constructor on the base - TypeEngine class, with a deprecation warning. - This so that code which does something like - Integer(11) still succeeds. - - .. change:: - :tags: sql - :tickets: 2104 - - Fixed regression whereby MetaData() coming - back from unpickling did not keep track of - new things it keeps track of now, i.e. - collection of Sequence objects, list - of schema names. - - .. change:: - :tags: sql - :tickets: 2116 - - The limit/offset keywords to select() as well - as the value passed to select.limit()/offset() - will be coerced to integer. - (also in 0.6.7) - - .. change:: - :tags: sql - :tickets: - - fixed bug where "from" clause gathering from an - over() clause would be an itertools.chain() and - not a list, causing "can only concatenate list" - TypeError when combined with other clauses. - - .. change:: - :tags: sql - :tickets: 2134 - - Fixed incorrect usage of "," in over() clause - being placed between the "partition" and "order by" - clauses. - - .. change:: - :tags: sql - :tickets: 2105 - - Before/after attach events for PrimaryKeyConstraint - now function, tests added for before/after events - on all constraint types. - - .. change:: - :tags: sql - :tickets: 2117 - - Added explicit true()/false() constructs to expression - lib - coercion rules will intercept "False"/"True" - into these constructs. In 0.6, the constructs were - typically converted straight to string, which was - no longer accepted in 0.7. - - .. change:: - :tags: engine - :tickets: 2129 - - The C extension is now enabled by default on CPython - 2.x with a fallback to pure python if it fails to - compile. - - .. change:: - :tags: schema - :tickets: 2109 - - The 'useexisting' flag on Table has been superseded - by a new pair of flags 'keep_existing' and - 'extend_existing'. 'extend_existing' is equivalent - to 'useexisting' - the existing Table is returned, - and additional constructor elements are added. - With 'keep_existing', the existing Table is returned, - but additional constructor elements are not added - - these elements are only applied when the Table - is newly created. - - .. change:: - :tags: types - :tickets: 2081 - - REAL has been added to the core types. Supported - by PostgreSQL, SQL Server, MySQL, SQLite. Note - that the SQL Server and MySQL versions, which - add extra arguments, are also still available - from those dialects. - - .. change:: - :tags: types - :tickets: 2106 - - Added @event.listens_for() decorator, given - target + event name, applies the decorated - function as a listener. - - .. change:: - :tags: pool - :tickets: 2103 - - AssertionPool now stores the traceback indicating - where the currently checked out connection was - acquired; this traceback is reported within - the assertion raised upon a second concurrent - checkout; courtesy Gunnlaugur Briem - - .. change:: - :tags: pool - :tickets: - - The "pool.manage" feature doesn't use pickle - anymore to hash the arguments for each pool. - - .. change:: - :tags: sqlite - :tickets: 2115 - - Fixed bug where reflection of foreign key - created as "REFERENCES " without - col name would fail. - (also in 0.6.7) - - .. change:: - :tags: postgresql - :tickets: - - Psycopg2 for Python 3 is now supported. - - .. change:: - :tags: postgresql - :tickets: 2132 - - Fixed support for precision numerics when using - pg8000. - - .. change:: - :tags: oracle - :tickets: 2100 - - Using column names that would require quotes - for the column itself or for a name-generated - bind parameter, such as names with special - characters, underscores, non-ascii characters, - now properly translate bind parameter keys when - talking to cx_oracle. (Also - in 0.6.7) - - .. change:: - :tags: oracle - :tickets: 2116 - - Oracle dialect adds use_binds_for_limits=False - create_engine() flag, will render the LIMIT/OFFSET - values inline instead of as binds, reported to - modify the execution plan used by Oracle. (Also in 0.6.7) - - .. change:: - :tags: documentation - :tickets: 2029 - - Documented SQLite DATE/TIME/DATETIME types. (also in 0.6.7) - - .. change:: - :tags: documentation - :tickets: 2118 - - Fixed mutable extension docs to show the - correct type-association methods. - -.. changelog:: - :version: 0.7.0b3 - :released: Sun Mar 20 2011 - - .. change:: - :tags: general - :tickets: - - Lots of fixes to unit tests when run under PyPy - (courtesy Alex Gaynor). - - .. change:: - :tags: orm - :tickets: 2093 - - Changed the underlying approach to query.count(). - query.count() is now in all cases exactly: - - query. - from_self(func.count(literal_column('1'))). - scalar() - - That is, "select count(1) from ()". - This produces a subquery in all cases, but - vastly simplifies all the guessing count() - tried to do previously, which would still - fail in many scenarios particularly when - joined table inheritance and other joins - were involved. If the subquery produced - for an otherwise very simple count is really - an issue, use query(func.count()) as an - optimization. - - .. change:: - :tags: orm - :tickets: 2087 - - some changes to the identity map regarding - rare weakref callbacks during iterations. - The mutex has been removed as it apparently - can cause a reentrant (i.e. in one thread) deadlock, - perhaps when gc collects objects at the point of - iteration in order to gain more memory. It is hoped - that "dictionary changed during iteration" will - be exceedingly rare as iteration methods internally - acquire the full list of objects in a single values() - call. Note 0.6.7 has a more conservative fix here - which still keeps the mutex in place. - - .. change:: - :tags: orm - :tickets: 2082 - - A tweak to the unit of work causes it to order - the flush along relationship() dependencies even if - the given objects don't have any inter-attribute - references in memory, which was the behavior in - 0.5 and earlier, so a flush of Parent/Child with - only foreign key/primary key set will succeed. - This while still maintaining 0.6 and above's not - generating a ton of useless internal dependency - structures within the flush that don't correspond - to state actually within the current flush. - - .. change:: - :tags: orm - :tickets: 2069 - - Improvements to the error messages emitted when - querying against column-only entities in conjunction - with (typically incorrectly) using loader options, - where the parent entity is not fully present. - - .. change:: - :tags: orm - :tickets: 2098 - - Fixed bug in query.options() whereby a path - applied to a lazyload using string keys could - overlap a same named attribute on the wrong - entity. Note 0.6.7 has a more conservative fix - to this. - - .. change:: - :tags: declarative - :tickets: 2091 - - Arguments in __mapper_args__ that aren't "hashable" - aren't mistaken for always-hashable, possibly-column - arguments. (also in 0.6.7) - - .. change:: - :tags: sql - :tickets: - - Added a fully descriptive error message for the - case where Column is subclassed and _make_proxy() - fails to make a copy due to TypeError on the - constructor. The method _constructor should - be implemented in this case. - - .. change:: - :tags: sql - :tickets: 2095 - - Added new event "column_reflect" for Table objects. - Receives the info dictionary about a Column before - the object is generated within reflection, and allows - modification to the dictionary for control over - most aspects of the resulting Column including - key, name, type, info dictionary. - - .. change:: - :tags: sql - :tickets: - - To help with the "column_reflect" event being used - with specific Table objects instead of all instances - of Table, listeners can be added to a Table object - inline with its construction using a new argument - "listeners", a list of tuples of the form - (, ), which are applied to the Table - before the reflection process begins. - - .. change:: - :tags: sql - :tickets: 2085 - - Added new generic function "next_value()", accepts - a Sequence object as its argument and renders the - appropriate "next value" generation string on the - target platform, if supported. Also provides - ".next_value()" method on Sequence itself. - - .. change:: - :tags: sql - :tickets: 2084 - - func.next_value() or other SQL expression can - be embedded directly into an insert() construct, - and if implicit or explicit "returning" is used - in conjunction with a primary key column, - the newly generated value will be present in - result.inserted_primary_key. - - .. change:: - :tags: sql - :tickets: 2089 - - Added accessors to ResultProxy "returns_rows", - "is_insert" (also in 0.6.7) - - .. change:: - :tags: engine - :tickets: 2097 - - Fixed AssertionPool regression bug. - - .. change:: - :tags: engine - :tickets: 2060 - - Changed exception raised to ArgumentError when an - invalid dialect is specified. - - .. change:: - :tags: postgresql - :tickets: 2092 - - Added RESERVED_WORDS for postgresql dialect. - (also in 0.6.7) - - .. change:: - :tags: postgresql - :tickets: 2073 - - Fixed the BIT type to allow a "length" parameter, "varying" - parameter. Reflection also fixed. - (also in 0.6.7) - - .. change:: - :tags: mssql - :tickets: 2071 - - Rewrote the query used to get the definition of a view, - typically when using the Inspector interface, to - use sys.sql_modules instead of the information schema, - thereby allowing views definitions longer than 4000 - characters to be fully returned. - (also in 0.6.7) - - .. change:: - :tags: firebird - :tickets: 2083 - - The "implicit_returning" flag on create_engine() is - honored if set to False. (also in 0.6.7) - - .. change:: - :tags: informix - :tickets: 2092 - - Added RESERVED_WORDS informix dialect. - (also in 0.6.7) - - .. change:: - :tags: ext - :tickets: 2090 - - The horizontal_shard ShardedSession class accepts the common - Session argument "query_cls" as a constructor argument, - to enable further subclassing of ShardedQuery. (also in 0.6.7) - - .. change:: - :tags: examples - :tickets: - - Updated the association, association proxy examples - to use declarative, added a new example - dict_of_sets_with_default.py, a "pushing the envelope" - example of association proxy. - - .. change:: - :tags: examples - :tickets: 2090 - - The Beaker caching example allows a "query_cls" argument - to the query_callable() function. - (also in 0.6.7) - -.. changelog:: - :version: 0.7.0b2 - :released: Sat Feb 19 2011 - - .. change:: - :tags: orm - :tickets: 2053 - - Fixed bug whereby Session.merge() would call the - load() event with one too few arguments. - - .. change:: - :tags: orm - :tickets: 2052 - - Added logic which prevents the generation of - events from a MapperExtension or SessionExtension - from generating do-nothing events for all the methods - not overridden. - - .. change:: - :tags: declarative - :tickets: 2058 - - Fixed regression whereby composite() with - Column objects placed inline would fail - to initialize. The Column objects can now - be inline with the composite() or external - and pulled in via name or object ref. - - .. change:: - :tags: declarative - :tickets: 2061 - - Fix error message referencing old @classproperty - name to reference @declared_attr - (also in 0.6.7) - - .. change:: - :tags: declarative - :tickets: 1468 - - the dictionary at the end of the __table_args__ - tuple is now optional. - - .. change:: - :tags: sql - :tickets: 2059 - - Renamed the EngineEvents event class to - ConnectionEvents. As these classes are never - accessed directly by end-user code, this strictly - is a documentation change for end users. Also - simplified how events get linked to engines - and connections internally. - - .. change:: - :tags: sql - :tickets: 2055 - - The Sequence() construct, when passed a MetaData() - object via its 'metadata' argument, will be - included in CREATE/DROP statements within - metadata.create_all() and metadata.drop_all(), - including "checkfirst" logic. - - .. change:: - :tags: sql - :tickets: 2064 - - The Column.references() method now returns True - if it has a foreign key referencing the - given column exactly, not just its parent - table. - - .. change:: - :tags: postgresql - :tickets: 2065 - - Fixed regression from 0.6 where SMALLINT and - BIGINT types would both generate SERIAL - on an integer PK column, instead of - SMALLINT and BIGSERIAL - - .. change:: - :tags: ext - :tickets: 2054 - - Association proxy now has correct behavior for - any(), has(), and contains() when proxying - a many-to-one scalar attribute to a one-to-many - collection (i.e. the reverse of the 'typical' - association proxy use case) - - .. change:: - :tags: examples - :tickets: - - Beaker example now takes into account 'limit' - and 'offset', bind params within embedded - FROM clauses (like when you use union() or - from_self()) when generating a cache key. - -.. changelog:: - :version: 0.7.0b1 - :released: Sat Feb 12 2011 - - .. change:: - :tags: - :tickets: - - Detailed descriptions of each change below are - described at: - https://docs.sqlalchemy.org/en/latest/changelog/migration_07.html - - .. change:: - :tags: general - :tickets: 1902 - - New event system, supersedes all extensions, listeners, - etc. - - .. change:: - :tags: general - :tickets: 1926 - - Logging enhancements - - .. change:: - :tags: general - :tickets: 1949 - - Setup no longer installs a Nose plugin - - .. change:: - :tags: general - :tickets: - - The "sqlalchemy.exceptions" alias in sys.modules - has been removed. Base SQLA exceptions are - available via "from sqlalchemy import exc". - The "exceptions" alias for "exc" remains in - "sqlalchemy" for now, it's just not patched into - sys.modules. - - .. change:: - :tags: orm - :tickets: 1923 - - More succinct form of query.join(target, onclause) - - .. change:: - :tags: orm - :tickets: 1903 - - Hybrid Attributes, implements/supersedes synonym() - - .. change:: - :tags: orm - :tickets: 2008 - - Rewrite of composites - - .. change:: - :tags: orm - :tickets: - - Mutation Event Extension, supersedes "mutable=True" - - .. seealso:: - - :ref:`07_migration_mutation_extension` - - .. change:: - :tags: orm - :tickets: 1980 - - PickleType and ARRAY mutability turned off by default - - .. change:: - :tags: orm - :tickets: 1895 - - Simplified polymorphic_on assignment - - .. change:: - :tags: orm - :tickets: 1912 - - Flushing of Orphans that have no parent is allowed - - .. change:: - :tags: orm - :tickets: 2041 - - Adjusted flush accounting step to occur before - the commit in the case of autocommit=True. This allows - autocommit=True to work appropriately with - expire_on_commit=True, and also allows post-flush session - hooks to operate in the same transactional context - as when autocommit=False. - - .. change:: - :tags: orm - :tickets: 1973 - - Warnings generated when collection members, scalar referents - not part of the flush - - .. change:: - :tags: orm - :tickets: 1876 - - Non-`Table`-derived constructs can be mapped - - .. change:: - :tags: orm - :tickets: 1942 - - Tuple label names in Query Improved - - .. change:: - :tags: orm - :tickets: 1892 - - Mapped column attributes reference the most specific - column first - - .. change:: - :tags: orm - :tickets: 1896 - - Mapping to joins with two or more same-named columns - requires explicit declaration - - .. change:: - :tags: orm - :tickets: 1875 - - Mapper requires that polymorphic_on column be present - in the mapped selectable - - .. change:: - :tags: orm - :tickets: 1966 - - compile_mappers() renamed configure_mappers(), simplified - configuration internals - - .. change:: - :tags: orm - :tickets: 2018 - - the aliased() function, if passed a SQL FromClause element - (i.e. not a mapped class), will return element.alias() - instead of raising an error on AliasedClass. - - .. change:: - :tags: orm - :tickets: 2027 - - Session.merge() will check the version id of the incoming - state against that of the database, assuming the mapping - uses version ids and incoming state has a version_id - assigned, and raise StaleDataError if they don't - match. - - .. change:: - :tags: orm - :tickets: 1996 - - Session.connection(), Session.execute() accept 'bind', - to allow execute/connection operations to participate - in the open transaction of an engine explicitly. - - .. change:: - :tags: orm - :tickets: - - Query.join(), Query.outerjoin(), eagerload(), - eagerload_all(), others no longer allow lists - of attributes as arguments (i.e. option([x, y, z]) - form, deprecated since 0.5) - - .. change:: - :tags: orm - :tickets: - - ScopedSession.mapper is removed (deprecated since 0.5). - - .. change:: - :tags: orm - :tickets: 2031 - - Horizontal shard query places 'shard_id' in - context.attributes where it's accessible by the - "load()" event. - - .. change:: - :tags: orm - :tickets: 2032 - - A single contains_eager() call across - multiple entities will indicate all collections - along that path should load, instead of requiring - distinct contains_eager() calls for each endpoint - (which was never correctly documented). - - .. change:: - :tags: orm - :tickets: - - The "name" field used in orm.aliased() now renders - in the resulting SQL statement. - - .. change:: - :tags: orm - :tickets: 1473 - - Session weak_instance_dict=False is deprecated. - - .. change:: - :tags: orm - :tickets: 2046 - - An exception is raised in the unusual case that an - append or similar event on a collection occurs after - the parent object has been dereferenced, which - prevents the parent from being marked as "dirty" - in the session. Was a warning in 0.6.6. - - .. change:: - :tags: orm - :tickets: 1069 - - Query.distinct() now accepts column expressions - as \*args, interpreted by the PostgreSQL dialect - as DISTINCT ON (). - - .. change:: - :tags: orm - :tickets: 2049 - - Additional tuning to "many-to-one" relationship - loads during a flush(). A change in version 0.6.6 - ([ticket:2002]) required that more "unnecessary" m2o - loads during a flush could occur. Extra loading modes have - been added so that the SQL emitted in this - specific use case is trimmed back, while still - retrieving the information the flush needs in order - to not miss anything. - - .. change:: - :tags: orm - :tickets: - - the value of "passive" as passed to - attributes.get_history() should be one of the - constants defined in the attributes package. Sending - True or False is deprecated. - - .. change:: - :tags: orm - :tickets: 2030 - - Added a `name` argument to `Query.subquery()`, to allow - a fixed name to be assigned to the alias object. (also in 0.6.7) - - .. change:: - :tags: orm - :tickets: 2019 - - A warning is emitted when a joined-table inheriting mapper - has no primary keys on the locally mapped table - (but has pks on the superclass table). - (also in 0.6.7) - - .. change:: - :tags: orm - :tickets: 2038 - - Fixed bug where "middle" class in a polymorphic hierarchy - would have no 'polymorphic_on' column if it didn't also - specify a 'polymorphic_identity', leading to strange - errors upon refresh, wrong class loaded when querying - from that target. Also emits the correct WHERE criterion - when using single table inheritance. - (also in 0.6.7) - - .. change:: - :tags: orm - :tickets: 1995 - - Fixed bug where a column with a SQL or server side default - that was excluded from a mapping with include_properties - or exclude_properties would result in UnmappedColumnError. (also in 0.6.7) - - .. change:: - :tags: orm - :tickets: 2046 - - A warning is emitted in the unusual case that an - append or similar event on a collection occurs after - the parent object has been dereferenced, which - prevents the parent from being marked as "dirty" - in the session. This will be an exception in 0.7. (also in 0.6.7) - - .. change:: - :tags: declarative - :tickets: 2050 - - Added an explicit check for the case that the name - 'metadata' is used for a column attribute on a - declarative class. (also in 0.6.7) - - .. change:: - :tags: sql - :tickets: 1844 - - Added over() function, method to FunctionElement - classes, produces the _Over() construct which - in turn generates "window functions", i.e. - " OVER (PARTITION BY , - ORDER BY )". - - .. change:: - :tags: sql - :tickets: 805 - - LIMIT/OFFSET clauses now use bind parameters - - .. change:: - :tags: sql - :tickets: 1069 - - select.distinct() now accepts column expressions - as \*args, interpreted by the PostgreSQL dialect - as DISTINCT ON (). Note this was already - available via passing a list to the `distinct` - keyword argument to select(). - - .. change:: - :tags: sql - :tickets: - - select.prefix_with() accepts multiple expressions - (i.e. \*expr), 'prefix' keyword argument to select() - accepts a list or tuple. - - .. change:: - :tags: sql - :tickets: - - Passing a string to the `distinct` keyword argument - of `select()` for the purpose of emitting special - MySQL keywords (DISTINCTROW etc.) is deprecated - - use `prefix_with()` for this. - - .. change:: - :tags: sql - :tickets: 2006, 2005 - - TypeDecorator works with primary key columns - - .. change:: - :tags: sql - :tickets: 1897 - - DDL() constructs now escape percent signs - - .. change:: - :tags: sql - :tickets: 1917, 1893 - - Table.c / MetaData.tables refined a bit, don't allow direct - mutation - - .. change:: - :tags: sql - :tickets: 1950 - - Callables passed to `bindparam()` don't get evaluated - - .. change:: - :tags: sql - :tickets: 1870 - - types.type_map is now private, types._type_map - - .. change:: - :tags: sql - :tickets: 1982 - - Non-public Pool methods underscored - - .. change:: - :tags: sql - :tickets: 723 - - Added NULLS FIRST and NULLS LAST support. It's implemented - as an extension to the asc() and desc() operators, called - nullsfirst() and nullslast(). - - .. change:: - :tags: sql - :tickets: - - The Index() construct can be created inline with a Table - definition, using strings as column names, as an alternative - to the creation of the index outside of the Table. - - .. change:: - :tags: sql - :tickets: 2001 - - execution_options() on Connection accepts - "isolation_level" argument, sets transaction isolation - level for that connection only until returned to the - connection pool, for those backends which support it - (SQLite, PostgreSQL) - - .. change:: - :tags: sql - :tickets: 2005 - - A TypeDecorator of Integer can be used with a primary key - column, and the "autoincrement" feature of various dialects - as well as the "sqlite_autoincrement" flag will honor - the underlying database type as being Integer-based. - - .. change:: - :tags: sql - :tickets: 2020, 2021 - - Established consistency when server_default is present - on an Integer PK column. SQLA doesn't pre-fetch these, - nor do they come back in cursor.lastrowid (DBAPI). - Ensured all backends consistently return None - in result.inserted_primary_key for these. Regarding - reflection for this case, reflection of an int PK col - with a server_default sets the "autoincrement" flag to False, - except in the case of a PG SERIAL col where we detected a - sequence default. - - .. change:: - :tags: sql - :tickets: 2006 - - Result-row processors are applied to pre-executed SQL - defaults, as well as cursor.lastrowid, when determining - the contents of result.inserted_primary_key. - - .. change:: - :tags: sql - :tickets: - - Bind parameters present in the "columns clause" of a select - are now auto-labeled like other "anonymous" clauses, - which among other things allows their "type" to be meaningful - when the row is fetched, as in result row processors. - - .. change:: - :tags: sql - :tickets: - - TypeDecorator is present in the "sqlalchemy" import space. - - .. change:: - :tags: sql - :tickets: 2015 - - Non-DBAPI errors which occur in the scope of an `execute()` - call are now wrapped in sqlalchemy.exc.StatementError, - and the text of the SQL statement and repr() of params - is included. This makes it easier to identify statement - executions which fail before the DBAPI becomes - involved. - - .. change:: - :tags: sql - :tickets: 2048 - - The concept of associating a ".bind" directly with a - ClauseElement has been explicitly moved to Executable, - i.e. the mixin that describes ClauseElements which represent - engine-executable constructs. This change is an improvement - to internal organization and is unlikely to affect any - real-world usage. - - .. change:: - :tags: sql - :tickets: 2028 - - Column.copy(), as used in table.tometadata(), copies the - 'doc' attribute. (also in 0.6.7) - - .. change:: - :tags: sql - :tickets: 2023 - - Added some defs to the resultproxy.c extension so that - the extension compiles and runs on Python 2.4. (also in 0.6.7) - - .. change:: - :tags: sql - :tickets: 2042 - - The compiler extension now supports overriding the default - compilation of expression._BindParamClause including that - the auto-generated binds within the VALUES/SET clause - of an insert()/update() statement will also use the new - compilation rules. (also in 0.6.7) - - .. change:: - :tags: sql - :tickets: 1921 - - SQLite dialect now uses `NullPool` for file-based databases - - .. change:: - :tags: sql - :tickets: 2036 - - The path given as the location of a sqlite database is now - normalized via os.path.abspath(), so that directory changes - within the process don't affect the ultimate location - of a relative file path. - - .. change:: - :tags: postgresql - :tickets: 1083 - - When explicit sequence execution derives the name - of the auto-generated sequence of a SERIAL column, - which currently only occurs if implicit_returning=False, - now accommodates if the table + column name is greater - than 63 characters using the same logic PostgreSQL uses. (also in 0.6.7) - - .. change:: - :tags: postgresql - :tickets: 2044 - - Added an additional libpq message to the list of "disconnect" - exceptions, "could not receive data from server" (also in 0.6.7) - - .. change:: - :tags: mssql - :tickets: 1833 - - the String/Unicode types, and their counterparts VARCHAR/ - NVARCHAR, emit "max" as the length when no length is - specified, so that the default length, normally '1' - as per SQL server documentation, is instead - 'unbounded'. This also occurs for the VARBINARY type.. - - This behavior makes these types more closely compatible - with PostgreSQL's VARCHAR type which is similarly unbounded - when no length is specified. - - .. change:: - :tags: mysql - :tickets: 1991 - - New DBAPI support for pymysql, a pure Python port - of MySQL-python. - - .. change:: - :tags: mysql - :tickets: 2047 - - oursql dialect accepts the same "ssl" arguments in - create_engine() as that of MySQLdb. - (also in 0.6.7) - - .. change:: - :tags: firebird - :tickets: 1885 - - Some adjustments so that Interbase is supported as well. - FB/Interbase version idents are parsed into a structure - such as (8, 1, 1, 'interbase') or (2, 1, 588, 'firebird') - so they can be distinguished. diff --git a/doc/build/changelog/changelog_08.rst b/doc/build/changelog/changelog_08.rst deleted file mode 100644 index 363f5aeb1b8..00000000000 --- a/doc/build/changelog/changelog_08.rst +++ /dev/null @@ -1,3740 +0,0 @@ -============= -0.8 Changelog -============= - -.. changelog_imports:: - - .. include:: changelog_07.rst - :start-line: 5 - - -.. changelog:: - :version: 0.8.7 - :released: July 22, 2014 - - .. change:: - :tags: bug, mssql - :versions: 1.0.0b1, 0.9.7 - - Added statement encoding to the "SET IDENTITY_INSERT" - statements which operate when an explicit INSERT is being - interjected into an IDENTITY column, to support non-ascii table - identifiers on drivers such as pyodbc + unix + py2k that don't - support unicode statements. - - .. change:: - :tags: bug, mssql - :versions: 1.0.0b1, 0.9.7 - :tickets: 3091 - - In the SQL Server pyodbc dialect, repaired the implementation - for the ``description_encoding`` dialect parameter, which when - not explicitly set was preventing cursor.description from - being parsed correctly in the case of result sets that - contained names in alternate encodings. This parameter - shouldn't be needed going forward. - - .. change:: - :tags: bug, sql - :versions: 1.0.0b1, 0.9.7 - :tickets: 3124 - - Fixed bug in :class:`.Enum` and other :class:`.SchemaType` - subclasses where direct association of the type with a - :class:`_schema.MetaData` would lead to a hang when events - (like create events) were emitted on the :class:`_schema.MetaData`. - - .. change:: - :tags: bug, sql - :versions: 1.0.0b1, 0.9.7 - :tickets: 3102 - - Fixed a bug within the custom operator plus :meth:`.TypeEngine.with_variant` - system, whereby using a :class:`.TypeDecorator` in conjunction with - variant would fail with an MRO error when a comparison operator was used. - - .. change:: - :tags: bug, mysql - :versions: 1.0.0b1, 0.9.7 - :tickets: 3101 - - MySQL error 2014 "commands out of sync" appears to be raised as a - ProgrammingError, not OperationalError, in modern MySQL-Python versions; - all MySQL error codes that are tested for "is disconnect" are now - checked within OperationalError and ProgrammingError regardless. - - .. change:: - :tags: bug, mysql - :versions: 1.0.0b1, 0.9.5 - :tickets: 3085 - - Fixed bug where column names added to ``mysql_length`` parameter - on an index needed to have the same quoting for quoted names in - order to be recognized. The fix makes the quotes optional but - also provides the old behavior for backwards compatibility with those - using the workaround. - - .. change:: - :tags: bug, declarative - :versions: 1.0.0b1, 0.9.5 - :tickets: 3062 - - The ``__mapper_args__`` dictionary is copied from a declarative - mixin or abstract class when accessed, so that modifications made - to this dictionary by declarative itself won't conflict with that - of other mappings. The dictionary is modified regarding the - ``version_id_col`` and ``polymorphic_on`` arguments, replacing the - column within with the one that is officially mapped to the local - class/table. - - .. change:: - :tags: bug, sql - :versions: 0.9.5, 1.0.0b1 - :tickets: 3044 - - Fixed bug in INSERT..FROM SELECT construct where selecting from a - UNION would wrap the union in an anonymous (e.g. unlabeled) subquery. - - .. change:: - :tags: bug, postgresql - :versions: 0.9.5, 1.0.0b1 - :tickets: 3053 - - Added the ``hashable=False`` flag to the PG :class:`.HSTORE` type, which - is needed to allow the ORM to skip over trying to "hash" an ORM-mapped - HSTORE column when requesting it in a mixed column/entity list. - Patch courtesy Gunnlaugur Þór Briem. - - .. change:: - :tags: bug, orm - :versions: 0.9.5, 1.0.0b1 - :tickets: 3055 - - Fixed bug in subquery eager loading where a long chain of - eager loads across a polymorphic-subclass boundary in conjunction - with polymorphic loading would fail to locate the subclass-link in the - chain, erroring out with a missing property name on an - :class:`.AliasedClass`. - - .. change:: - :tags: bug, ext - :versions: 0.9.5, 1.0.0b1 - :tickets: 3051, 3093 - - Fixed bug in mutable extension where :class:`.MutableDict` did not - report change events for the ``setdefault()`` dictionary operation. - - .. change:: - :tags: bug, ext - :versions: 0.9.5, 1.0.0b1 - :tickets: 3093, 3051 - - Fixed bug where :meth:`.MutableDict.setdefault` didn't return the - existing or new value (this bug was not released in any 0.8 version). - Pull request courtesy Thomas Hervé. - - .. change:: - :tags: bug, mysql - :versions: 0.9.5, 1.0.0b1 - - Added support for reflecting tables where an index includes - KEY_BLOCK_SIZE using an equal sign. Pull request courtesy - Sean McGivern. - - .. change:: - :tags: bug, orm - :tickets: 3047 - :versions: 0.9.5, 1.0.0b1 - - Fixed ORM bug where the :func:`.class_mapper` function would mask - AttributeErrors or KeyErrors that should raise during mapper - configuration due to user errors. The catch for attribute/keyerror - has been made more specific to not include the configuration step. - - .. change:: - :tags: bug, sql - :tickets: 3045 - :versions: 0.9.5, 1.0.0b1 - - Fixed bug where :meth:`_schema.Table.update` and :meth:`_schema.Table.delete` - would produce an empty WHERE clause when an empty :func:`.and_()` - or :func:`.or_()` or other blank expression were applied. This is - now consistent with that of :func:`_expression.select`. - - .. change:: - :tags: bug, postgresql - :versions: 0.9.5, 1.0.0b1 - - Added a new "disconnect" message "connection has been closed unexpectedly". - This appears to be related to newer versions of SSL. - Pull request courtesy Antti Haapala. - -.. changelog:: - :version: 0.8.6 - :released: March 28, 2014 - - .. change:: - :tags: bug, orm - :tickets: 3006 - :versions: 0.9.4 - - Fixed ORM bug where changing the primary key of an object, then marking - it for DELETE would fail to target the correct row for DELETE. - - .. change:: - :tags: feature, postgresql - :versions: 0.9.4 - - Enabled "sane multi-row count" checking for the psycopg2 DBAPI, as - this seems to be supported as of psycopg2 2.0.9. - - .. change:: - :tags: bug, postgresql - :tickets: 3000 - :versions: 0.9.4 - - Fixed regression caused by release 0.8.5 / 0.9.3's compatibility - enhancements where index reflection on PostgreSQL versions specific - to only the 8.1, 8.2 series again - broke, surrounding the ever problematic int2vector type. While - int2vector supports array operations as of 8.1, apparently it only - supports CAST to a varchar as of 8.3. - - .. change:: - :tags: bug, orm - :tickets: 2995, - :versions: 0.9.4 - - Fixed regression from 0.8.3 as a result of :ticket:`2818` - where :meth:`_query.Query.exists` wouldn't work on a query that only - had a :meth:`_query.Query.select_from` entry but no other entities. - - .. change:: - :tags: bug, general - :tickets: 2986 - :versions: 0.9.4 - - Adjusted ``setup.py`` file to support the possible future - removal of the ``setuptools.Feature`` extension from setuptools. - If this keyword isn't present, the setup will still succeed - with setuptools rather than falling back to distutils. C extension - building can be disabled now also by setting the - DISABLE_SQLALCHEMY_CEXT environment variable. This variable works - whether or not setuptools is even available. - - .. change:: - :tags: bug, ext - :versions: 0.9.4 - :tickets: 2997 - - Fixed bug in mutable extension as well as - :func:`.attributes.flag_modified` where the change event would not be - propagated if the attribute had been reassigned to itself. - - .. change:: - :tags: bug, orm - :versions: 0.9.4 - - Improved an error message which would occur if a query() were made - against a non-selectable, such as a :func:`_expression.literal_column`, and then - an attempt was made to use :meth:`_query.Query.join` such that the "left" - side would be determined as ``None`` and then fail. This condition - is now detected explicitly. - - .. change:: - :tags: bug, sql - :versions: 0.9.4 - :tickets: 2977 - - Fixed bug in :func:`.tuple_` construct where the "type" of essentially - the first SQL expression would be applied as the "comparison type" - to a compared tuple value; this has the effect in some cases of an - inappropriate "type coercion" occurring, such as when a tuple that - has a mix of String and Binary values improperly coerces target - values to Binary even though that's not what they are on the left - side. :func:`.tuple_` now expects heterogeneous types within its - list of values. - - .. change:: - :tags: orm, bug - :versions: 0.9.4 - :tickets: 2975 - - Removed stale names from ``sqlalchemy.orm.interfaces.__all__`` and - refreshed with current names, so that an ``import *`` from this - module again works. - -.. changelog:: - :version: 0.8.5 - :released: February 19, 2014 - - .. change:: - :tags: postgresql, bug - :versions: 0.9.3 - :tickets: 2936 - - Added an additional message to psycopg2 disconnect detection, - "could not send data to server", which complements the existing - "could not receive data from server" and has been observed by users. - - .. change:: - :tags: postgresql, bug - :versions: 0.9.3 - - Support has been improved for PostgreSQL reflection behavior on very old - (pre 8.1) versions of PostgreSQL, and potentially other PG engines - such as Redshift (assuming Redshift reports the version as < 8.1). - The query for "indexes" as well as "primary keys" relies upon inspecting - a so-called "int2vector" datatype, which refuses to coerce to an array - prior to 8.1 causing failures regarding the "ANY()" operator used - in the query. Extensive googling has located the very hacky, but - recommended-by-PG-core-developer query to use when PG version < 8.1 - is in use, so index and primary key constraint reflection now work - on these versions. - - - .. change:: - :tags: feature, mysql - :versions: 0.9.3 - :tickets: 2941 - - Added new MySQL-specific :class:`.mysql.DATETIME` which includes - fractional seconds support; also added fractional seconds support - to :class:`.mysql.TIMESTAMP`. DBAPI support is limited, though - fractional seconds are known to be supported by MySQL Connector/Python. - Patch courtesy Geert JM Vanderkelen. - - .. change:: - :tags: bug, mysql - :versions: 0.9.3 - :tickets: 2966 - - Added support for the ``PARTITION BY`` and ``PARTITIONS`` - MySQL table keywords, specified as ``mysql_partition_by='value'`` and - ``mysql_partitions='value'`` to :class:`_schema.Table`. Pull request - courtesy Marcus McCurdy. - - .. change:: - :tags: bug, sql - :versions: 0.9.3 - :tickets: 2944 - - Fixed bug where calling :meth:`_expression.Insert.values` with an empty list - or tuple would raise an IndexError. It now produces an empty - insert construct as would be the case with an empty dictionary. - - .. change:: - :tags: bug, engine, pool - :versions: 0.9.3 - :tickets: 2880, 2964 - - Fixed a critical regression caused by :ticket:`2880` where the newly - concurrent ability to return connections from the pool means that the - "first_connect" event is now no longer synchronized either, thus leading - to dialect mis-configurations under even minimal concurrency situations. - - .. change:: - :tags: bug, sqlite - - Restored a change that was missed in the backport of unique - constraint reflection to 0.8, where :class:`.UniqueConstraint` - with SQLite would fail if reserved keywords were included in the - names of columns. Pull request courtesy Roman Podolyaka. - - .. change:: - :tags: bug, postgresql - :tickets: 2291 - :versions: 0.9.3 - - Revised this very old issue where the PostgreSQL "get primary key" - reflection query were updated to take into account primary key constraints - that were renamed; the newer query fails on very old versions of - PostgreSQL such as version 7, so the old query is restored in those cases - when server_version_info < (8, 0) is detected. - - .. change:: - :tags: bug, sql - :tickets: 2957 - :versions: 0.9.3 - - Fixed bug where :meth:`.ColumnOperators.in_` would go into an endless - loop if erroneously passed a column expression whose comparator - included the ``__getitem__()`` method, such as a column that uses the - :class:`_postgresql.ARRAY` type. - - .. change:: - :tags: bug, orm - :tickets: 2951 - :versions: 0.9.3 - - Fixed bug where :meth:`_query.Query.get` would fail to consistently - raise the :class:`.InvalidRequestError` that invokes when called - on a query with existing criterion, when the given identity is - already present in the identity map. - - .. change:: - :tags: bug, mysql - :tickets: 2933 - :versions: 0.9.3 - - Fixed bug which prevented MySQLdb-based dialects (e.g. - pymysql) from working in Py3K, where a check for "connection - charset" would fail due to Py3K's more strict value comparison - rules. The call in question wasn't taking the database - version into account in any case as the server version was - still None at that point, so the method overall has been - simplified to rely upon connection.character_set_name(). - - .. change:: - :tags: bug, mysql - :versions: 0.9.2 - - Some missing methods added to the cymysql dialect, including - _get_server_version_info() and _detect_charset(). Pullreq - courtesy Hajime Nakagami. - - .. change:: - :tags: bug, py3k - - Fixed Py3K bug where a missing import would cause "literal binary" - mode to fail to import "util.binary_type" when rendering a bound - parameter. 0.9 handles this differently. Pull request courtesy - Andreas Zeidler. - - .. change:: - :tags: bug, orm - :versions: 0.9.2 - - Fixed error message when an iterator object is passed to - :func:`.class_mapper` or similar, where the error would fail to - render on string formatting. Pullreq courtesy Kyle Stark. - - .. change:: - :tags: bug, firebird - :versions: 0.9.0 - :tickets: 2897 - - The firebird dialect will quote identifiers which begin with an - underscore. Courtesy Treeve Jelbert. - - .. change:: - :tags: bug, firebird - :versions: 0.9.0 - - Fixed bug in Firebird index reflection where the columns within the - index were not sorted correctly; they are now sorted - in order of RDB$FIELD_POSITION. - - .. change:: - :tags: bug, mssql, firebird - :versions: 0.9.0 - - The "asdecimal" flag used with the :class:`.Float` type will now - work with Firebird as well as the mssql+pyodbc dialects; previously the - decimal conversion was not occurring. - - .. change:: - :tags: bug, mssql, pymssql - :versions: 0.9.0 - - Added "Net-Lib error during Connection reset by peer" message - to the list of messages checked for "disconnect" within the - pymssql dialect. Courtesy John Anderson. - - .. change:: - :tags: bug, sql - :versions: 0.9.0 - :tickets: 2896 - - Fixed issue where a primary key column that has a Sequence on it, - yet the column is not the "auto increment" column, either because - it has a foreign key constraint or ``autoincrement=False`` set, - would attempt to fire the Sequence on INSERT for backends that don't - support sequences, when presented with an INSERT missing the primary - key value. This would take place on non-sequence backends like - SQLite, MySQL. - - .. change:: - :tags: bug, sql - :versions: 0.9.0 - :tickets: 2895 - - Fixed bug with :meth:`_expression.Insert.from_select` method where the order - of the given names would not be taken into account when generating - the INSERT statement, thus producing a mismatch versus the column - names in the given SELECT statement. Also noted that - :meth:`_expression.Insert.from_select` implies that Python-side insert defaults - cannot be used, since the statement has no VALUES clause. - - .. change:: - :tags: enhancement, sql - :versions: 0.9.0 - - The exception raised when a :class:`.BindParameter` is present - in a compiled statement without a value now includes the key name - of the bound parameter in the error message. - - .. change:: - :tags: bug, orm - :versions: 0.9.0 - :tickets: 2887 - - An adjustment to the :func:`.subqueryload` strategy which ensures that - the query runs after the loading process has begun; this is so that - the subqueryload takes precedence over other loaders that may be - hitting the same attribute due to other eager/noload situations - at the wrong time. - - .. change:: - :tags: bug, orm - :versions: 0.9.0 - :tickets: 2885 - - Fixed bug when using joined table inheritance from a table to a - select/alias on the base, where the PK columns were also not same - named; the persistence system would fail to copy primary key values - from the base table to the inherited table upon INSERT. - - .. change:: - :tags: bug, orm - :versions: 0.9.0 - :tickets: 2889 - - :func:`.composite` will raise an informative error message when the - columns/attribute (names) passed don't resolve to a Column or mapped - attribute (such as an erroneous tuple); previously raised an unbound - local. - - .. change:: - :tags: bug, declarative - :versions: 0.9.0 - :tickets: 2888 - - Error message when a string arg sent to :func:`_orm.relationship` which - doesn't resolve to a class or mapper has been corrected to work - the same way as when a non-string arg is received, which indicates - the name of the relationship which had the configurational error. - -.. changelog:: - :version: 0.8.4 - :released: December 8, 2013 - - .. change:: - :tags: bug, engine - :versions: 0.9.0 - :tickets: 2881 - - A DBAPI that raises an error on ``connect()`` which is not a subclass - of dbapi.Error (such as ``TypeError``, ``NotImplementedError``, etc.) - will propagate the exception unchanged. Previously, - the error handling specific to the ``connect()`` routine would both - inappropriately run the exception through the dialect's - :meth:`.Dialect.is_disconnect` routine as well as wrap it in - a :class:`sqlalchemy.exc.DBAPIError`. It is now propagated unchanged - in the same way as occurs within the execute process. - - .. change:: - :tags: bug, engine, pool - :versions: 0.9.0 - :tickets: 2880 - - The :class:`.QueuePool` has been enhanced to not block new connection - attempts when an existing connection attempt is blocking. Previously, - the production of new connections was serialized within the block - that monitored overflow; the overflow counter is now altered within - its own critical section outside of the connection process itself. - - .. change:: - :tags: bug, engine, pool - :versions: 0.9.0 - :tickets: 2522 - - Made a slight adjustment to the logic which waits for a pooled - connection to be available, such that for a connection pool - with no timeout specified, it will every half a second break out of - the wait to check for the so-called "abort" flag, which allows the - waiter to break out in case the whole connection pool was dumped; - normally the waiter should break out due to a notify_all() but it's - possible this notify_all() is missed in very slim cases. - This is an extension of logic first introduced in 0.8.0, and the - issue has only been observed occasionally in stress tests. - - .. change:: - :tags: bug, mssql - :versions: 0.9.0 - - Fixed bug introduced in 0.8.0 where the ``DROP INDEX`` - statement for an index in MSSQL would render incorrectly if the - index were in an alternate schema; the schemaname/tablename - would be reversed. The format has been also been revised to - match current MSSQL documentation. Courtesy Derek Harland. - - .. change:: - :tags: feature, sql - :tickets: 1443 - :versions: 0.9.0b1 - - Added support for "unique constraint" reflection, via the - :meth:`_reflection.Inspector.get_unique_constraints` method. - Thanks for Roman Podolyaka for the patch. - - .. change:: - :tags: bug, oracle - :tickets: 2864 - :versions: 0.9.0 - - Added ORA-02396 "maximum idle time" error code to list of - "is disconnect" codes with cx_oracle. - - .. change:: - :tags: bug, engine - :tickets: 2871 - :versions: 0.9.0 - - Fixed bug where SQL statement would be improperly ASCII-encoded - when a pre-DBAPI :class:`.StatementError` were raised within - :meth:`_engine.Connection.execute`, causing encoding errors for - non-ASCII statements. The stringification now remains within - Python unicode thus avoiding encoding errors. - - .. change:: - :tags: bug, oracle - :tickets: 2870 - :versions: 0.9.0 - - Fixed bug where Oracle ``VARCHAR`` types given with no length - (e.g. for a ``CAST`` or similar) would incorrectly render ``None CHAR`` - or similar. - - .. change:: - :tags: bug, ext - :tickets: 2869 - :versions: 0.9.0 - - Fixed bug which prevented the ``serializer`` extension from working - correctly with table or column names that contain non-ASCII - characters. - - .. change:: - :tags: bug, orm - :tickets: 2818 - :versions: 0.9.0 - - Fixed a regression introduced by :ticket:`2818` where the EXISTS - query being generated would produce a "columns being replaced" - warning for a statement with two same-named columns, - as the internal SELECT wouldn't have use_labels set. - - .. change:: - :tags: bug, postgresql - :tickets: 2855 - :versions: 0.9.0 - - Fixed bug where index reflection would mis-interpret indkey values - when using the pypostgresql adapter, which returns these values - as lists vs. psycopg2's return type of string. - -.. changelog:: - :version: 0.8.3 - :released: October 26, 2013 - - .. change:: - :tags: bug, oracle - :tickets: 2853 - :versions: 0.9.0b1 - - Fixed bug where Oracle table reflection using synonyms would fail - if the synonym and the table were in different remote schemas. - Patch to fix courtesy Kyle Derr. - - .. change:: - :tags: bug, sql - :tickets: 2849 - :versions: 0.9.0b1 - - Fixed bug where :func:`.type_coerce` would not interpret ORM - elements with a ``__clause_element__()`` method properly. - - .. change:: - :tags: bug, sql - :tickets: 2842 - :versions: 0.9.0b1 - - The :class:`.Enum` and :class:`.Boolean` types now bypass - any custom (e.g. TypeDecorator) type in use when producing the - CHECK constraint for the "non native" type. This so that the custom type - isn't involved in the expression within the CHECK, since this - expression is against the "impl" value and not the "decorated" value. - - .. change:: - :tags: bug, postgresql - :tickets: 2844 - :versions: 0.9.0b1 - - Removed a 128-character truncation from the reflection of the - server default for a column; this code was original from - PG system views which truncated the string for readability. - - .. change:: - :tags: bug, mysql - :tickets: 2721, 2839 - :versions: 0.9.0b1 - - The change in :ticket:`2721`, which is that the ``deferrable`` keyword - of :class:`_schema.ForeignKeyConstraint` is silently ignored on the MySQL - backend, will be reverted as of 0.9; this keyword will now render again, raising - errors on MySQL as it is not understood - the same behavior will also - apply to the ``initially`` keyword. In 0.8, the keywords will remain - ignored but a warning is emitted. Additionally, the ``match`` keyword - now raises a :exc:`.CompileError` on 0.9 and emits a warning on 0.8; - this keyword is not only silently ignored by MySQL but also breaks - the ON UPDATE/ON DELETE options. - - To use a :class:`_schema.ForeignKeyConstraint` - that does not render or renders differently on MySQL, use a custom - compilation option. An example of this usage has been added to the - documentation, see :ref:`mysql_foreign_keys`. - - .. change:: - :tags: bug, sql - :tickets: 2825 - :versions: 0.9.0b1 - - The ``.unique`` flag on :class:`.Index` could be produced as ``None`` - if it was generated from a :class:`_schema.Column` that didn't specify ``unique`` - (where it defaults to ``None``). The flag will now always be ``True`` or - ``False``. - - .. change:: - :tags: feature, orm - :tickets: 2836 - :versions: 0.9.0b1 - - Added new option to :func:`_orm.relationship` ``distinct_target_key``. - This enables the subquery eager loader strategy to apply a DISTINCT - to the innermost SELECT subquery, to assist in the case where - duplicate rows are generated by the innermost query which corresponds - to this relationship (there's not yet a general solution to the issue - of dupe rows within subquery eager loading, however, when joins outside - of the innermost subquery produce dupes). When the flag - is set to ``True``, the DISTINCT is rendered unconditionally, and when - it is set to ``None``, DISTINCT is rendered if the innermost relationship - targets columns that do not comprise a full primary key. - The option defaults to False in 0.8 (e.g. off by default in all cases), - None in 0.9 (e.g. automatic by default). Thanks to Alexander Koval - for help with this. - - .. seealso:: - - :ref:`change_2836` - - .. change:: - :tags: bug, mysql - :tickets: 2515 - :versions: 0.9.0b1 - - MySQL-connector dialect now allows options in the create_engine - query string to override those defaults set up in the connect, - including "buffered" and "raise_on_warnings". - - .. change:: - :tags: bug, postgresql - :tickets: 2742 - :versions: 0.9.0b1 - - Parenthesis will be applied to a compound SQL expression as - rendered in the column list of a CREATE INDEX statement. - - .. change:: - :tags: bug, sql - :tickets: 2742 - :versions: 0.9.0b1 - - Fixed bug in default compiler plus those of postgresql, mysql, and - mssql to ensure that any literal SQL expression values are - rendered directly as literals, instead of as bound parameters, - within a CREATE INDEX statement. This also changes the rendering - scheme for other DDL such as constraints. - - .. change:: - :tags: bug, sql - :tickets: 2815 - :versions: 0.9.0b1 - - A :func:`_expression.select` that is made to refer to itself in its FROM clause, - typically via in-place mutation, will raise an informative error - message rather than causing a recursion overflow. - - .. change:: - :tags: bug, orm - :tickets: 2813 - :versions: 0.9.0b1 - - Fixed bug where using an annotation such as :func:`.remote` or - :func:`.foreign` on a :class:`_schema.Column` before association with a parent - :class:`_schema.Table` could produce issues related to the parent table not - rendering within joins, due to the inherent copy operation performed - by an annotation. - - .. change:: - :tags: bug, sql - :tickets: 2831 - - Non-working "schema" argument on :class:`_schema.ForeignKey` is deprecated; - raises a warning. Removed in 0.9. - - .. change:: - :tags: bug, postgresql - :tickets: 2819 - :versions: 0.9.0b1 - - Fixed bug where PostgreSQL version strings that had a prefix preceding - the words "PostgreSQL" or "EnterpriseDB" would not parse. - Courtesy Scott Schaefer. - - .. change:: - :tags: feature, engine - :tickets: 2821 - :versions: 0.9.0b1 - - ``repr()`` for the :class:`.URL` of an :class:`_engine.Engine` - will now conceal the password using asterisks. - Courtesy Gunnlaugur Þór Briem. - - .. change:: - :tags: bug, orm - :tickets: 2818 - :versions: 0.9.0b1 - - Fixed bug where :meth:`_query.Query.exists` failed to work correctly - without any WHERE criterion. Courtesy Vladimir Magamedov. - - .. change:: - :tags: bug, sql - :tickets: 2811 - :versions: 0.9.0b1 - - Fixed bug where using the ``column_reflect`` event to change the ``.key`` - of the incoming :class:`_schema.Column` would prevent primary key constraints, - indexes, and foreign key constraints from being correctly reflected. - - .. change:: - :tags: feature - :versions: 0.9.0b1 - - Added a new flag ``system=True`` to :class:`_schema.Column`, which marks - the column as a "system" column which is automatically made present - by the database (such as PostgreSQL ``oid`` or ``xmin``). The - column will be omitted from the ``CREATE TABLE`` statement but will - otherwise be available for querying. In addition, the - :class:`.CreateColumn` construct can be applied to a custom - compilation rule which allows skipping of columns, by producing - a rule that returns ``None``. - - .. change:: - :tags: bug, orm - :tickets: 2779 - - Backported a change from 0.9 whereby the iteration of a hierarchy - of mappers used in polymorphic inheritance loads is sorted, - which allows the SELECT statements generated for polymorphic queries - to have deterministic rendering, which in turn helps with caching - schemes that cache on the SQL string itself. - - .. change:: - :tags: bug, orm - :tickets: 2794 - :versions: 0.9.0b1 - - Fixed a potential issue in an ordered sequence implementation used - by the ORM to iterate mapper hierarchies; under the Jython interpreter - this implementation wasn't ordered, even though cPython and PyPy - maintained ordering. - - .. change:: - :tags: bug, examples - :versions: 0.9.0b1 - - Added "autoincrement=False" to the history table created in the - versioning example, as this table shouldn't have autoinc on it - in any case, courtesy Patrick Schmid. - - .. change:: - :tags: bug, sql - :versions: 0.9.0b1 - - The :meth:`.ColumnOperators.notin_` operator added in 0.8 now properly - produces the negation of the expression "IN" returns - when used against an empty collection. - - .. change:: - :tags: feature, examples - :versions: 0.9.0b1 - - Improved the examples in ``examples/generic_associations``, including - that ``discriminator_on_association.py`` makes use of single table - inheritance do the work with the "discriminator". Also - added a true "generic foreign key" example, which works similarly - to other popular frameworks in that it uses an open-ended integer - to point to any other table, foregoing traditional referential - integrity. While we don't recommend this pattern, information wants - to be free. - - .. change:: - :tags: feature, orm, declarative - :versions: 0.9.0b1 - - Added a convenience class decorator :func:`.as_declarative`, is - a wrapper for :func:`.declarative_base` which allows an existing base - class to be applied using a nifty class-decorated approach. - - .. change:: - :tags: bug, orm - :tickets: 2786 - :versions: 0.9.0b1 - - Fixed bug in ORM-level event registration where the "raw" or - "propagate" flags could potentially be mis-configured in some - "unmapped base class" configurations. - - .. change:: - :tags: bug, orm - :tickets: 2778 - :versions: 0.9.0b1 - - A performance fix related to the usage of the :func:`.defer` option - when loading mapped entities. The function overhead of applying - a per-object deferred callable to an instance at load time was - significantly higher than that of just loading the data from the row - (note that ``defer()`` is meant to reduce DB/network overhead, not - necessarily function call count); the function call overhead is now - less than that of loading data from the column in all cases. There - is also a reduction in the number of "lazy callable" objects created - per load from N (total deferred values in the result) to 1 (total - number of deferred cols). - - .. change:: - :tags: bug, sqlite - :tickets: 2781 - :versions: 0.9.0b1 - - The newly added SQLite DATETIME arguments storage_format and - regexp apparently were not fully implemented correctly; while the - arguments were accepted, in practice they would have no effect; - this has been fixed. - - .. change:: - :tags: bug, sql, postgresql - :tickets: 2780 - :versions: 0.9.0b1 - - Fixed bug where the expression system relied upon the ``str()`` - form of a some expressions when referring to the ``.c`` collection - on a ``select()`` construct, but the ``str()`` form isn't available - since the element relies on dialect-specific compilation constructs, - notably the ``__getitem__()`` operator as used with a PostgreSQL - ``ARRAY`` element. The fix also adds a new exception class - :exc:`.UnsupportedCompilationError` which is raised in those cases - where a compiler is asked to compile something it doesn't know - how to. - - .. change:: - :tags: bug, engine, oracle - :tickets: 2776 - :versions: 0.9.0b1 - - Dialect.initialize() is not called a second time if an :class:`_engine.Engine` - is recreated, due to a disconnect error. This fixes a particular - issue in the Oracle 8 dialect, but in general the dialect.initialize() - phase should only be once per dialect. - - .. change:: - :tags: feature, sql - :tickets: 722 - - Added new method to the :func:`_expression.insert` construct - :meth:`_expression.Insert.from_select`. Given a list of columns and - a selectable, renders ``INSERT INTO (table) (columns) SELECT ..``. - - .. change:: - :tags: feature, sql - :versions: 0.9.0b1 - - The :func:`_expression.update`, :func:`_expression.insert`, and :func:`_expression.delete` constructs - will now interpret ORM entities as target tables to be operated upon, - e.g.:: - - from sqlalchemy import insert, update, delete - - ins = insert(SomeMappedClass).values(x=5) - - del_ = delete(SomeMappedClass).where(SomeMappedClass.id == 5) - - upd = update(SomeMappedClass).where(SomeMappedClass.id == 5).values(name="ed") - - .. change:: - :tags: bug, orm - :tickets: 2773 - :versions: 0.9.0b1 - - Fixed bug whereby attribute history functions would fail - when an object we moved from "persistent" to "pending" - using the :func:`.make_transient` function, for operations - involving collection-based backrefs. - - .. change:: - :tags: bug, engine, pool - :tickets: 2772 - :versions: 0.9.0b1 - - Fixed bug where :class:`.QueuePool` would lose the correct - checked out count if an existing pooled connection failed to reconnect - after an invalidate or recycle event. - -.. changelog:: - :version: 0.8.2 - :released: July 3, 2013 - - .. change:: - :tags: bug, mysql - :tickets: 2768 - :versions: 0.9.0b1 - - Fixed bug when using multi-table UPDATE where a supplemental - table is a SELECT with its own bound parameters, where the positioning - of the bound parameters would be reversed versus the statement - itself when using MySQL's special syntax. - - .. change:: - :tags: bug, sqlite - :tickets: 2764 - :versions: 0.9.0b1 - - Added :class:`sqlalchemy.types.BIGINT` to the list of type names that can be - reflected by the SQLite dialect; courtesy Russell Stuart. - - .. change:: - :tags: feature, orm, declarative - :tickets: 2761 - :versions: 0.9.0b1 - - ORM descriptors such as hybrid properties can now be referenced - by name in a string argument used with ``order_by``, - ``primaryjoin``, or similar in :func:`_orm.relationship`, - in addition to column-bound attributes. - - .. change:: - :tags: feature, firebird - :tickets: 2763 - :versions: 0.9.0b1 - - Added new flag ``retaining=True`` to the kinterbasdb and fdb dialects. - This controls the value of the ``retaining`` flag sent to the - ``commit()`` and ``rollback()`` methods of the DBAPI connection. - Due to historical concerns, this flag defaults to ``True`` in 0.8.2, - however in 0.9.0b1 this flag defaults to ``False``. - - .. change:: - :tags: requirements - :versions: 0.9.0b1 - - The Python `mock `_ library - is now required in order to run the unit test suite. While part - of the standard library as of Python 3.3, previous Python installations - will need to install this in order to run unit tests or to - use the ``sqlalchemy.testing`` package for external dialects. - - .. change:: - :tags: bug, orm - :tickets: 2750 - :versions: 0.9.0b1 - - A warning is emitted when trying to flush an object of an inherited - class where the polymorphic discriminator has been assigned - to a value that is invalid for the class. - - .. change:: - :tags: bug, postgresql - :tickets: 2740 - :versions: 0.9.0b1 - - The behavior of :func:`.extract` has been simplified on the - PostgreSQL dialect to no longer inject a hardcoded ``::timestamp`` - or similar cast into the given expression, as this interfered - with types such as timezone-aware datetimes, but also - does not appear to be at all necessary with modern versions - of psycopg2. - - - .. change:: - :tags: bug, firebird - :tickets: 2757 - :versions: 0.9.0b1 - - Type lookup when reflecting the Firebird types LONG and - INT64 has been fixed so that LONG is treated as INTEGER, - INT64 treated as BIGINT, unless the type has a "precision" - in which case it's treated as NUMERIC. Patch courtesy - Russell Stuart. - - .. change:: - :tags: bug, postgresql - :tickets: 2766 - :versions: 0.9.0b1 - - Fixed bug in HSTORE type where keys/values that contained - backslashed quotes would not be escaped correctly when - using the "non native" (i.e. non-psycopg2) means - of translating HSTORE data. Patch courtesy Ryan Kelly. - - .. change:: - :tags: bug, postgresql - :tickets: 2767 - :versions: 0.9.0b1 - - Fixed bug where the order of columns in a multi-column - PostgreSQL index would be reflected in the wrong order. - Courtesy Roman Podolyaka. - - .. change:: - :tags: bug, sql - :tickets: 2746, 2668 - :versions: 0.9.0b1 - - Multiple fixes to the correlation behavior of - :class:`_expression.Select` constructs, first introduced in 0.8.0: - - * To satisfy the use case where FROM entries should be - correlated outwards to a SELECT that encloses another, - which then encloses this one, correlation now works - across multiple levels when explicit correlation is - established via :meth:`_expression.Select.correlate`, provided - that the target select is somewhere along the chain - contained by a WHERE/ORDER BY/columns clause, not - just nested FROM clauses. This makes - :meth:`_expression.Select.correlate` act more compatibly to - that of 0.7 again while still maintaining the new - "smart" correlation. - - * When explicit correlation is not used, the usual - "implicit" correlation limits its behavior to just - the immediate enclosing SELECT, to maximize compatibility - with 0.7 applications, and also prevents correlation - across nested FROMs in this case, maintaining compatibility - with 0.8.0/0.8.1. - - * The :meth:`_expression.Select.correlate_except` method was not - preventing the given FROM clauses from correlation in - all cases, and also would cause FROM clauses to be incorrectly - omitted entirely (more like what 0.7 would do), - this has been fixed. - - * Calling `select.correlate_except(None)` will enter - all FROM clauses into correlation as would be expected. - - .. change:: - :tags: bug, ext - :versions: 0.9.0b1 - - Fixed bug whereby if a composite type were set up - with a function instead of a class, the mutable extension - would trip up when it tried to check that column - for being a :class:`.MutableComposite` (which it isn't). - Courtesy asldevi. - - .. change:: - :tags: feature, sql - :tickets: 2744, 2734 - - Provided a new attribute for :class:`.TypeDecorator` - called :attr:`.TypeDecorator.coerce_to_is_types`, - to make it easier to control how comparisons using - ``==`` or ``!=`` to ``None`` and boolean types goes - about producing an ``IS`` expression, or a plain - equality expression with a bound parameter. - - .. change:: - :tags: feature, postgresql - :versions: 0.9.0b1 - - Support for PostgreSQL 9.2 range types has been added. - Currently, no type translation is provided, so works - directly with strings or psycopg2 2.5 range extension types - at the moment. Patch courtesy Chris Withers. - - .. change:: - :tags: bug, examples - :versions: 0.9.0b1 - - Fixed an issue with the "versioning" recipe whereby a many-to-one - reference could produce a meaningless version for the target, - even though it was not changed, when backrefs were present. - Patch courtesy Matt Chisholm. - - .. change:: - :tags: feature, postgresql - :tickets: 2072 - :versions: 0.9.0b1 - - Added support for "AUTOCOMMIT" isolation when using the psycopg2 - DBAPI. The keyword is available via the ``isolation_level`` - execution option. Patch courtesy Roman Podolyaka. - - .. change:: - :tags: bug, orm - :tickets: 2759 - :versions: 0.9.0b1 - - Fixed bug in polymorphic SQL generation where multiple joined-inheritance - entities against the same base class joined to each other as well - would not track columns on the base table independently of each other if - the string of joins were more than two entities long. - - .. change:: - :tags: bug, engine - :versions: 0.9.0b1 - - Fixed bug where the ``reset_on_return`` argument to various :class:`_pool.Pool` - implementations would not be propagated when the pool was regenerated. - Courtesy Eevee. - - .. change:: - :tags: bug, orm - :tickets: 2754 - :versions: 0.9.0b1 - - Fixed bug where sending a composite attribute into :meth:`_query.Query.order_by` - would produce a parenthesized expression not accepted by some databases. - - .. change:: - :tags: bug, orm - :tickets: 2755 - :versions: 0.9.0b1 - - Fixed the interaction between composite attributes and - the :func:`.aliased` function. Previously, composite attributes - wouldn't work correctly in comparison operations when aliasing - was applied. - - .. change:: - :tags: bug, mysql - :tickets: 2715 - :versions: 0.9.0b1 - - Added another conditional to the ``mysql+gaerdbms`` dialect to - detect so-called "development" mode, where we should use the - ``rdbms_mysqldb`` DBAPI. Patch courtesy Brett Slatkin. - - .. change:: - :tags: feature, mysql - :tickets: 2704 - :versions: 0.9.0b1 - - The ``mysql_length`` parameter used with :class:`.Index` can now - be passed as a dictionary of column names/lengths, for use - with composite indexes. Big thanks to Roman Podolyaka for the - patch. - - .. change:: - :tags: bug, mssql - :tickets: 2747 - :versions: 0.9.0b1 - - When querying the information schema on SQL Server 2000, removed - a CAST call that was added in 0.8.1 to help with driver issues, - which apparently is not compatible on 2000. - The CAST remains in place for SQL Server 2005 and greater. - - .. change:: - :tags: bug, mysql - :tickets: 2721 - :versions: 0.9.0b1 - - The ``deferrable`` keyword argument on :class:`_schema.ForeignKey` and - :class:`_schema.ForeignKeyConstraint` will not render the ``DEFERRABLE`` keyword - on the MySQL dialect. For a long time we left this in place because - a non-deferrable foreign key would act very differently than a deferrable - one, but some environments just disable FKs on MySQL, so we'll be less - opinionated here. - - .. change:: - :tags: bug, ext, orm - :tickets: 2730 - :versions: 0.9.0b1 - - Fixed bug where :class:`.MutableDict` didn't report a change event - when ``clear()`` was called. - - .. change:: - :tags: bug, sql - :tickets: 2738 - :versions: 0.9.0b1 - - Fixed bug whereby joining a select() of a table "A" with multiple - foreign key paths to a table "B", to that table "B", would fail - to produce the "ambiguous join condition" error that would be - reported if you join table "A" directly to "B"; it would instead - produce a join condition with multiple criteria. - - .. change:: - :tags: bug, sql, reflection - :tickets: 2728 - :versions: 0.9.0b1 - - Fixed bug whereby using :meth:`_schema.MetaData.reflect` across a remote - schema as well as a local schema could produce wrong results - in the case where both schemas had a table of the same name. - - .. change:: - :tags: bug, sql - :tickets: 2726 - :versions: 0.9.0b1 - - Removed the "not implemented" ``__iter__()`` call from the base - :class:`.ColumnOperators` class, while this was introduced - in 0.8.0 to prevent an endless, memory-growing loop when one also - implements a ``__getitem__()`` method on a custom - operator and then calls erroneously ``list()`` on that object, - it had the effect of causing column elements to report that they - were in fact iterable types which then throw an error when you try - to iterate. There's no real way to have both sides here so we - stick with Python best practices. Careful with implementing - ``__getitem__()`` on your custom operators! - - .. change:: - :tags: feature, orm - :tickets: 2736 - - Added a new method :meth:`_query.Query.select_entity_from` which - will in 0.9 replace part of the functionality of - :meth:`_query.Query.select_from`. In 0.8, the two methods perform - the same function, so that code can be migrated to use the - :meth:`_query.Query.select_entity_from` method as appropriate. - See the 0.9 migration guide for details. - - .. change:: - :tags: bug, orm - :tickets: 2737 - - Fixed a regression caused by :ticket:`2682` whereby the - evaluation invoked by :meth:`_query.Query.update` and :meth:`_query.Query.delete` - would hit upon unsupported ``True`` and ``False`` symbols - which now appear due to the usage of ``IS``. - - .. change:: - :tags: bug, postgresql - :tickets: 2735 - - Fixed the HSTORE type to correctly encode/decode for unicode. - This is always on, as the hstore is a textual type, and - matches the behavior of psycopg2 when using Python 3. - Courtesy Dmitry Mugtasimov. - - .. change:: - :tags: bug, examples - - Fixed a small bug in the dogpile example where the generation - of SQL cache keys wasn't applying deduping labels to the - statement the same way :class:`_query.Query` normally does. - - .. change:: - :tags: bug, engine, sybase - :tickets: 2732 - - Fixed a bug where the routine to detect the correct kwargs - being sent to :func:`_sa.create_engine` would fail in some cases, - such as with the Sybase dialect. - - .. change:: - :tags: bug, orm - :tickets: 2481 - - Fixed a regression from 0.7 caused by this ticket, which - made the check for recursion overflow in self-referential - eager joining too loose, missing a particular circumstance - where a subclass had lazy="joined" or "subquery" configured - and the load was a "with_polymorphic" against the base. - - .. change:: - :tags: bug, orm - :tickets: 2718 - - Fixed a regression from 0.7 where the contextmanager feature - of :meth:`.Session.begin_nested` would fail to correctly - roll back the transaction when a flush error occurred, instead - raising its own exception while leaving the session still - pending a rollback. - - .. change:: - :tags: bug, mysql - - Updated mysqlconnector dialect to check for disconnect based - on the apparent string message sent in the exception; tested - against mysqlconnector 1.0.9. - - .. change:: - :tags: bug, sql, mssql - :tickets: 2682 - - Regression from this ticket caused the unsupported keyword - "true" to render, added logic to convert this to 1/0 - for SQL server. - -.. changelog:: - :version: 0.8.1 - :released: April 27, 2013 - - .. change:: - :tags: bug, orm - :tickets: 2698 - - Fixes to the ``sqlalchemy.ext.serializer`` extension, including - that the "id" passed from the pickler is turned into a string - to prevent against bytes being parsed on Py3K, as well as that - ``relationship()`` and ``orm.join()`` constructs are now properly - serialized. - - .. change:: - :tags: bug, orm - :tickets: 2714 - - A significant improvement to the inner workings of query.join(), - such that the decisionmaking involved on how to join has been - dramatically simplified. New test cases now pass such as - multiple joins extending from the middle of an already complex - series of joins involving inheritance and such. Joining from - deeply nested subquery structures is still complicated and - not without caveats, but with these improvements the edge - cases are hopefully pushed even farther out to the edges. - - .. change:: - :tags: feature, orm - :tickets: 2673 - - Added a convenience method to Query that turns a query into an - EXISTS subquery of the form - ``EXISTS (SELECT 1 FROM ... WHERE ...)``. - - .. change:: - :tags: bug, orm - - Added a conditional to the unpickling process for ORM - mapped objects, such that if the reference to the object - were lost when the object was pickled, we don't - erroneously try to set up _sa_instance_state - fixes - a NoneType error. - - .. change:: - :tags: bug, postgresql - :tickets: 2712 - - Opened up the checking for "disconnect" with psycopg2/libpq - to check for all the various "disconnect" messages within - the full exception hierarchy. Specifically the - "closed the connection unexpectedly" message has now been - seen in at least three different exception types. - Courtesy Eli Collins. - - .. change:: - :tags: bug, sql, mysql - :tickets: 2682 - - Fully implemented the IS and IS NOT operators with - regards to the True/False constants. An expression like - ``col.is_(True)`` will now render ``col IS true`` - on the target platform, rather than converting the True/ - False constant to an integer bound parameter. - This allows the ``is_()`` operator to work on MySQL when - given True/False constants. - - .. change:: - :tags: bug, postgresql - :tickets: 2681 - - The operators for the PostgreSQL ARRAY type supports - input types of sets, generators, etc. even when - a dimension is not specified, by turning the given - iterable into a collection unconditionally. - - .. change:: - :tags: bug, mysql - - Fixes to support the latest cymysql DBAPI, courtesy - Hajime Nakagami. - - .. change:: - :tags: bug, mysql - :tickets: 2663 - - Improvements to the operation of the pymysql dialect on - Python 3, including some important decode/bytes steps. - Issues remain with BLOB types due to driver issues. - Courtesy Ben Trofatter. - - .. change:: - :tags: bug, orm - :tickets: 2710 - - Fixed bug where many-to-many relationship with uselist=False - would fail to delete the association row and raise an error - if the scalar attribute were set to None. This was a - regression introduced by the changes for :ticket:`2229`. - - .. change:: - :tags: bug, orm - :tickets: 2708 - - Improved the behavior of instance management regarding - the creation of strong references within the Session; - an object will no longer have an internal reference cycle - created if it's in the transient state or moves into the - detached state - the strong ref is created only when the - object is attached to a Session and is removed when the - object is detached. This makes it somewhat safer for an - object to have a `__del__()` method, even though this is - not recommended, as relationships with backrefs produce - cycles too. A warning has been added when a class with - a `__del__()` method is mapped. - - .. change:: - :tags: bug, sql - :tickets: 2702 - - A major fix to the way in which a select() object produces - labeled columns when apply_labels() is used; this mode - produces a SELECT where each column is labeled as in - _, to remove column name collisions - for a multiple table select. The fix is that if two labels - collide when combined with the table name, i.e. - "foo.bar_id" and "foo_bar.id", anonymous aliasing will be - applied to one of the dupes. This allows the ORM to handle - both columns independently; previously, 0.7 - would in some cases silently emit a second SELECT for the - column that was "duped", and in 0.8 an ambiguous column error - would be emitted. The "keys" applied to the .c. collection - of the select() will also be deduped, so that the "column - being replaced" warning will no longer emit for any select() - that specifies use_labels, though the dupe key will be given - an anonymous label which isn't generally user-friendly. - - .. change:: - :tags: bug, mysql - - Updated a regexp to correctly extract error code on - google app engine v1.7.5 and newer. Courtesy - Dan Ring. - - .. change:: - :tags: bug, examples - - Fixed a long-standing bug in the caching example, where - the limit/offset parameter values wouldn't be taken into - account when computing the cache key. The - _key_from_query() function has been simplified to work - directly from the final compiled statement in order to get - at both the full statement as well as the fully processed - parameter list. - - .. change:: - :tags: bug, mssql - :tickets: 2355 - - Part of a longer series of fixes needed for pyodbc+ - mssql, a CAST to NVARCHAR(max) has been added to the bound - parameter for the table name and schema name in all information schema - queries to avoid the issue of comparing NVARCHAR to NTEXT, - which seems to be rejected by the ODBC driver in some cases, - such as FreeTDS (0.91 only?) plus unicode bound parameters being passed. - The issue seems to be specific to the SQL Server information - schema tables and the workaround is harmless for those cases - where the problem doesn't exist in the first place. - - .. change:: - :tags: bug, sql - :tickets: 2691 - - Fixed bug where disconnect detect on error would - raise an attribute error if the error were being - raised after the Connection object had already - been closed. - - .. change:: - :tags: bug, sql - :tickets: 2703 - - Reworked internal exception raises that emit - a rollback() before re-raising, so that the stack - trace is preserved from sys.exc_info() before entering - the rollback. This so that the traceback is preserved - when using coroutine frameworks which may have switched - contexts before the rollback function returns. - - .. change:: - :tags: bug, orm - :tickets: 2697 - - Fixed bug whereby ORM would run the wrong kind of - query when refreshing an inheritance-mapped class - where the superclass was mapped to a non-Table - object, like a custom join() or a select(), - running a query that assumed a hierarchy that's - mapped to individual Table-per-class. - - .. change:: - :tags: bug, orm - - Fixed `__repr__()` on mapper property constructs - to work before the object is initialized, so - that Sphinx builds with recent Sphinx versions - can read them. - - .. change:: - :tags: bug, sql, postgresql - - The _Binary base type now converts values through - the bytes() callable when run on Python 3; in particular - psycopg2 2.5 with Python 3.3 seems to now be returning - the "memoryview" type, so this is converted to bytes - before return. - - .. change:: - :tags: bug, sql - :tickets: 2695 - - Improvements to Connection auto-invalidation - handling. If a non-disconnect error occurs, - but leads to a delayed disconnect error within error - handling (happens with MySQL), the disconnect condition - is detected. The Connection can now also be closed - when in an invalid state, meaning it will raise "closed" - on next usage, and additionally the "close with result" - feature will work even if the autorollback in an error - handling routine fails and regardless of whether the - condition is a disconnect or not. - - - .. change:: - :tags: bug, orm, declarative - :tickets: 2656 - - Fixed indirect regression regarding :func:`.has_inherited_table`, - where since it considers the current class' ``__table__``, was - sensitive to when it was called. This is 0.7's behavior also, - but in 0.7 things tended to "work out" within events like - ``__mapper_args__()``. :func:`.has_inherited_table` now only - considers superclasses, so should return the same answer - regarding the current class no matter when it's called - (obviously assuming the state of the superclass). - - .. change:: - :tags: bug, mssql - - Added support for additional "disconnect" messages - to the pymssql dialect. Courtesy John Anderson. - - .. change:: - :tags: feature, sql - - Loosened the check on dialect-specific argument names - passed to Table(); since we want to support external dialects - and also want to support args without a certain dialect - being installed, it only checks the format of the arg now, - rather than looking for that dialect in sqlalchemy.dialects. - - .. change:: - :tags: bug, sql - - Fixed bug whereby a DBAPI that can return "0" - for cursor.lastrowid would not function correctly - in conjunction with :attr:`_engine.ResultProxy.inserted_primary_key`. - - .. change:: - :tags: bug, mssql - :tickets: 2683 - - Fixed Py3K bug regarding "binary" types and - pymssql. Courtesy Marc Abramowitz. - - .. change:: - :tags: bug, postgresql - :tickets: 2680 - - Added missing HSTORE type to postgresql type names - so that the type can be reflected. - -.. changelog:: - :version: 0.8.0 - :released: March 9, 2013 - - .. note:: - - There are some new behavioral changes as of 0.8.0 - not present in 0.8.0b2. They are present in the - migration document as follows: - - * :ref:`legacy_is_orphan_addition` - - * :ref:`metadata_create_drop_tables` - - * :ref:`correlation_context_specific` - - .. change:: - :tags: feature, orm - :tickets: 2675 - - A meaningful :attr:`.QueryableAttribute.info` attribute is - added, which proxies down to the ``.info`` attribute on either - the :class:`_schema.Column` object if directly present, or - the :class:`.MapperProperty` otherwise. The full behavior - is documented and ensured by tests to remain stable. - - .. change:: - :tags: bug, sql - :tickets: 2668 - - The behavior of SELECT correlation has been improved such that - the :meth:`_expression.Select.correlate` and :meth:`_expression.Select.correlate_except` - methods, as well as their ORM analogues, will still retain - "auto-correlation" behavior in that the FROM clause is modified - only if the output would be legal SQL; that is, the FROM clause - is left intact if the correlated SELECT is not used in the context - of an enclosing SELECT inside of the WHERE, columns, or HAVING clause. - The two methods now only specify conditions to the default - "auto correlation", rather than absolute FROM lists. - - .. change:: - :tags: feature, mysql - - New dialect for CyMySQL added, courtesy Hajime Nakagami. - - .. change:: - :tags: bug, orm - :tickets: 2674 - - Improved checking for an existing backref name conflict during - mapper configuration; will now test for name conflicts on - superclasses and subclasses, in addition to the current mapper, - as these conflicts break things just as much. This is new for - 0.8, but see below for a warning that will also be triggered - in 0.7.11. - - .. change:: - :tags: bug, orm - :tickets: 2674 - - Improved the error message emitted when a "backref loop" is detected, - that is when an attribute event triggers a bidirectional - assignment between two other attributes with no end. - This condition can occur not just when an object of the wrong - type is assigned, but also when an attribute is mis-configured - to backref into an existing backref pair. Also in 0.7.11. - - .. change:: - :tags: bug, orm - :tickets: 2674 - - A warning is emitted when a MapperProperty is assigned to a mapper - that replaces an existing property, if the properties in question - aren't plain column-based properties. Replacement of relationship - properties is rarely (ever?) what is intended and usually refers to a - mapper mis-configuration. Also in 0.7.11. - - .. change:: - :tags: feature, orm - - Can set/change the "cascade" attribute on a :func:`_orm.relationship` - construct after it's been constructed already. This is not - a pattern for normal use but we like to change the setting - for demonstration purposes in tutorials. - - .. change:: - :tags: bug, schema - :tickets: 2664 - - :meth:`_schema.MetaData.create_all` and :meth:`_schema.MetaData.drop_all` will - now accommodate an empty list as an instruction to not create/drop - any items, rather than ignoring the collection. - - - .. change:: - :tags: bug, tests - :tickets: 2669 - - Fixed an import of "logging" in test_execute which was not - working on some linux platforms. Also in 0.7.11. - - .. change:: - :tags: bug, orm - :tickets: 2662 - - A clear error message is emitted if an event handler - attempts to emit SQL on a Session within the after_commit() - handler, where there is not a viable transaction in progress. - - .. change:: - :tags: bug, orm - :tickets: 2665 - - Detection of a primary key change within the process - of cascading a natural primary key update will succeed - even if the key is composite and only some of the - attributes have changed. - - .. change:: - :tags: feature, orm - :tickets: 2658 - - Added new helper function :func:`.was_deleted`, returns True - if the given object was the subject of a :meth:`.Session.delete` - operation. - - .. change:: - :tags: bug, orm - :tickets: 2658 - - An object that's deleted from a session will be de-associated with - that session fully after the transaction is committed, that is - the :func:`.object_session` function will return None. - - .. change:: - :tags: bug, oracle - - The cx_oracle dialect will no longer run the bind parameter names - through ``encode()``, as this is not valid on Python 3, and prevented - statements from functioning correctly on Python 3. We now - encode only if ``supports_unicode_binds`` is False, which is not - the case for cx_oracle when at least version 5 of cx_oracle is used. - - .. change:: - :tags: bug, orm - :tickets: 2661 - - Fixed bug whereby :meth:`_query.Query.yield_per` would set the execution - options incorrectly, thereby breaking subsequent usage of the - :meth:`_query.Query.execution_options` method. Courtesy Ryan Kelly. - - .. change:: - :tags: bug, orm - :tickets: 1768 - - Fixed the consideration of the ``between()`` operator - so that it works correctly with the new relationship local/remote - system. - - .. change:: - :tags: bug, sql - :tickets: 2660, 1768 - - Fixed a bug regarding column annotations which in particular - could impact some usages of the new :func:`_orm.remote` and - :func:`_orm.local` annotation functions, where annotations - could be lost when the column were used in a subsequent - expression. - - .. change:: - :tags: bug, mysql, gae - :tickets: 2649 - - Added a conditional import to the ``gaerdbms`` dialect which attempts - to import rdbms_apiproxy vs. rdbms_googleapi to work - on both dev and production platforms. Also now honors the - ``instance`` attribute. Courtesy Sean Lynch. - Also in 0.7.10. - - .. change:: - :tags: bug, sql - :tickets: 2496 - - The :meth:`.ColumnOperators.in_` operator will now coerce - values of ``None`` to :func:`.null`. - - .. change:: - :tags: feature, sql - :tickets: 2657 - - Added a new argument to :class:`.Enum` and its base - :class:`.SchemaType` ``inherit_schema``. When set to ``True``, - the type will set its ``schema`` attribute of that of the - :class:`_schema.Table` to which it is associated. This also occurs - during a :meth:`_schema.Table.tometadata` operation; the :class:`.SchemaType` - is now copied in all cases when :meth:`_schema.Table.tometadata` happens, - and if ``inherit_schema=True``, the type will take on the new - schema name passed to the method. The ``schema`` is important - when used with the PostgreSQL backend, as the type results in - a ``CREATE TYPE`` statement. - - .. change:: - :tags: feature, postgresql - - Added :meth:`.postgresql.ARRAY.Comparator.any` and - :meth:`.postgresql.ARRAY.Comparator.all` - methods, as well as standalone expression constructs. Big thanks - to Audrius Kažukauskas for the terrific work here. - - .. change:: - :tags: sql, bug - :tickets: 2643 - - Fixed bug where :meth:`_schema.Table.tometadata` would fail if a - :class:`_schema.Column` had both a foreign key as well as an - alternate ".key" name for the column. Also in 0.7.10. - - .. change:: - :tags: sql, bug - :tickets: 2629 - - insert().returning() raises an informative CompileError if attempted - to compile on a dialect that doesn't support RETURNING. - - .. change:: - :tags: orm, bug - :tickets: 2655 - - the consideration of a pending object as - an "orphan" has been modified to more closely match the - behavior as that of persistent objects, which is that the object - is expunged from the :class:`.Session` as soon as it is - de-associated from any of its orphan-enabled parents. Previously, - the pending object would be expunged only if de-associated - from all of its orphan-enabled parents. The new flag ``legacy_is_orphan`` - is added to :class:`_orm.Mapper` which re-establishes the - legacy behavior. - - See the change note and example case at :ref:`legacy_is_orphan_addition` - for a detailed discussion of this change. - - .. change:: - :tags: orm, bug - :tickets: 2653 - - Fixed the (most likely never used) "@collection.link" collection - method, which fires off each time the collection is associated - or de-associated with a mapped object - the decorator - was not tested or functional. The decorator method - is now named :meth:`.collection.linker` though the name "link" - remains for backwards compatibility. Courtesy Luca Wehrstedt. - - .. change:: - :tags: orm, bug - :tickets: 2654 - - Made some fixes to the system of producing custom instrumented - collections, mainly that the usage of the @collection decorators - will now honor the __mro__ of the given class, applying the - logic of the sub-most classes' version of a particular collection - method. Previously, it wasn't predictable when subclassing - an existing instrumented class such as :class:`.MappedCollection` - whether or not custom methods would resolve correctly. - - .. change:: - :tags: orm, removed - - The undocumented (and hopefully unused) system of producing - custom collections using an ``__instrumentation__`` datastructure - associated with the collection has been removed, as this was a complex - and untested feature which was also essentially redundant versus the - decorator approach. Other internal simplifications to the - orm.collections module have been made as well. - - .. change:: - :tags: mssql, feature - - Added ``mssql_include`` and ``mssql_clustered`` options to - :class:`.Index`, renders the ``INCLUDE`` and ``CLUSTERED`` keywords, - respectively. Courtesy Derek Harland. - - .. change:: - :tags: sql, feature - :tickets: 695 - - :class:`.Index` now supports arbitrary SQL expressions and/or - functions, in addition to straight columns. Common modifiers - include using ``somecolumn.desc()`` for a descending index and - ``func.lower(somecolumn)`` for a case-insensitive index, depending on the - capabilities of the target backend. - - .. change:: - :tags: mssql, bug - :tickets: 2638 - - Added a py3K conditional around unnecessary .decode() - call in mssql information schema, fixes reflection - in Py3K. Also in 0.7.10. - - .. change:: - :tags: orm, bug - :tickets: 2650 - - Fixed potential memory leak which could occur if an - arbitrary number of :class:`.sessionmaker` objects - were created. The anonymous subclass created by - the sessionmaker, when dereferenced, would not be garbage - collected due to remaining class-level references from the - event package. This issue also applies to any custom system - that made use of ad-hoc subclasses in conjunction with - an event dispatcher. Also in 0.7.10. - - .. change:: - :tags: mssql, bug - - Fixed a regression whereby the "collation" parameter - of the character types CHAR, NCHAR, etc. stopped working, - as "collation" is now supported by the base string types. - The TEXT, NCHAR, CHAR, VARCHAR types within the - MSSQL dialect are now synonyms for the base types. - - .. change:: - :tags: mssql, feature - :tickets: 2644 - - DDL for IDENTITY columns is now supported on - non-primary key columns, by establishing a - :class:`.Sequence` construct on any - integer column. Courtesy Derek Harland. - - .. change:: - :tags: examples, bug - - Fixed a regression in the examples/dogpile_caching example - which was due to the change in :ticket:`2614`. - - .. change:: - :tags: orm, bug - :tickets: 2640 - - :meth:`_query.Query.merge_result` can now load rows from an outer join - where an entity may be ``None`` without throwing an error. - Also in 0.7.10. - - .. change:: - :tags: sql, bug - :tickets: 2648 - - Tweaked the "REQUIRED" symbol used by the compiler to identify - INSERT/UPDATE bound parameters that need to be passed, so that - it's more easily identifiable when writing custom bind-handling - code. - - .. change:: - :tags: postgresql, bug - - Fixed bug in :class:`~sqlalchemy.dialects.postgresql.array()` construct whereby using it - inside of an :func:`_expression.insert` construct would produce an - error regarding a parameter issue in the ``self_group()`` method. - - .. change:: - :tags: orm, feature - - Extended the :doc:`/core/inspection` system so that all Python descriptors - associated with the ORM or its extensions can be retrieved. - This fulfills the common request of being able to inspect - all :class:`.QueryableAttribute` descriptors in addition to - extension types such as :class:`.hybrid_property` and - :class:`.AssociationProxy`. See :attr:`_orm.Mapper.all_orm_descriptors`. - - .. change:: - :tags: mysql, feature - - GAE dialect now accepts username/password arguments in the URL, - courtesy Owen Nelson. - - .. change:: - :tags: mysql, bug - - GAE dialect won't fail on None match if the error code can't be extracted - from the exception throw; courtesy Owen Nelson. - - .. change:: - :tags: orm, bug - :tickets: 2637 - - Fixes to the "dynamic" loader on :func:`_orm.relationship`, includes - that backrefs will work properly even when autoflush is disabled, - history events are more accurate in scenarios where multiple add/remove - of the same object occurs. - -.. changelog:: - :version: 0.8.0b2 - :released: December 14, 2012 - - .. change:: - :tags: orm, bug - :tickets: 2635 - - The :meth:`_query.Query.select_from` method can now be used with a - :func:`.aliased` construct without it interfering with the entities - being selected. Basically, a statement like this:: - - ua = aliased(User) - session.query(User.name).select_from(ua).join(User, User.name > ua.name) - - Will maintain the columns clause of the SELECT as coming from the - unaliased "user", as specified; the select_from only takes place in the - FROM clause: - - .. sourcecode:: sql - - SELECT users.name AS users_name FROM users AS users_1 - JOIN users ON users.name < users_1.name - - Note that this behavior is in contrast - to the original, older use case for :meth:`_query.Query.select_from`, which is that - of restating the mapped entity in terms of a different selectable:: - - session.query(User.name).select_from(user_table.select().where(user_table.c.id > 5)) - - Which produces: - - .. sourcecode:: sql - - SELECT anon_1.name AS anon_1_name FROM (SELECT users.id AS id, - users.name AS name FROM users WHERE users.id > :id_1) AS anon_1 - - It was the "aliasing" behavior of the latter use case that was - getting in the way of the former use case. The method now - specifically considers a SQL expression like - :func:`_expression.select` or :func:`_expression.alias` - separately from a mapped entity like a :func:`.aliased` - construct. - - .. change:: - :tags: sql, bug - :tickets: 2633 - - Fixed a regression caused by :ticket:`2410` whereby a - :class:`.CheckConstraint` would apply itself back to the - original table during a :meth:`_schema.Table.tometadata` operation, as - it would parse the SQL expression for a parent table. The - operation now copies the given expression to correspond to the - new table. - - .. change:: - :tags: oracle, bug - :tickets: 2619 - - Fixed table reflection for Oracle when accessing a synonym that refers - to a DBLINK remote database; while the syntax has been present in the - Oracle dialect for some time, up until now it has never been tested. - The syntax has been tested against a sample database linking to itself, - however there's still some uncertainty as to what should be used for the - "owner" when querying the remote database for table information. - Currently, the value of "username" from user_db_links is used to - match the "owner". - - .. change:: - :tags: orm, feature - :tickets: 2601 - - Added :meth:`.KeyedTuple._asdict` and :attr:`.KeyedTuple._fields` - to the :class:`.KeyedTuple` class to provide some degree of compatibility - with the Python standard library ``collections.namedtuple()``. - - .. change:: - :tags: sql, bug - :tickets: 2610 - - Fixed bug whereby using a label_length on dialect that was smaller - than the size of actual column identifiers would fail to render - the columns correctly in a SELECT statement. - - .. change:: - :tags: sql, feature - :tickets: 2623 - - The :class:`_expression.Insert` construct now supports multi-valued inserts, - that is, an INSERT that renders like - "INSERT INTO table VALUES (...), (...), ...". - Supported by PostgreSQL, SQLite, and MySQL. - Big thanks to Idan Kamara for doing the legwork on this one. - - .. seealso:: - - :ref:`feature_2623` - - .. change:: - :tags: oracle, bug - :tickets: 2620 - - The Oracle LONG type, while an unbounded text type, does not appear - to use the cx_Oracle.LOB type when result rows are returned, - so the dialect has been repaired to exclude LONG from - having cx_Oracle.LOB filtering applied. Also in 0.7.10. - - .. change:: - :tags: oracle, bug - :tickets: 2611 - - Repaired the usage of ``.prepare()`` in conjunction with - cx_Oracle so that a return value of ``False`` will result - in no call to ``connection.commit()``, hence avoiding - "no transaction" errors. Two-phase transactions have - now been shown to work in a rudimental fashion with - SQLAlchemy and cx_oracle, however are subject to caveats - observed with the driver; check the documentation - for details. Also in 0.7.10. - - .. change:: - :tags: sql, bug - :tickets: 2618 - - The :class:`~sqlalchemy.types.DECIMAL` type now honors the "precision" and - "scale" arguments when rendering DDL. - - .. change:: - :tags: orm, bug - :tickets: 2624 - - The :class:`.MutableComposite` type did not allow for the - :meth:`.MutableBase.coerce` method to be used, even though - the code seemed to indicate this intent, so this now works - and a brief example is added. As a side-effect, - the mechanics of this event handler have been changed so that - new :class:`.MutableComposite` types no longer add per-type - global event handlers. Also in 0.7.10. - - .. change:: - :tags: sql, bug - :tickets: 2621 - - Made an adjustment to the "boolean", (i.e. ``__nonzero__``) - evaluation of binary expressions, i.e. ``x1 == x2``, such - that the "auto-grouping" applied by :class:`.BinaryExpression` - in some cases won't get in the way of this comparison. - Previously, an expression like:: - - expr1 = mycolumn > 2 - bool(expr1 == expr1) - - Would evaluate as ``False``, even though this is an identity - comparison, because ``mycolumn > 2`` would be "grouped" before - being placed into the :class:`.BinaryExpression`, thus changing - its identity. :class:`.BinaryExpression` now keeps track - of the "original" objects passed in. - Additionally the ``__nonzero__`` method now only returns if - the operator is ``==`` or ``!=`` - all others raise ``TypeError``. - - .. change:: - :tags: firebird, bug - :tickets: 2622 - - Added missing import for "fdb" to the experimental - "firebird+fdb" dialect. - - .. change:: - :tags: orm, feature - - Allow synonyms to be used when defining primary and secondary - joins for relationships. - - .. change:: - :tags: orm, bug - :tickets: 2614 - - A second overhaul of aliasing/internal pathing mechanics - now allows two subclasses to have different relationships - of the same name, supported with subquery or joined eager - loading on both simultaneously when a full polymorphic - load is used. - - .. change:: - :tags: orm, bug - :tickets: 2617 - - Fixed bug whereby a multi-hop subqueryload within - a particular with_polymorphic load would produce a KeyError. - Takes advantage of the same internal pathing overhaul - as :ticket:`2614`. - - .. change:: - :tags: sql, bug - - Fixed a gotcha where inadvertently calling list() on a - :class:`_expression.ColumnElement` would go into an endless loop, if - :meth:`.ColumnOperators.__getitem__` were implemented. - A new NotImplementedError is emitted via ``__iter__()``. - - .. change:: - :tags: orm, extensions, feature - - The :mod:`sqlalchemy.ext.mutable` extension now includes the - example :class:`.MutableDict` class as part of the extension. - - .. change:: - :tags: postgresql, feature - :tickets: 2606 - - :class:`.HSTORE` is now available in the PostgreSQL dialect. - Will also use psycopg2's extensions if available. Courtesy - Audrius Kažukauskas. - - .. change:: - :tags: sybase, feature - :tickets: 1753 - - Reflection support has been added to the Sybase dialect. - Big thanks to Ben Trofatter for all the work developing and - testing this. - - .. change:: - :tags: engine, feature - - The :meth:`_engine.Connection.connect` and :meth:`_engine.Connection.contextual_connect` - methods now return a "branched" version so that the :meth:`_engine.Connection.close` - method can be called on the returned connection without affecting the - original. Allows symmetry when using :class:`_engine.Engine` and - :class:`_engine.Connection` objects as context managers:: - - with conn.connect() as c: # leaves the Connection open - c.execute("...") - - with engine.connect() as c: # closes the Connection - c.execute("...") - - .. change:: - :tags: engine - - The "reflect=True" argument to :class:`~sqlalchemy.schema.MetaData` is deprecated. - Please use the :meth:`_schema.MetaData.reflect` method. - - .. change:: - :tags: sql, bug - :tickets: 2603 - - Fixed bug in type_coerce() whereby typing information - could be lost if the statement were used as a subquery - inside of another statement, as well as other similar - situations. Among other things, would cause - typing information to be lost when the Oracle/mssql dialects - would apply limit/offset wrappings. - - .. change:: - :tags: orm, bug - :tickets: 2602 - - Fixed regression where query.update() would produce - an error if an object matched by the "fetch" - synchronization strategy wasn't locally present. - Courtesy Scott Torborg. - - .. change:: - :tags: sql, bug - :tickets: 2597 - - Fixed bug whereby the ".key" of a Column wasn't being - used when producing a "proxy" of the column against - a selectable. This probably didn't occur in 0.7 - since 0.7 doesn't respect the ".key" in a wider - range of scenarios. - - .. change:: - :tags: mssql, feature - :tickets: 2600 - - Support for reflection of the "name" of primary key - constraints added, courtesy Dave Moore. - - .. change:: - :tags: informix - - Some cruft regarding informix transaction handling has been - removed, including a feature that would skip calling - commit()/rollback() as well as some hardcoded isolation level - assumptions on begin().. The status of this dialect is not - well understood as we don't have any users working with it, - nor any access to an Informix database. If someone with - access to Informix wants to help test this dialect, please - let us know. - - .. change:: - :tags: pool, feature - - The :class:`_pool.Pool` will now log all connection.close() - operations equally, including closes which occur for - invalidated connections, detached connections, and connections - beyond the pool capacity. - - .. change:: - :tags: pool, feature - :tickets: 2611 - - The :class:`_pool.Pool` now consults the :class:`.Dialect` for - functionality regarding how the connection should be - "auto rolled back", as well as closed. This grants more - control of transaction scope to the dialect, so that we - will be better able to implement transactional workarounds - like those potentially needed for pysqlite and cx_oracle. - - .. change:: - :tags: pool, feature - - Added new :meth:`_events.PoolEvents.reset` hook to capture - the event before a connection is auto-rolled back, upon - return to the pool. Together with - :meth:`_events.ConnectionEvents.rollback` this allows all rollback - events to be intercepted. - -.. changelog:: - :version: 0.8.0b1 - :released: October 30, 2012 - - .. change:: - :tags: sql, bug - :tickets: 2593 - - Fixed bug where keyword arguments passed to - :meth:`.Compiler.process` wouldn't get propagated - to the column expressions present in the columns - clause of a SELECT statement. In particular this would - come up when used by custom compilation schemes that - relied upon special flags. - - .. change:: - :tags: sql, feature - - Added a new method :meth:`_engine.Engine.execution_options` - to :class:`_engine.Engine`. This method works similarly to - :meth:`_engine.Connection.execution_options` in that it creates - a copy of the parent object which will refer to the new - set of options. The method can be used to build - sharding schemes where each engine shares the same - underlying pool of connections. The method - has been tested against the horizontal shard - recipe in the ORM as well. - - .. seealso:: - - :meth:`_engine.Engine.execution_options` - - .. change:: - :tags: sql, orm, bug - :tickets: 2595 - - The auto-correlation feature of :func:`_expression.select`, and - by proxy that of :class:`_query.Query`, will not - take effect for a SELECT statement that is being - rendered directly in the FROM list of the enclosing - SELECT. Correlation in SQL only applies to column - expressions such as those in the WHERE, ORDER BY, - columns clause. - - .. change:: - :tags: sqlite - :changeset: c3addcc9ffad - - Added :class:`_types.NCHAR`, :class:`_types.NVARCHAR` - to the SQLite dialect's list of recognized type names - for reflection. SQLite returns the name given - to a type as the name returned. - - .. change:: - :tags: examples - :tickets: 2589 - - The Beaker caching example has been converted - to use `dogpile.cache `_. - This is a new caching library written by the same - creator of Beaker's caching internals, and represents a - vastly improved, simplified, and modernized system of caching. - - .. seealso:: - - :ref:`examples_caching` - - .. change:: - :tags: general - :tickets: - - SQLAlchemy 0.8 now targets Python 2.5 and - above. Python 2.4 is no longer supported. - - .. change:: - :tags: removed, general - :tickets: 2433 - - The "sqlalchemy.exceptions" - synonym for "sqlalchemy.exc" is removed - fully. - - .. change:: - :tags: removed, orm - :tickets: 2442 - - The legacy "mutable" system of the - ORM, including the MutableType class as well - as the mutable=True flag on PickleType - and postgresql.ARRAY has been removed. - In-place mutations are detected by the ORM - using the sqlalchemy.ext.mutable extension, - introduced in 0.7. The removal of MutableType - and associated constructs removes a great - deal of complexity from SQLAlchemy's internals. - The approach performed poorly as it would incur - a scan of the full contents of the Session - when in use. - - .. change:: - :tags: orm, moved - :tickets: - - The InstrumentationManager interface - and the entire related system of alternate - class implementation is now moved out - to sqlalchemy.ext.instrumentation. This is - a seldom used system that adds significant - complexity and overhead to the mechanics of - class instrumentation. The new architecture - allows it to remain unused until - InstrumentationManager is actually imported, - at which point it is bootstrapped into - the core. - - .. change:: - :tags: orm, feature - :tickets: 1401 - - Major rewrite of relationship() - internals now allow join conditions which - include columns pointing to themselves - within composite foreign keys. A new - API for very specialized primaryjoin conditions - is added, allowing conditions based on - SQL functions, CAST, etc. to be handled - by placing the annotation functions - remote() and foreign() inline within the - expression when necessary. Previous recipes - using the semi-private _local_remote_pairs - approach can be upgraded to this new - approach. - - .. seealso:: - - :ref:`feature_relationship_08` - - .. change:: - :tags: orm, bug - :tickets: 2527 - - ORM will perform extra effort to determine - that an FK dependency between two tables is - not significant during flush if the tables - are related via joined inheritance and the FK - dependency is not part of the inherit_condition, - saves the user a use_alter directive. - - .. change:: - :tags: orm, feature - :tickets: 2333 - - New standalone function with_polymorphic() - provides the functionality of query.with_polymorphic() - in a standalone form. It can be applied to any - entity within a query, including as the target - of a join in place of the "of_type()" modifier. - - .. change:: - :tags: orm, feature - :tickets: 1106, 2438 - - The of_type() construct on attributes - now accepts aliased() class constructs as well - as with_polymorphic constructs, and works with - query.join(), any(), has(), and also - eager loaders subqueryload(), joinedload(), - contains_eager() - - .. change:: - :tags: orm, feature - :tickets: 2585 - - Improvements to event listening for - mapped classes allows that unmapped classes - can be specified for instance- and mapper-events. - The established events will be automatically - set up on subclasses of that class when the - propagate=True flag is passed, and the - events will be set up for that class itself - if and when it is ultimately mapped. - - .. change:: - :tags: orm, bug - :tickets: 2590 - - The instrumentation events class_instrument(), - class_uninstrument(), and attribute_instrument() - will now fire off only for descendant classes - of the class assigned to listen(). Previously, - an event listener would be assigned to listen - for all classes in all cases regardless of the - "target" argument passed. - - .. change:: - :tags: orm, bug - :tickets: 1900 - - with_polymorphic() produces JOINs - in the correct order and with correct inheriting - tables in the case of sending multi-level - subclasses in an arbitrary order or with - intermediary classes missing. - - .. change:: - :tags: orm, feature - :tickets: 2485 - - The "deferred declarative - reflection" system has been moved into the - declarative extension itself, using the - new DeferredReflection class. This - class is now tested with both single - and joined table inheritance use cases. - - .. change:: - :tags: orm, feature - :tickets: 2208 - - Added new core function "inspect()", - which serves as a generic gateway to - introspection into mappers, objects, - others. The Mapper and InstanceState - objects have been enhanced with a public - API that allows inspection of mapped - attributes, including filters for column-bound - or relationship-bound properties, inspection - of current object state, history of - attributes, etc. - - .. change:: - :tags: orm, feature - :tickets: 2452 - - Calling rollback() within a - session.begin_nested() will now only expire - those objects that had net changes within the - scope of that transaction, that is objects which - were dirty or were modified on a flush. This - allows the typical use case for begin_nested(), - that of altering a small subset of objects, to - leave in place the data from the larger enclosing - set of objects that weren't modified in - that sub-transaction. - - .. change:: - :tags: orm, feature - :tickets: 2372 - - Added utility feature - Session.enable_relationship_loading(), - supersedes relationship.load_on_pending. - Both features should be avoided, however. - - .. change:: - :tags: orm, feature - :tickets: - - Added support for .info dictionary argument to - column_property(), relationship(), composite(). - All MapperProperty classes have an auto-creating .info - dict available overall. - - .. change:: - :tags: orm, feature - :tickets: 2229 - - Adding/removing None from a mapped collection - now generates attribute events. Previously, a None - append would be ignored in some cases. Related - to. - - .. change:: - :tags: orm, feature - :tickets: 2229 - - The presence of None in a mapped collection - now raises an error during flush. Previously, - None values in collections would be silently ignored. - - .. change:: - :tags: orm, feature - :tickets: - - The Query.update() method is now - more lenient as to the table - being updated. Plain Table objects are better - supported now, and additional a joined-inheritance - subclass may be used with update(); the subclass - table will be the target of the update, - and if the parent table is referenced in the - WHERE clause, the compiler will call upon - UPDATE..FROM syntax as allowed by the dialect - to satisfy the WHERE clause. MySQL's multi-table - update feature is also supported if columns - are specified by object in the "values" dictionary. - PG's DELETE..USING is also not available - in Core yet. - - .. change:: - :tags: orm, feature - :tickets: - - New session events after_transaction_create - and after_transaction_end - allows tracking of new SessionTransaction objects. - If the object is inspected, can be used to determine - when a session first becomes active and when - it deactivates. - - .. change:: - :tags: orm, feature - :tickets: 2592 - - The Query can now load entity/scalar-mixed - "tuple" rows that contain - types which aren't hashable, by setting the flag - "hashable=False" on the corresponding TypeEngine object - in use. Custom types that return unhashable types - (typically lists) can set this flag to False. - - .. change:: - :tags: orm, bug - :tickets: 2481 - - Improvements to joined/subquery eager - loading dealing with chains of subclass entities - sharing a common base, with no specific "join depth" - provided. Will chain out to - each subclass mapper individually before detecting - a "cycle", rather than considering the base class - to be the source of the "cycle". - - .. change:: - :tags: orm, bug - :tickets: 2320 - - The "passive" flag on Session.is_modified() - no longer has any effect. is_modified() in - all cases looks only at local in-memory - modified flags and will not emit any - SQL or invoke loader callables/initializers. - - .. change:: - :tags: orm, bug - :tickets: 2405 - - The warning emitted when using - delete-orphan cascade with one-to-many - or many-to-many without single-parent=True - is now an error. The ORM - would fail to function subsequent to this - warning in any case. - - .. change:: - :tags: orm, bug - :tickets: 2350 - - Lazy loads emitted within flush events - such as before_flush(), before_update(), - etc. will now function as they would - within non-event code, regarding consideration - of the PK/FK values used in the lazy-emitted - query. Previously, - special flags would be established that - would cause lazy loads to load related items - based on the "previous" value of the - parent PK/FK values specifically when called - upon within a flush; the signal to load - in this way is now localized to where the - unit of work actually needs to load that - way. Note that the UOW does - sometimes load these collections before - the before_update() event is called, - so the usage of "passive_updates" or not - can affect whether or not a collection will - represent the "old" or "new" data, when - accessed within a flush event, based - on when the lazy load was emitted. - The change is backwards incompatible in - the exceedingly small chance that - user event code depended on the old - behavior. - - .. change:: - :tags: orm, feature - :tickets: 2179 - - Query now "auto correlates" by - default in the same way as select() does. - Previously, a Query used as a subquery - in another would require the correlate() - method be called explicitly in order to - correlate a table on the inside to the - outside. As always, correlate(None) - disables correlation. - - .. change:: - :tags: orm, feature - :tickets: 2464 - - The after_attach event is now - emitted after the object is established - in Session.new or Session.identity_map - upon Session.add(), Session.merge(), - etc., so that the object is represented - in these collections when the event - is called. Added before_attach - event to accommodate use cases that - need autoflush w pre-attached object. - - .. change:: - :tags: orm, feature - :tickets: - - The Session will produce warnings - when unsupported methods are used inside the - "execute" portion of the flush. These are - the familiar methods add(), delete(), etc. - as well as collection and related-object - manipulations, as called within mapper-level - flush events - like after_insert(), after_update(), etc. - It's been prominently documented for a long - time that SQLAlchemy cannot guarantee - results when the Session is manipulated within - the execution of the flush plan, - however users are still doing it, so now - there's a warning. Maybe someday the Session - will be enhanced to support these operations - inside of the flush, but for now, results - can't be guaranteed. - - .. change:: - :tags: orm, bug - :tickets: 2582, 2566 - - Continuing regarding extra - state post-flush due to event listeners; - any states that are marked as "dirty" from an - attribute perspective, usually via column-attribute - set events within after_insert(), after_update(), - etc., will get the "history" flag reset - in all cases, instead of only those instances - that were part of the flush. This has the effect - that this "dirty" state doesn't carry over - after the flush and won't result in UPDATE - statements. A warning is emitted to this - effect; the set_committed_state() - method can be used to assign attributes on objects - without producing history events. - - .. change:: - :tags: orm, feature - :tickets: 2245 - - ORM entities can be passed - to the core select() construct as well - as to the select_from(), - correlate(), and correlate_except() - methods of select(), where they will be unwrapped - into selectables. - - .. change:: - :tags: orm, feature - :tickets: 2245 - - Some support for auto-rendering of a - relationship join condition based on the mapped - attribute, with usage of core SQL constructs. - E.g. select([SomeClass]).where(SomeClass.somerelationship) - would render SELECT from "someclass" and use the - primaryjoin of "somerelationship" as the WHERE - clause. This changes the previous meaning - of "SomeClass.somerelationship" when used in a - core SQL context; previously, it would "resolve" - to the parent selectable, which wasn't generally - useful. Also works with query.filter(). - Related to. - - .. change:: - :tags: orm, feature - :tickets: 2526 - - The registry of classes - in declarative_base() is now a - WeakValueDictionary. So subclasses of - "Base" that are dereferenced will be - garbage collected, *if they are not - referred to by any other mappers/superclass - mappers*. See the next note for this ticket. - - .. change:: - :tags: orm, feature - :tickets: 2472 - - Conflicts between columns on - single-inheritance declarative subclasses, - with or without using a mixin, can be resolved - using a new @declared_attr usage described - in the documentation. - - .. change:: - :tags: orm, feature - :tickets: 2472 - - declared_attr can now be used - on non-mixin classes, even though this is generally - only useful for single-inheritance subclass - column conflict resolution. - - .. change:: - :tags: orm, feature - :tickets: 2517 - - declared_attr can now be used with - attributes that are not Column or MapperProperty; - including any user-defined value as well - as association proxy objects. - - .. change:: - :tags: orm, bug - :tickets: 2565 - - Fixed a disconnect that slowly evolved - between a @declared_attr Column and a - directly-defined Column on a mixin. In both - cases, the Column will be applied to the - declared class' table, but not to that of a - joined inheritance subclass. Previously, - the directly-defined Column would be placed - on both the base and the sub table, which isn't - typically what's desired. - - .. change:: - :tags: orm, feature - :tickets: 2526 - - *Very limited* support for - inheriting mappers to be GC'ed when the - class itself is deferenced. The mapper - must not have its own table (i.e. - single table inh only) without polymorphic - attributes in place. - This allows for the use case of - creating a temporary subclass of a declarative - mapped class, with no table or mapping - directives of its own, to be garbage collected - when dereferenced by a unit test. - - .. change:: - :tags: orm, feature - :tickets: 2338 - - Declarative now maintains a registry - of classes by string name as well as by full - module-qualified name. Multiple classes with the - same name can now be looked up based on a module-qualified - string within relationship(). Simple class name - lookups where more than one class shares the same - name now raises an informative error message. - - .. change:: - :tags: orm, feature - :tickets: 2535 - - Can now provide class-bound attributes - that override columns which are of any - non-ORM type, not just descriptors. - - .. change:: - :tags: orm, feature - :tickets: 1729 - - Added with_labels and - reduce_columns keyword arguments to - Query.subquery(), to provide two alternate - strategies for producing queries with uniquely- - named columns. . - - .. change:: - :tags: orm, feature - :tickets: 2476 - - A warning is emitted when a reference - to an instrumented collection is no longer - associated with the parent class due to - expiration/attribute refresh/collection - replacement, but an append - or remove operation is received on the - now-detached collection. - - .. change:: - :tags: orm, bug - :tickets: 2549 - - Declarative can now propagate a column - declared on a single-table inheritance subclass - up to the parent class' table, when the parent - class is itself mapped to a join() or select() - statement, directly or via joined inheritance, - and not just a Table. - - .. change:: - :tags: orm, bug - :tickets: - - An error is emitted when uselist=False - is combined with a "dynamic" loader. - This is a warning in 0.7.9. - - .. change:: - :tags: removed, orm - :tickets: - - Deprecated identifiers removed: - - * allow_null_pks mapper() argument - (use allow_partial_pks) - - * _get_col_to_prop() mapper method - (use get_property_by_column()) - - * dont_load argument to Session.merge() - (use load=True) - - * sqlalchemy.orm.shard module - (use sqlalchemy.ext.horizontal_shard) - - .. change:: - :tags: engine, feature - :tickets: 2511 - - Connection event listeners can - now be associated with individual - Connection objects, not just Engine - objects. - - .. change:: - :tags: engine, feature - :tickets: 2459 - - The before_cursor_execute event - fires off for so-called "_cursor_execute" - events, which are usually special-case - executions of primary-key bound sequences - and default-generation SQL - phrases that invoke separately when RETURNING - is not used with INSERT. - - .. change:: - :tags: engine, feature - :tickets: - - The libraries used by the test suite - have been moved around a bit so that they are - part of the SQLAlchemy install again. In addition, - a new suite of tests is present in the - new sqlalchemy.testing.suite package. This is - an under-development system that hopes to provide - a universal testing suite for external dialects. - Dialects which are maintained outside of SQLAlchemy - can use the new test fixture as the framework - for their own tests, and will get for free a - "compliance" suite of dialect-focused tests, - including an improved "requirements" system - where specific capabilities and features can - be enabled or disabled for testing. - - .. change:: - :tags: engine, bug - :tickets: - - The Inspector.get_table_names() - order_by="foreign_key" feature now sorts - tables by dependee first, to be consistent - with util.sort_tables and metadata.sorted_tables. - - .. change:: - :tags: engine, bug - :tickets: 2522 - - Fixed bug whereby if a database restart - affected multiple connections, each - connection would individually invoke a new - disposal of the pool, even though only - one disposal is needed. - - .. change:: - :tags: engine, feature - :tickets: 2462 - - Added a new system - for registration of new dialects in-process - without using an entrypoint. See the - docs for "Registering New Dialects". - - .. change:: - :tags: engine, feature - :tickets: 2556 - - The "required" flag is set to - True by default, if not passed explicitly, - on bindparam() if the "value" or "callable" - parameters are not passed. - This will cause statement execution to check - for the parameter being present in the final - collection of bound parameters, rather than - implicitly assigning None. - - .. change:: - :tags: engine, feature - :tickets: - - Various API tweaks to the "dialect" - API to better support highly specialized - systems such as the Akiban database, including - more hooks to allow an execution context to - access type processors. - - .. change:: - :tags: engine, bug - :tickets: 2397 - - The names of the columns on the - .c. attribute of a select().apply_labels() - is now based on _ instead - of _, for those columns - that have a distinctly named .key. - - .. change:: - :tags: engine, feature - :tickets: 2422 - - Inspector.get_primary_keys() is - deprecated; use Inspector.get_pk_constraint(). - Courtesy Diana Clarke. - - .. change:: - :tags: engine, bug - :tickets: - - The autoload_replace flag on Table, - when False, will cause any reflected foreign key - constraints which refer to already-declared - columns to be skipped, assuming that the - in-Python declared column will take over - the task of specifying in-Python ForeignKey - or ForeignKeyConstraint declarations. - - .. change:: - :tags: engine, bug - :tickets: 2498 - - The ResultProxy methods inserted_primary_key, - last_updated_params(), last_inserted_params(), - postfetch_cols(), prefetch_cols() all - assert that the given statement is a compiled - construct, and is an insert() or update() - statement as is appropriate, else - raise InvalidRequestError. - - .. change:: - :tags: engine, feature - :tickets: - - New C extension module "utils" has - been added for additional function speedups - as we have time to implement. - - .. change:: - :tags: engine - :tickets: - - ResultProxy.last_inserted_ids is removed, - replaced by inserted_primary_key. - - .. change:: - :tags: feature, sql - :tickets: 2547 - - Major rework of operator system - in Core, to allow redefinition of existing - operators as well as addition of new operators - at the type level. New types can be created - from existing ones which add or redefine - operations that are exported out to column - expressions, in a similar manner to how the - ORM has allowed comparator_factory. The new - architecture moves this capability into the - Core so that it is consistently usable in - all cases, propagating cleanly using existing - type propagation behavior. - - .. change:: - :tags: feature, sql - :tickets: 1534, 2547 - - To complement, types - can now provide "bind expressions" and - "column expressions" which allow compile-time - injection of SQL expressions into statements - on a per-column or per-bind level. This is - to suit the use case of a type which needs - to augment bind- and result- behavior at the - SQL level, as opposed to in the Python level. - Allows for schemes like transparent encryption/ - decryption, usage of PostGIS functions, etc. - - .. change:: - :tags: feature, sql - :tickets: - - The Core operator system now includes - the `getitem` operator, i.e. the bracket - operator in Python. This is used at first - to provide index and slice behavior to the - PostgreSQL ARRAY type, and also provides a hook - for end-user definition of custom __getitem__ - schemes which can be applied at the type - level as well as within ORM-level custom - operator schemes. `lshift` (<<) - and `rshift` (>>) are also supported as - optional operators. - - Note that this change has the effect that - descriptor-based __getitem__ schemes used by - the ORM in conjunction with synonym() or other - "descriptor-wrapped" schemes will need - to start using a custom comparator in order - to maintain this behavior. - - .. change:: - :tags: feature, sql - :tickets: 2537 - - Revised the rules used to determine - the operator precedence for the user-defined - operator, i.e. that granted using the ``op()`` - method. Previously, the smallest precedence - was applied in all cases, now the default - precedence is zero, lower than all operators - except "comma" (such as, used in the argument - list of a ``func`` call) and "AS", and is - also customizable via the "precedence" argument - on the ``op()`` method. - - .. change:: - :tags: feature, sql - :tickets: 2276 - - Added "collation" parameter to all - String types. When present, renders as - COLLATE . This to support the - COLLATE keyword now supported by several - databases including MySQL, SQLite, and PostgreSQL. - - .. change:: - :tags: change, sql - :tickets: - - The Text() type renders the length - given to it, if a length was specified. - - .. change:: - :tags: feature, sql - :tickets: - - Custom unary operators can now be - used by combining operators.custom_op() with - UnaryExpression(). - - .. change:: - :tags: bug, sql - :tickets: 2564 - - A tweak to column precedence which moves the - "concat" and "match" operators to be the same as - that of "is", "like", and others; this helps with - parenthesization rendering when used in conjunction - with "IS". - - .. change:: - :tags: feature, sql - :tickets: - - Enhanced GenericFunction and func.* - to allow for user-defined GenericFunction - subclasses to be available via the func.* - namespace automatically by classname, - optionally using a package name, as well - as with the ability to have the rendered - name different from the identified name - in func.*. - - .. change:: - :tags: feature, sql - :tickets: 2562 - - The cast() and extract() constructs - will now be produced via the func.* accessor - as well, as users naturally try to access these - names from func.* they might as well do - what's expected, even though the returned - object is not a FunctionElement. - - .. change:: - :tags: changed, sql - :tickets: - - Most classes in expression.sql - are no longer preceded with an underscore, - i.e. Label, SelectBase, Generative, CompareMixin. - _BindParamClause is also renamed to - BindParameter. The old underscore names for - these classes will remain available as synonyms - for the foreseeable future. - - .. change:: - :tags: feature, sql - :tickets: 2208 - - The Inspector object can now be - acquired using the new inspect() service, - part of - - .. change:: - :tags: feature, sql - :tickets: 2418 - - The column_reflect event now - accepts the Inspector object as the first - argument, preceding "table". Code which - uses the 0.7 version of this very new - event will need modification to add the - "inspector" object as the first argument. - - .. change:: - :tags: feature, sql - :tickets: 2423 - - The behavior of column targeting - in result sets is now case sensitive by - default. SQLAlchemy for many years would - run a case-insensitive conversion on these values, - probably to alleviate early case sensitivity - issues with dialects like Oracle and - Firebird. These issues have been more cleanly - solved in more modern versions so the performance - hit of calling lower() on identifiers is removed. - The case insensitive comparisons can be re-enabled - by setting "case_insensitive=False" on - create_engine(). - - .. change:: - :tags: bug, sql - :tickets: 2591 - - Applying a column expression to a select - statement using a label with or without other - modifying constructs will no longer "target" that - expression to the underlying Column; this affects - ORM operations that rely upon Column targeting - in order to retrieve results. That is, a query - like query(User.id, User.id.label('foo')) will now - track the value of each "User.id" expression separately - instead of munging them together. It is not expected - that any users will be impacted by this; however, - a usage that uses select() in conjunction with - query.from_statement() and attempts to load fully - composed ORM entities may not function as expected - if the select() named Column objects with arbitrary - .label() names, as these will no longer target to - the Column objects mapped by that entity. - - .. change:: - :tags: feature, sql - :tickets: 2415 - - The "unconsumed column names" warning emitted - when keys are present in insert.values() or update.values() - that aren't in the target table is now an exception. - - .. change:: - :tags: feature, sql - :tickets: 2502 - - Added "MATCH" clause to ForeignKey, - ForeignKeyConstraint, courtesy Ryan Kelly. - - .. change:: - :tags: feature, sql - :tickets: 2507 - - Added support for DELETE and UPDATE from - an alias of a table, which would assumedly - be related to itself elsewhere in the query, - courtesy Ryan Kelly. - - .. change:: - :tags: feature, sql - :tickets: - - select() features a correlate_except() - method, auto correlates all selectables except those - passed. - - .. change:: - :tags: feature, sql - :tickets: 2431 - - The prefix_with() method is now available - on each of select(), insert(), update(), delete(), - all with the same API, accepting multiple - prefix calls, as well as a "dialect name" so that - the prefix can be limited to one kind of dialect. - - .. change:: - :tags: feature, sql - :tickets: 1729 - - Added reduce_columns() method - to select() construct, replaces columns inline - using the util.reduce_columns utility function - to remove equivalent columns. reduce_columns() - also adds "with_only_synonyms" to limit the - reduction just to those columns which have the same - name. The deprecated fold_equivalents() feature is - removed. - - .. change:: - :tags: feature, sql - :tickets: 2470 - - Reworked the startswith(), endswith(), - contains() operators to do a better job with - negation (NOT LIKE), and also to assemble them - at compilation time so that their rendered SQL - can be altered, such as in the case for Firebird - STARTING WITH - - .. change:: - :tags: feature, sql - :tickets: 2463 - - Added a hook to the system of rendering - CREATE TABLE that provides access to the render for each - Column individually, by constructing a @compiles - function against the new schema.CreateColumn - construct. - - .. change:: - :tags: feature, sql - :tickets: - - "scalar" selects now have a WHERE method - to help with generative building. Also slight adjustment - regarding how SS "correlates" columns; the new methodology - no longer applies meaning to the underlying - Table column being selected. This improves - some fairly esoteric situations, and the logic - that was there didn't seem to have any purpose. - - .. change:: - :tags: bug, sql - :tickets: 2520 - - Fixes to the interpretation of the - Column "default" parameter as a callable - to not pass ExecutionContext into a keyword - argument parameter. - - .. change:: - :tags: bug, sql - :tickets: 2410 - - All of UniqueConstraint, ForeignKeyConstraint, - CheckConstraint, and PrimaryKeyConstraint will - attach themselves to their parent table automatically - when they refer to a Table-bound Column object directly - (i.e. not just string column name), and refer to - one and only one Table. Prior to 0.8 this behavior - occurred for UniqueConstraint and PrimaryKeyConstraint, - but not ForeignKeyConstraint or CheckConstraint. - - .. change:: - :tags: bug, sql - :tickets: 2594 - - TypeDecorator now includes a generic repr() - that works in terms of the "impl" type by default. - This is a behavioral change for those TypeDecorator - classes that specify a custom __init__ method; those - types will need to re-define __repr__() if they need - __repr__() to provide a faithful constructor representation. - - .. change:: - :tags: bug, sql - :tickets: 2168 - - column.label(None) now produces an - anonymous label, instead of returning the - column object itself, consistent with the behavior - of label(column, None). - - .. change:: - :tags: feature, sql - :tickets: 2455 - - An explicit error is raised when - a ForeignKeyConstraint() that was - constructed to refer to multiple remote tables - is first used. - - .. change:: - :tags: access, feature - :tickets: - - the MS Access dialect has been - moved to its own project on Bitbucket, - taking advantage of the new SQLAlchemy - dialect compliance suite. The dialect is - still in very rough shape and probably not - ready for general use yet, however - it does have *extremely* rudimental - functionality now. - https://bitbucket.org/zzzeek/sqlalchemy-access - - .. change:: - :tags: maxdb, moved - :tickets: - - The MaxDB dialect, which hasn't been - functional for several years, is - moved out to a pending bitbucket project, - https://bitbucket.org/zzzeek/sqlalchemy-maxdb. - - .. change:: - :tags: sqlite, feature - :tickets: 2363 - - the SQLite date and time types - have been overhauled to support a more open - ended format for input and output, using - name based format strings and regexps. A - new argument "microseconds" also provides - the option to omit the "microseconds" - portion of timestamps. Thanks to - Nathan Wright for the work and tests on - this. - - .. change:: - :tags: mssql, feature - :tickets: - - SQL Server dialect can be given - database-qualified schema names, - i.e. "schema='mydatabase.dbo'"; reflection - operations will detect this, split the schema - among the "." to get the owner separately, - and emit a "USE mydatabase" statement before - reflecting targets within the "dbo" owner; - the existing database returned from - DB_NAME() is then restored. - - .. change:: - :tags: mssql, bug - :tickets: 2277 - - removed legacy behavior whereby - a column comparison to a scalar SELECT via - == would coerce to an IN with the SQL server - dialect. This is implicit - behavior which fails in other scenarios - so is removed. Code which relies on this - needs to be modified to use column.in_(select) - explicitly. - - .. change:: - :tags: mssql, feature - :tickets: - - updated support for the mxodbc - driver; mxodbc 3.2.1 is recommended for full - compatibility. - - .. change:: - :tags: postgresql, feature - :tickets: 2441 - - postgresql.ARRAY features an optional - "dimension" argument, will assign a specific - number of dimensions to the array which will - render in DDL as ARRAY[][]..., also improves - performance of bind/result processing. - - .. change:: - :tags: postgresql, feature - :tickets: - - postgresql.ARRAY now supports - indexing and slicing. The Python [] operator - is available on all SQL expressions that are - of type ARRAY; integer or simple slices can be - passed. The slices can also be used on the - assignment side in the SET clause of an UPDATE - statement by passing them into Update.values(); - see the docs for examples. - - .. change:: - :tags: postgresql, feature - :tickets: - - Added new "array literal" construct - postgresql.array(). Basically a "tuple" that - renders as ARRAY[1,2,3]. - - .. change:: - :tags: postgresql, feature - :tickets: 2506 - - Added support for the PostgreSQL ONLY - keyword, which can appear corresponding to a - table in a SELECT, UPDATE, or DELETE statement. - The phrase is established using with_hint(). - Courtesy Ryan Kelly - - .. change:: - :tags: postgresql, feature - :tickets: - - The "ischema_names" dictionary of the - PostgreSQL dialect is "unofficially" customizable. - Meaning, new types such as PostGIS types can - be added into this dictionary, and the PG type - reflection code should be able to handle simple - types with variable numbers of arguments. - The functionality here is "unofficial" for - three reasons: - - 1. this is not an "official" API. Ideally - an "official" API would allow custom type-handling - callables at the dialect or global level - in a generic way. - 2. This is only implemented for the PG dialect, - in particular because PG has broad support - for custom types vs. other database backends. - A real API would be implemented at the - default dialect level. - 3. The reflection code here is only tested against - simple types and probably has issues with more - compositional types. - - patch courtesy Éric Lemoine. - - .. change:: - :tags: firebird, feature - :tickets: 2470 - - The "startswith()" operator renders - as "STARTING WITH", "~startswith()" renders - as "NOT STARTING WITH", using FB's more efficient - operator. - - .. change:: - :tags: firebird, bug - :tickets: 2505 - - CompileError is raised when VARCHAR with - no length is attempted to be emitted, same - way as MySQL. - - .. change:: - :tags: firebird, bug - :tickets: - - Firebird now uses strict "ansi bind rules" - so that bound parameters don't render in the - columns clause of a statement - they render - literally instead. - - .. change:: - :tags: firebird, bug - :tickets: - - Support for passing datetime as date when - using the DateTime type with Firebird; other - dialects support this. - - .. change:: - :tags: firebird, feature - :tickets: 2504 - - An experimental dialect for the fdb - driver is added, but is untested as I cannot - get the fdb package to build. - - .. change:: - :tags: bug, mysql - :tickets: 2404 - - Dialect no longer emits expensive server - collations query, as well as server casing, - on first connect. These functions are still - available as semi-private. - - .. change:: - :tags: feature, mysql - :tickets: 2534 - - Added TIME type to mysql dialect, - accepts "fst" argument which is the new - "fractional seconds" specifier for recent - MySQL versions. The datatype will interpret - a microseconds portion received from the driver, - however note that at this time most/all MySQL - DBAPIs do not support returning this value. - - .. change:: - :tags: oracle, bug - :tickets: 2437 - - Quoting information is now passed along - from a Column with quote=True when generating - a same-named bound parameter to the bindparam() - object, as is the case in generated INSERT and UPDATE - statements, so that unknown reserved names can - be fully supported. - - .. change:: - :tags: oracle, feature - :tickets: 2561 - - The types of columns excluded from the - setinputsizes() set can be customized by sending - a list of string DBAPI type names to exclude, - using the exclude_setinputsizes dialect parameter. - This list was previously fixed. The list also - now defaults to STRING, UNICODE, removing - CLOB, NCLOB from the list. - - .. change:: - :tags: oracle, bug - :tickets: - - The CreateIndex construct in Oracle - will now schema-qualify the name of the index - to be that of the parent table. Previously this - name was omitted which apparently creates the - index in the default schema, rather than that - of the table. - - .. change:: - :tags: sql, feature - :tickets: 2580 - - Added :meth:`.ColumnOperators.notin_`, - :meth:`.ColumnOperators.notlike`, - :meth:`.ColumnOperators.notilike` to :class:`.ColumnOperators`. - - .. change:: - :tags: sql, removed - - The long-deprecated and non-functional ``assert_unicode`` flag on - :func:`_sa.create_engine` as well as :class:`.String` is removed. diff --git a/doc/build/changelog/changelog_09.rst b/doc/build/changelog/changelog_09.rst deleted file mode 100644 index d00e043326e..00000000000 --- a/doc/build/changelog/changelog_09.rst +++ /dev/null @@ -1,3340 +0,0 @@ -============= -0.9 Changelog -============= - -.. changelog_imports:: - - .. include:: changelog_08.rst - :start-line: 5 - - - .. include:: changelog_07.rst - :start-line: 5 - - -.. _unreleased_changelog:: - :version: 0.9.11 - - .. change:: - :tags: bug, oracle, py3k - :tickets: 3491 - :versions: 1.0.9 - - Fixed support for cx_Oracle version 5.2, which was tripping - up SQLAlchemy's version detection under Python 3 and inadvertently - not using the correct unicode mode for Python 3. This would cause - issues such as bound variables mis-interpreted as NULL and rows - silently not being returned. - - .. change:: - :tags: bug, engine - :tickets: 3497 - :versions: 1.0.8 - - Fixed critical issue whereby the pool "checkout" event handler - may be called against a stale connection without the "connect" - event handler having been called, in the case where the pool - attempted to reconnect after being invalidated and failed; the stale - connection would remain present and would be used on a subsequent - attempt. This issue has a greater impact in the 1.0 series subsequent - to 1.0.2, as it also delivers a blanked-out ``.info`` dictionary to - the event handler; prior to 1.0.2 the ``.info`` dictionary is still - the previous one. - -.. changelog:: - :version: 0.9.10 - :released: July 22, 2015 - - .. change:: - :tags: bug, sqlite - :tickets: 3495 - :versions: 1.0.8 - - Fixed bug in SQLite dialect where reflection of UNIQUE constraints - that included non-alphabetic characters in the names, like dots or - spaces, would not be reflected with their name. - - .. change:: - :tags: feature, sql - :tickets: 3418 - :versions: 1.0.5 - - Added official support for a CTE used by the SELECT present - inside of :meth:`_expression.Insert.from_select`. This behavior worked - accidentally up until 0.9.9, when it no longer worked due to - unrelated changes as part of :ticket:`3248`. Note that this - is the rendering of the WITH clause after the INSERT, before the - SELECT; the full functionality of CTEs rendered at the top - level of INSERT, UPDATE, DELETE is a new feature targeted for a - later release. - - .. change:: - :tags: bug, ext - :tickets: 3408 - :versions: 1.0.4 - - Fixed bug where when using extended attribute instrumentation system, - the correct exception would not be raised when :func:`.class_mapper` - were called with an invalid input that also happened to not - be weak referencable, such as an integer. - - .. change:: - :tags: bug, tests, pypy - :tickets: 3406 - :versions: 1.0.4 - - Fixed an import that prevented "pypy setup.py test" from working - correctly. - - .. change:: - :tags: bug, engine - :tickets: 3375 - :versions: 1.0.1 - - Added the string value ``"none"`` to those accepted by the - :paramref:`_pool.Pool.reset_on_return` parameter as a synonym for ``None``, - so that string values can be used for all settings, allowing - utilities like :func:`.engine_from_config` to be usable without - issue. - - .. change:: - :tags: bug, sql - :tickets: 3362 - :versions: 1.0.0 - - Fixed issue where a :class:`_schema.MetaData` object that used a naming - convention would not properly work with pickle. The attribute was - skipped leading to inconsistencies and failures if the unpickled - :class:`_schema.MetaData` object were used to base additional tables - from. - - .. change:: - :tags: bug, postgresql - :tickets: 3354 - :versions: 1.0.0 - - Fixed a long-standing bug where the :class:`.Enum` type as used - with the psycopg2 dialect in conjunction with non-ascii values - and ``native_enum=False`` would fail to decode return results properly. - This stemmed from when the PG :class:`_postgresql.ENUM` type used - to be a standalone type without a "non native" option. - - .. change:: - :tags: bug, orm - :tickets: 3349 - - :class:`_query.Query` doesn't support joins, subselects, or special - FROM clauses when using the :meth:`_query.Query.update` or - :meth:`_query.Query.delete` methods; instead of silently ignoring these - fields if methods like :meth:`_query.Query.join` or - :meth:`_query.Query.select_from` has been called, a warning is emitted. - As of 1.0.0b5 this will raise an error. - - .. change:: - :tags: bug, orm - :tickets: 3352 - :versions: 1.0.0b5 - - Fixed bug where the state tracking within multiple, nested - :meth:`.Session.begin_nested` operations would fail to propagate - the "dirty" flag for an object that had been updated within - the inner savepoint, such that if the enclosing savepoint were - rolled back, the object would not be part of the state that was - expired and therefore reverted to its database state. - - .. change:: - :tags: bug, mysql, pymysql - :tickets: 3337 - :versions: 1.0.0b4 - - Fixed unicode support for PyMySQL when using an "executemany" - operation with unicode parameters. SQLAlchemy now passes both - the statement as well as the bound parameters as unicode - objects, as PyMySQL generally uses string interpolation - internally to produce the final statement, and in the case of - executemany does the "encode" step only on the final statement. - - .. change:: - :tags: bug, py3k, mysql - :tickets: 3333 - :versions: 1.0.0b2 - - Fixed the :class:`.mysql.BIT` type on Py3K which was not using the - ``ord()`` function correctly. Pull request courtesy David Marin. - - .. change:: - :tags: bug, ext - :tickets: 3324 - - Fixed regression from 0.9.9 where the :func:`.as_declarative` - symbol was removed from the ``sqlalchemy.ext.declarative`` - namespace. - - .. change:: - :tags: feature, orm - :tickets: 3320 - :versions: 1.0.0b1 - - Added a new entry ``"entity"`` to the dictionaries returned by - :attr:`_query.Query.column_descriptions`. This refers to the primary ORM - mapped class or aliased class that is referred to by the expression. - Compared to the existing entry for ``"type"``, it will always be - a mapped entity, even if extracted from a column expression, or - None if the given expression is a pure core expression. - See also :ticket:`3403` which repaired a regression in this feature - which was unreleased in 0.9.10 but was released in the 1.0 version. - - -.. changelog:: - :version: 0.9.9 - :released: March 10, 2015 - - .. change:: - :tags: feature, postgresql - :versions: 1.0.0b1 - - Added support for the ``CONCURRENTLY`` keyword with PostgreSQL - indexes, established using ``postgresql_concurrently``. Pull - request courtesy Iuri de Silvio. - - .. seealso:: - - :ref:`postgresql_index_concurrently` - - .. change:: - :tags: bug, ext, py3k - :versions: 1.0.0b1 - - Fixed bug where the association proxy list class would not interpret - slices correctly under Py3K. Pull request courtesy - Gilles Dartiguelongue. - - .. change:: - :tags: feature, sqlite - :versions: 1.0.0b1 - - Added support for partial indexes (e.g. with a WHERE clause) on - SQLite. Pull request courtesy Kai Groner. - - .. seealso:: - - :ref:`sqlite_partial_index` - - .. change:: - :tags: bug, orm - :tickets: 3310 - :versions: 1.0.0b1 - - Fixed bugs in ORM object comparisons where comparison of - many-to-one ``!= None`` would fail if the source were an aliased - class, or if the query needed to apply special aliasing to the - expression due to aliased joins or polymorphic querying; also fixed - bug in the case where comparing a many-to-one to an object state - would fail if the query needed to apply special aliasing - due to aliased joins or polymorphic querying. - - .. change:: - :tags: bug, orm - :tickets: 3309 - :versions: 1.0.0b1 - - Fixed bug where internal assertion would fail in the case where - an ``after_rollback()`` handler for a :class:`.Session` incorrectly - adds state to that :class:`.Session` within the handler, and the task - to warn and remove this state (established by :ticket:`2389`) attempts - to proceed. - - .. change:: - :tags: bug, orm - :versions: 1.0.0b1 - - Fixed bug where TypeError raised when :meth:`_query.Query.join` called - with unknown kw arguments would raise its own TypeError due - to broken formatting. Pull request courtesy Malthe Borch. - - .. change:: - :tags: bug, engine - :tickets: 3302 - :versions: 1.0.0b1 - - Fixed bug in :class:`_engine.Connection` and pool where the - :meth:`_engine.Connection.invalidate` method, or an invalidation due - to a database disconnect, would fail if the - ``isolation_level`` parameter had been used with - :meth:`_engine.Connection.execution_options`; the "finalizer" that resets - the isolation level would be called on the no longer opened connection. - - .. change:: - :tags: feature, orm - :tickets: 3296 - :versions: 1.0.0b1 - - Added new parameter :paramref:`.Session.connection.execution_options` - which may be used to set up execution options on a :class:`_engine.Connection` - when it is first checked out, before the transaction has begun. - This is used to set up options such as isolation level on the - connection before the transaction starts. - - .. seealso:: - - :ref:`session_transaction_isolation` - new documentation section - detailing best practices for setting transaction isolation with - sessions. - - .. change:: - :tags: bug, engine - :tickets: 3296 - :versions: 1.0.0b1 - - A warning is emitted if the ``isolation_level`` parameter is used - with :meth:`_engine.Connection.execution_options` when a :class:`.Transaction` - is in play; DBAPIs and/or SQLAlchemy dialects such as psycopg2, - MySQLdb may implicitly rollback or commit the transaction, or - not change the setting til next transaction, so this is never safe. - - .. change:: - :tags: bug, orm - :tickets: 3300 - :versions: 1.0.0b1 - - Fixed bug in lazy loading SQL construction whereby a complex - primaryjoin that referred to the same "local" column multiple - times in the "column that points to itself" style of self-referential - join would not be substituted in all cases. The logic to determine - substitutions here has been reworked to be more open-ended. - - .. change:: - :tags: bug, postgresql - :tickets: 2940 - :versions: 1.0.0b1 - - Repaired support for PostgreSQL UUID types in conjunction with - the ARRAY type when using psycopg2. The psycopg2 dialect now - employs use of the psycopg2.extras.register_uuid() hook - so that UUID values are always passed to/from the DBAPI as - UUID() objects. The :paramref:`.UUID.as_uuid` flag is still - honored, except with psycopg2 we need to convert returned - UUID objects back into strings when this is disabled. - - .. change:: - :tags: bug, postgresql - :versions: 1.0.0b1 - - Added support for the :class:`postgresql.JSONB` datatype when - using psycopg2 2.5.4 or greater, which features native conversion - of JSONB data so that SQLAlchemy's converters must be disabled; - additionally, the newly added psycopg2 extension - ``extras.register_default_jsonb`` is used to establish a JSON - deserializer passed to the dialect via the ``json_deserializer`` - argument. Also repaired the PostgreSQL integration tests which - weren't actually round-tripping the JSONB type as opposed to the - JSON type. Pull request courtesy Mateusz Susik. - - .. change:: - :tags: bug, postgresql - :versions: 1.0.0b1 - - Repaired the use of the "array_oid" flag when registering the - HSTORE type with older psycopg2 versions < 2.4.3, which does not - support this flag, as well as use of the native json serializer - hook "register_default_json" with user-defined ``json_deserializer`` - on psycopg2 versions < 2.5, which does not include native json. - - .. change:: - :tags: bug, schema - :tickets: 3298, 1765 - - Fixed bug in 0.9's foreign key setup system, such that - the logic used to link a :class:`_schema.ForeignKey` to its parent could fail - when the foreign key used "link_to_name=True" in conjunction with - a target :class:`_schema.Table` that would not receive its parent column until - later, such as within a reflection + "useexisting" scenario, - if the target column in fact had a key value different from its name, - as would occur in reflection if column reflect events were used to - alter the .key of reflected :class:`_schema.Column` objects so that the - link_to_name becomes significant. Also repaired support for column - type via FK transmission in a similar way when target columns had a - different key and were referenced using link_to_name. - - .. change:: - :tags: feature, engine - :versions: 1.0.0b1 - - Added new user-space accessors for viewing transaction isolation - levels; :meth:`_engine.Connection.get_isolation_level`, - :attr:`_engine.Connection.default_isolation_level`. - - .. change:: - :tags: bug, postgresql - :versions: 1.0.0b1 - :tickets: 3174 - - Fixed bug where PostgreSQL dialect would fail to render an - expression in an :class:`.Index` that did not correspond directly - to a table-bound column; typically when a :func:`_expression.text` construct - was one of the expressions within the index; or could misinterpret the - list of expressions if one or more of them were such an expression. - - .. change:: - :tags: bug, orm - :versions: 1.0.0b1 - :tickets: 3287 - - The "wildcard" loader options, in particular the one set up by - the :func:`_orm.load_only` option to cover all attributes not - explicitly mentioned, now takes into account the superclasses - of a given entity, if that entity is mapped with inheritance mapping, - so that attribute names within the superclasses are also omitted - from the load. Additionally, the polymorphic discriminator column - is unconditionally included in the list, just in the same way that - primary key columns are, so that even with load_only() set up, - polymorphic loading of subtypes continues to function correctly. - - .. change:: - :tags: bug, sql - :versions: 1.0.0b1 - - Added the ``native_enum`` flag to the ``__repr__()`` output - of :class:`.Enum`, which is mostly important when using it with - Alembic autogenerate. Pull request courtesy Dimitris Theodorou. - - .. change:: - :tags: bug, orm, pypy - :versions: 1.0.0b1 - :tickets: 3285 - - Fixed bug where if an exception were thrown at the start of a - :class:`_query.Query` before it fetched results, particularly when - row processors can't be formed, the cursor would stay open with - results pending and not actually be closed. This is typically only - an issue on an interpreter like PyPy where the cursor isn't - immediately GC'ed, and can in some circumstances lead to transactions/ - locks being open longer than is desirable. - - .. change:: - :tags: change, mysql - :versions: 1.0.0b1 - :tickets: 3275 - - The ``gaerdbms`` dialect is no longer necessary, and emits a - deprecation warning. Google now recommends using the MySQLdb - dialect directly. - - .. change:: - :tags: bug, sql - :versions: 1.0.0b1 - :tickets: 3278 - - Fixed bug where using a :class:`.TypeDecorator` that implemented - a type that was also a :class:`.TypeDecorator` would fail with - Python's "Cannot create a consistent method resolution order (MRO)" - error, when any kind of SQL comparison expression were used against - an object using this type. - - .. change:: - :tags: bug, mysql - :versions: 1.0.0b1 - :tickets: 3274 - - Added a version check to the MySQLdb dialect surrounding the - check for 'utf8_bin' collation, as this fails on MySQL server < 5.0. - - .. change:: - :tags: feature, orm - :versions: 1.0.0b1 - - Added new method :meth:`.Session.invalidate`, functions similarly - to :meth:`.Session.close`, except also calls - :meth:`_engine.Connection.invalidate` - on all connections, guaranteeing that they will not be returned to - the connection pool. This is useful in situations e.g. dealing - with gevent timeouts when it is not safe to use the connection further, - even for rollbacks. - - .. change:: - :tags: bug, examples - :versions: 1.0.0b1 - - Updated the :ref:`examples_versioned_history` example such that - mapped columns are re-mapped to - match column names as well as grouping of columns; in particular, - this allows columns that are explicitly grouped in a same-column-named - joined inheritance scenario to be mapped in the same way in the - history mappings, avoiding warnings added in the 0.9 series - regarding this pattern and allowing the same view of attribute - keys. - - .. change:: - :tags: bug, examples - :versions: 1.0.0b1 - - Fixed a bug in the examples/generic_associations/discriminator_on_association.py - example, where the subclasses of AddressAssociation were not being - mapped as "single table inheritance", leading to problems when trying - to use the mappings further. - - .. change:: - :tags: bug, orm - :versions: 1.0.0b1 - :tickets: 3251 - - Fixed a leak which would occur in the unsupported and highly - non-recommended use case of replacing a relationship on a fixed - mapped class many times, referring to an arbitrarily growing number of - target mappers. A warning is emitted when the old relationship is - replaced, however if the mapping were already used for querying, the - old relationship would still be referenced within some registries. - - .. change:: - :tags: bug, sql - :versions: 1.0.0b1 - :tickets: 3248 - - Fixed issue where the columns from a SELECT embedded in an - INSERT, either through the values clause or as a "from select", - would pollute the column types used in the result set produced by - the RETURNING clause when columns from both statements shared the - same name, leading to potential errors or mis-adaptation when - retrieving the returning rows. - - .. change:: - :tags: bug, orm, sqlite - :versions: 1.0.0b1 - :tickets: 3241 - - Fixed bug regarding expression mutations which could express - itself as a "Could not locate column" error when using - :class:`_query.Query` to select from multiple, anonymous column - entities when querying against SQLite, as a side effect of the - "join rewriting" feature used by the SQLite dialect. - - .. change:: - :tags: feature, sqlite - :versions: 1.0.0b1 - - Added a new SQLite backend for the SQLCipher backend. This backend - provides for encrypted SQLite databases using the pysqlcipher Python - driver, which is very similar to the pysqlite driver. - - .. seealso:: - - :mod:`~sqlalchemy.dialects.sqlite.pysqlcipher` - - .. change:: - :tags: bug, orm - :tickets: 3232 - :versions: 1.0.0b1 - - Fixed bug where the ON clause for :meth:`_query.Query.join`, - and :meth:`_query.Query.outerjoin` to a single-inheritance subclass - using ``of_type()`` would not render the "single table criteria" in - the ON clause if the ``from_joinpoint=True`` flag were set. - -.. changelog:: - :version: 0.9.8 - :released: October 13, 2014 - - .. change:: - :tags: bug, mysql, mysqlconnector - :versions: 1.0.0b1 - - Mysqlconnector as of version 2.0, probably as a side effect of - the python 3 merge, now does not expect percent signs (e.g. - as used as the modulus operator and others) to be doubled, - even when using the "pyformat" bound parameter format (this - change is not documented by Mysqlconnector). The dialect now - checks for py2k and for mysqlconnector less than version 2.0 - when detecting if the modulus operator should be rendered as - ``%%`` or ``%``. - - .. change:: - :tags: bug, mysql, mysqlconnector - :versions: 1.0.0b1 - - Unicode SQL is now passed for MySQLconnector version 2.0 and above; - for Py2k and MySQL < 2.0, strings are encoded. - - - .. change:: - :tags: bug, oracle - :versions: 1.0.0b1 - :tickets: 2138 - - Fixed long-standing bug in Oracle dialect where bound parameter - names that started with numbers would not be quoted, as Oracle - doesn't like numerics in bound parameter names. - - .. change:: - :tags: bug, sql - :versions: 1.0.0b1 - :tickets: 3195 - - Fixed bug where a fair number of SQL elements within - the sql package would fail to ``__repr__()`` successfully, - due to a missing ``description`` attribute that would then invoke - a recursion overflow when an internal AttributeError would then - re-invoke ``__repr__()``. - - .. change:: - :tags: bug, declarative, orm - :versions: 1.0.0b1 - :tickets: 3185 - - Fixed "'NoneType' object has no attribute 'concrete'" error - when using :class:`.AbstractConcreteBase` in conjunction with - a subclass that declares ``__abstract__``. - - .. change:: - :tags: bug, engine - :versions: 1.0.0b1 - :tickets: 3200 - - The execution options passed to an :class:`_engine.Engine` either via - :paramref:`_sa.create_engine.execution_options` or - :meth:`_engine.Engine.update_execution_options` are not passed to the - special :class:`_engine.Connection` used to initialize the dialect - within the "first connect" event; dialects will usually - perform their own queries in this phase, and none of the - current available options should be applied here. In - particular, the "autocommit" option was causing an attempt to - autocommit within this initial connect which would fail with - an AttributeError due to the non-standard state of the - :class:`_engine.Connection`. - - .. change:: - :tags: bug, sqlite - :versions: 1.0.0b1 - :tickets: 3211 - - When selecting from a UNION using an attached database file, - the pysqlite driver reports column names in cursor.description - as 'dbname.tablename.colname', instead of 'tablename.colname' as - it normally does for a UNION (note that it's supposed to just be - 'colname' for both, but we work around it). The column translation - logic here has been adjusted to retrieve the rightmost token, rather - than the second token, so it works in both cases. Workaround - courtesy Tony Roberts. - - .. change:: - :tags: bug, postgresql - :versions: 1.0.0b1 - :tickets: 3021 - - A revisit to this issue first patched in 0.9.5, apparently - psycopg2's ``.closed`` accessor is not as reliable as we assumed, - so we have added an explicit check for the exception messages - "SSL SYSCALL error: Bad file descriptor" and - "SSL SYSCALL error: EOF detected" when detecting an - is-disconnect scenario. We will continue to consult psycopg2's - connection.closed as a first check. - - .. change:: - :tags: bug, orm, engine - :versions: 1.0.0b1 - :tickets: 3197 - - Fixed bug that affected generally the same classes of event - as that of :ticket:`3199`, when the ``named=True`` parameter - would be used. Some events would fail to register, and others - would not invoke the event arguments correctly, generally in the - case of when an event was "wrapped" for adaption in some other way. - The "named" mechanics have been rearranged to not interfere with - the argument signature expected by internal wrapper functions. - - .. change:: - :tags: bug, declarative - :versions: 1.0.0b1 - :tickets: 3208 - - Fixed an unlikely race condition observed in some exotic end-user - setups, where the attempt to check for "duplicate class name" in - declarative would hit upon a not-totally-cleaned-up weak reference - related to some other class being removed; the check here now ensures - the weakref still references an object before calling upon it further. - - .. change:: - :tags: bug, orm - :versions: 1.0.0b1 - :tickets: 3199 - - Fixed bug that affected many classes of event, particularly - ORM events but also engine events, where the usual logic of - "de duplicating" a redundant call to :func:`.event.listen` - with the same arguments would fail, for those events where the - listener function is wrapped. An assertion would be hit within - registry.py. This assertion has now been integrated into the - deduplication check, with the added bonus of a simpler means - of checking deduplication across the board. - - .. change:: - :tags: bug, mssql - :versions: 1.0.0b1 - :tickets: 3151 - - Fixed the version string detection in the pymssql dialect to - work with Microsoft SQL Azure, which changes the word "SQL Server" - to "SQL Azure". - - .. change:: - :tags: bug, orm - :versions: 1.0.0b1 - :tickets: 3194 - - Fixed warning that would emit when a complex self-referential - primaryjoin contained functions, while at the same time remote_side - was specified; the warning would suggest setting "remote side". - It now only emits if remote_side isn't present. - - .. change:: - :tags: bug, ext - :versions: 1.0.0b1 - :tickets: 3191 - - Fixed bug in ordering list where the order of items would be - thrown off during a collection replace event, if the - reorder_on_append flag were set to True. The fix ensures that the - ordering list only impacts the list that is explicitly associated - with the object. - - .. change:: - :tags: bug, sql - :versions: 1.0.0b1 - :tickets: 3180 - - An adjustment to table/index reflection such that if an index - reports a column that isn't found to be present in the table, - a warning is emitted and the column is skipped. This can occur - for some special system column situations as has been observed - with Oracle. - - .. change:: - :tags: bug, ext - :versions: 1.0.0b1 - - Fixed bug where :class:`.ext.mutable.MutableDict` - failed to implement the ``update()`` dictionary method, thus - not catching changes. Pull request courtesy Matt Chisholm. - - .. change:: - :tags: bug, ext - :versions: 1.0.0b1 - - Fixed bug where a custom subclass of :class:`.ext.mutable.MutableDict` - would not show up in a "coerce" operation, and would instead - return a plain :class:`.ext.mutable.MutableDict`. Pull request - courtesy Matt Chisholm. - - .. change:: - :tags: bug, pool - :versions: 1.0.0b1 - :tickets: 3168 - - Fixed bug in connection pool logging where the "connection checked out" - debug logging message would not emit if the logging were set up using - ``logging.setLevel()``, rather than using the ``echo_pool`` flag. - Tests to assert this logging have been added. This is a - regression that was introduced in 0.9.0. - - .. change:: - :tags: feature, postgresql, pg8000 - :versions: 1.0.0b1 - - Support is added for "sane multi row count" with the pg8000 driver, - which applies mostly to when using versioning with the ORM. - The feature is version-detected based on pg8000 1.9.14 or greater - in use. Pull request courtesy Tony Locke. - - .. change:: - :tags: bug, engine - :versions: 1.0.0b1 - :tickets: 3165 - - The string keys that are used to determine the columns impacted - for an INSERT or UPDATE are now sorted when they contribute towards - the "compiled cache" cache key. These keys were previously not - deterministically ordered, meaning the same statement could be - cached multiple times on equivalent keys, costing both in terms of - memory as well as performance. - - .. change:: - :tags: bug, postgresql - :versions: 1.0.0b1 - :tickets: 3159 - - Fixed bug where PostgreSQL JSON type was not able to persist or - otherwise render a SQL NULL column value, rather than a JSON-encoded - ``'null'``. To support this case, changes are as follows: - - * The value :func:`.null` can now be specified, which will always - result in a NULL value resulting in the statement. - - * A new parameter :paramref:`_types.JSON.none_as_null` is added, which - when True indicates that the Python ``None`` value should be - persisted as SQL NULL, rather than JSON-encoded ``'null'``. - - Retrieval of NULL as None is also repaired for DBAPIs other than - psycopg2, namely pg8000. - - .. change:: - :tags: bug, sql - :versions: 1.0.0b1 - :tickets: 3154 - - Fixed bug in CTE where ``literal_binds`` compiler argument would not - be always be correctly propagated when one CTE referred to another - aliased CTE in a statement. - - .. change:: - :tags: bug, postgresql - :versions: 1.0.0b1 - :tickets: 3075 - - The exception wrapping system for DBAPI errors can now accommodate - non-standard DBAPI exceptions, such as the psycopg2 - TransactionRollbackError. These exceptions will now be raised - using the closest available subclass in ``sqlalchemy.exc``, in the - case of TransactionRollbackError, ``sqlalchemy.exc.OperationalError``. - - .. change:: - :tags: bug, sql - :versions: 1.0.0b1 - :tickets: 3144, 3067 - - Fixed 0.9.7 regression caused by :ticket:`3067` in conjunction with - a mis-named unit test such that so-called "schema" types like - :class:`.Boolean` and :class:`.Enum` could no longer be pickled. - - .. change:: - :tags: bug, postgresql - :versions: 1.0.0b1 - :tickets: 3141 - - Fixed bug in :class:`_postgresql.array` object where comparison - to a plain Python list would fail to use the correct array constructor. - Pull request courtesy Andrew. - - .. change:: - :tags: bug, postgresql - :versions: 1.0.0b1 - :tickets: 3137 - - Added a supported :meth:`.FunctionElement.alias` method to functions, - e.g. the ``func`` construct. Previously, behavior for this method - was undefined. The current behavior mimics that of pre-0.9.4, - which is that the function is turned into a single-column FROM - clause with the given alias name, where the column itself is - anonymously named. - -.. changelog:: - :version: 0.9.7 - :released: July 22, 2014 - - .. change:: - :tags: bug, postgresql, pg8000 - :tickets: 3134 - :versions: 1.0.0b1 - - Fixed bug introduced in 0.9.5 by new pg8000 isolation level feature - where engine-level isolation level parameter would raise an error - on connect. - - .. change:: - :tags: bug, oracle, tests - :tickets: 3128 - :versions: 1.0.0b1 - - Fixed bug in oracle dialect test suite where in one test, - 'username' was assumed to be in the database URL, even though - this might not be the case. - - .. change:: - :tags: bug, orm, eagerloading - :tickets: 3131 - :versions: 1.0.0b1 - - Fixed a regression caused by :ticket:`2976` released in 0.9.4 where - the "outer join" propagation along a chain of joined eager loads - would incorrectly convert an "inner join" along a sibling join path - into an outer join as well, when only descendant paths should be - receiving the "outer join" propagation; additionally, fixed related - issue where "nested" join propagation would take place inappropriately - between two sibling join paths. - - .. change:: - :tags: bug, sqlite - :tickets: 3130 - :versions: 1.0.0b1 - - Fixed a SQLite join rewriting issue where a subquery that is embedded - as a scalar subquery such as within an IN would receive inappropriate - substitutions from the enclosing query, if the same table were present - inside the subquery as were in the enclosing query such as in a - joined inheritance scenario. - - .. change:: - :tags: bug, sql - :tickets: 3067 - :versions: 1.0.0b1 - - Fix bug in naming convention feature where using a check - constraint convention that includes ``constraint_name`` would - then force all :class:`.Boolean` and :class:`.Enum` types to - require names as well, as these implicitly create a - constraint, even if the ultimate target backend were one that does - not require generation of the constraint such as PostgreSQL. - The mechanics of naming conventions for these particular - constraints has been reorganized such that the naming - determination is done at DDL compile time, rather than at - constraint/table construction time. - - .. change:: - :tags: bug, mssql - :tickets: 3025 - - Fixed a regression from 0.9.5 caused by :ticket:`3025` where the - query used to determine "default schema" is invalid in SQL Server 2000. - For SQL Server 2000 we go back to defaulting to the "schema name" - parameter of the dialect, which is configurable but defaults - to 'dbo'. - - .. change:: - :tags: bug, orm - :tickets: 3083, 2736 - :versions: 1.0.0b1 - - Fixed a regression from 0.9.0 due to :ticket:`2736` where the - :meth:`_query.Query.select_from` method no longer set up the "from - entity" of the :class:`_query.Query` object correctly, so that - subsequent :meth:`_query.Query.filter_by` or :meth:`_query.Query.join` - calls would fail to check the appropriate "from" entity when - searching for attributes by string name. - - .. change:: - :tags: bug, sql - :tickets: 3090 - :versions: 1.0.0b1 - - Fixed bug in common table expressions whereby positional bound - parameters could be expressed in the wrong final order - when CTEs were nested in certain ways. - - .. change:: - :tags: bug, sql - :tickets: 3069 - :versions: 1.0.0b1 - - Fixed bug where multi-valued :class:`_expression.Insert` construct would fail - to check subsequent values entries beyond the first one given - for literal SQL expressions. - - .. change:: - :tags: bug, sql - :tickets: 3123 - :versions: 1.0.0b1 - - Added a "str()" step to the dialect_kwargs iteration for - Python version < 2.6.5, working around the - "no unicode keyword arg" bug as these args are passed along as - keyword args within some reflection processes. - - .. change:: - :tags: bug, sql - :tickets: 3122 - :versions: 1.0.0b1 - - The :meth:`.TypeEngine.with_variant` method will now accept a - type class as an argument which is internally converted to an - instance, using the same convention long established by other - constructs such as :class:`_schema.Column`. - - .. change:: - :tags: bug, orm - :tickets: 3117 - - The "evaluator" for query.update()/delete() won't work with multi-table - updates, and needs to be set to `synchronize_session=False` or - `synchronize_session='fetch'`; a warning is now emitted. In - 1.0 this will be promoted to a full exception. - - .. change:: - :tags: bug, tests - :versions: 1.0.0b1 - - Fixed bug where "python setup.py test" wasn't calling into - distutils appropriately, and errors would be emitted at the end - of the test suite. - - .. change:: - :tags: feature, postgresql - :versions: 1.0.0b1 - :tickets: 3078 - - Added kw argument ``postgresql_regconfig`` to the - :meth:`.ColumnOperators.match` operator, allows the "reg config" argument - to be specified to the ``to_tsquery()`` function emitted. - Pull request courtesy Jonathan Vanasco. - - .. change:: - :tags: feature, postgresql - :versions: 1.0.0b1 - - Added support for PostgreSQL JSONB via :class:`_postgresql.JSONB`. Pull request - courtesy Damian Dimmich. - - .. change:: - :tags: feature, mssql - :versions: 1.0.0b1 - - Enabled "multivalues insert" for SQL Server 2008. Pull request - courtesy Albert Cervin. Also expanded the checks for "IDENTITY INSERT" - mode to include when the identity key is present in the - VALUEs clause of the statement. - - .. change:: - :tags: feature, engine - :tickets: 3076 - :versions: 1.0.0b1 - - Added new event :meth:`_events.ConnectionEvents.handle_error`, a more - fully featured and comprehensive replacement for - :meth:`_events.ConnectionEvents.dbapi_error`. - - .. change:: - :tags: bug, orm - :tickets: 3108 - :versions: 1.0.0b1 - - Fixed bug where items that were persisted, deleted, or had a - primary key change within a savepoint block would not - participate in being restored to their former state (not in - session, in session, previous PK) after the outer transaction - were rolled back. - - .. change:: - :tags: bug, orm - :tickets: 3106 - :versions: 1.0.0b1 - - Fixed bug in subquery eager loading in conjunction with - :func:`.with_polymorphic`, the targeting of entities and columns - in the subquery load has been made more accurate with respect - to this type of entity and others. - - .. change:: - :tags: bug, orm - :tickets: 3099 - - Fixed bug involving dynamic attributes, that was again a regression - of :ticket:`3060` from version 0.9.5. A self-referential relationship - with lazy='dynamic' would raise a TypeError within a flush operation. - - .. change:: - :tags: bug, declarative - :tickets: 3097 - :versions: 1.0.0b1 - - Fixed bug when the declarative ``__abstract__`` flag was not being - distinguished for when it was actually the value ``False``. - The ``__abstract__`` flag needs to actually evaluate to a True - value at the level being tested. - -.. changelog:: - :version: 0.9.6 - :released: June 23, 2014 - - .. change:: - :tags: bug, orm - :tickets: 3060 - - Reverted the change for :ticket:`3060` - this is a unit of work - fix that is updated more comprehensively in 1.0 via :ticket:`3061`. - The fix in :ticket:`3060` unfortunately produces a new issue whereby - an eager load of a many-to-one attribute can produce an event - that is interpreted into an attribute change. - -.. changelog:: - :version: 0.9.5 - :released: June 23, 2014 - - .. change:: - :tags: bug, orm - :tickets: 3042 - :versions: 1.0.0b1 - - Additional checks have been added for the case where an inheriting - mapper is implicitly combining one of its column-based attributes - with that of the parent, where those columns normally don't necessarily - share the same value. This is an extension of an existing check that - was added via :ticket:`1892`; however this new check emits only a - warning, instead of an exception, to allow for applications that may - be relying upon the existing behavior. - - .. seealso:: - - :ref:`faq_combining_columns` - - .. change:: - :tags: bug, sql - :tickets: 3023 - :versions: 1.0.0b1 - - The :paramref:`_schema.Column.nullable` flag is implicitly set to ``False`` - when that :class:`_schema.Column` is referred to in an explicit - :class:`.PrimaryKeyConstraint` for that table. This behavior now - matches that of when the :class:`_schema.Column` itself has the - :paramref:`_schema.Column.primary_key` flag set to ``True``, which is - intended to be an exactly equivalent case. - - .. change:: - :tags: enhancement, postgresql - :tickets: 3002 - :versions: 1.0.0b1 - - Added a new type :class:`_postgresql.OID` to the PostgreSQL dialect. - While "oid" is generally a private type within PG that is not exposed - in modern versions, there are some PG use cases such as large object - support where these types might be exposed, as well as within some - user-reported schema reflection use cases. - - .. change:: - :tags: bug, orm - :tickets: 3080 - :versions: 1.0.0b1 - - Modified the behavior of :func:`_orm.load_only` such that primary key - columns are always added to the list of columns to be "undeferred"; - otherwise, the ORM can't load the row's identity. Apparently, - one can defer the mapped primary keys and the ORM will fail, that - hasn't been changed. But as load_only is essentially saying - "defer all but X", it's more critical that PK cols not be part of this - deferral. - - .. change:: - :tags: feature, examples - :versions: 1.0.0b1 - - Added a new example illustrating materialized paths, using the - latest relationship features. Example courtesy Jack Zhou. - - .. change:: - :tags: bug, testsuite - :versions: 1.0.0b1 - - In public test suite, changed to use of ``String(40)`` from - less-supported ``Text`` in ``StringTest.test_literal_backslashes``. - Pullreq courtesy Jan. - - .. change:: - :tags: bug, engine - :versions: 1.0.0b1 - :tickets: 3063 - - Fixed bug which would occur if a DBAPI exception - occurs when the engine first connects and does its initial checks, - and the exception is not a disconnect exception, yet the cursor - raises an error when we try to close it. In this case the real - exception would be quashed as we tried to log the cursor close - exception via the connection pool and failed, as we were trying - to access the pool's logger in a way that is inappropriate - in this very specific scenario. - - .. change:: - :tags: feature, postgresql - :versions: 1.0.0b1 - - Added support for AUTOCOMMIT isolation level when using the pg8000 - DBAPI. Pull request courtesy Tony Locke. - - .. change:: - :tags: bug, postgresql - :tickets: 3021 - :versions: 1.0.0b1 - - The psycopg2 ``.closed`` accessor is now consulted when determining - if an exception is a "disconnect" error; ideally, this should remove - the need for any other inspection of the exception message to detect - disconnect, however we will leave those existing messages in place - as a fallback. This should be able to handle newer cases like - "SSL EOF" conditions. Pull request courtesy Dirk Mueller. - - .. change:: - :tags: bug, orm - :tickets: 3060 - :versions: 1.0.0b1 - - Fixed a few edge cases which arise in the so-called "row switch" - scenario, where an INSERT/DELETE can be turned into an UPDATE. - In this situation, a many-to-one relationship set to None, or - in some cases a scalar attribute set to None, may not be detected - as a net change in value, and therefore the UPDATE would not reset - what was on the previous row. This is due to some as-yet - unresolved side effects of the way attribute history works in terms - of implicitly assuming None isn't really a "change" for a previously - un-set attribute. See also :ticket:`3061`. - - .. note:: - - This change has been **REVERTED** in 0.9.6. The full fix - will be in version 1.0 of SQLAlchemy. - - - .. change:: - :tags: bug, orm - :versions: 1.0.0b1 - - Related to :ticket:`3060`, an adjustment has been made to the unit - of work such that loading for related many-to-one objects is slightly - more aggressive, in the case of a graph of self-referential objects - that are to be deleted; the load of related objects is to help - determine the correct order for deletion if passive_deletes is - not set. - - .. change:: - :tags: bug, orm - :tickets: 3057 - :versions: 1.0.0b1 - - Fixed bug in SQLite join rewriting where anonymized column names - due to repeats would not correctly be rewritten in subqueries. - This would affect SELECT queries with any kind of subquery + join. - - .. change:: - :tags: bug, sql - :tickets: 3012 - :versions: 1.0.0b1 - - Fixed bug where the :meth:`.Operators.__and__`, - :meth:`.Operators.__or__` and :meth:`.Operators.__invert__` - operator overload methods could not be overridden within a custom - :class:`.TypeEngine.Comparator` implementation. - - .. change:: - :tags: feature, postgresql - :tickets: 2785 - :versions: 1.0.0b1 - - Added a new flag :paramref:`_types.ARRAY.zero_indexes` to the PostgreSQL - :class:`_types.ARRAY` type. When set to ``True``, a value of one will be - added to all array index values before passing to the database, allowing - better interoperability between Python style zero-based indexes and - PostgreSQL one-based indexes. Pull request courtesy Alexey Terentev. - - .. change:: - :tags: bug, engine - :tickets: 3043 - :versions: 1.0.0b1 - - Fixed some "double invalidate" situations were detected where - a connection invalidation could occur within an already critical section - like a connection.close(); ultimately, these conditions are caused - by the change in :ticket:`2907`, in that the "reset on return" feature - calls out to the Connection/Transaction in order to handle it, where - "disconnect detection" might be caught. However, it's possible that - the more recent change in :ticket:`2985` made it more likely for this - to be seen as the "connection invalidate" operation is much quicker, - as the issue is more reproducible on 0.9.4 than 0.9.3. - - Checks are now added within any section that - an invalidate might occur to halt further disallowed operations - on the invalidated connection. This includes two fixes both at the - engine level and at the pool level. While the issue was observed - with highly concurrent gevent cases, it could in theory occur in - any kind of scenario where a disconnect occurs within the connection - close operation. - - .. change:: - :tags: feature, orm - :tickets: 3029 - :versions: 1.0.0b1 - - The "primaryjoin" model has been stretched a bit further to allow - a join condition that is strictly from a single column to itself, - translated through some kind of SQL function or expression. This - is kind of experimental, but the first proof of concept is a - "materialized path" join condition where a path string is compared - to itself using "like". The :meth:`.ColumnOperators.like` operator has - also been added to the list of valid operators to use in a primaryjoin - condition. - - .. change:: - :tags: feature, sql - :tickets: 3028 - :versions: 1.0.0b1 - - Liberalized the contract for :class:`.Index` a bit in that you can - specify a :func:`_expression.text` expression as the target; the index no longer - needs to have a table-bound column present if the index is to be - manually added to the table, either via inline declaration or via - :meth:`_schema.Table.append_constraint`. - - .. change:: - :tags: bug, firebird - :tickets: 3038 - - Fixed bug where the combination of "limit" rendering as - "SELECT FIRST n ROWS" using a bound parameter (only firebird has both), - combined with column-level subqueries - which also feature "limit" as well as "positional" bound parameters - (e.g. qmark style) would erroneously assign the subquery-level positions - before that of the enclosing SELECT, thus returning parameters which - are out of order. - - .. change:: - :tags: bug, mssql - :tickets: 3025 - :versions: 1.0.0b1 - - Revised the query used to determine the current default schema name - to use the ``database_principal_id()`` function in conjunction with - the ``sys.database_principals`` view so that we can determine - the default schema independently of the type of login in progress - (e.g., SQL Server, Windows, etc). - - .. change:: - :tags: bug, sql - :tickets: 3024 - :versions: 1.0.0b1 - - Fixed bug in new :meth:`.DialectKWArgs.argument_for` method where - adding an argument for a construct not previously included for any - special arguments would fail. - - .. change:: - :tags: bug, py3k, tests - :tickets: 2830 - :versions: 1.0.0b1 - - Corrected for some deprecation warnings involving the ``imp`` - module and Python 3.3 or greater, when running tests. Pull - request courtesy Matt Chisholm. - - .. change:: - :tags: bug, sql - :tickets: 3020, 1068 - :versions: 1.0.0b1 - - Fixed regression introduced in 0.9 where new "ORDER BY " - feature from :ticket:`1068` would not apply quoting rules to the - label name as rendered in the ORDER BY. - - .. change:: - :tags: feature, orm - :tickets: 3017 - :versions: 1.0.0b1 - - Added new utility function :func:`.make_transient_to_detached` which can - be used to manufacture objects that behave as though they were loaded - from a session, then detached. Attributes that aren't present - are marked as expired, and the object can be added to a Session - where it will act like a persistent one. - - .. change:: - :tags: bug, sql - :versions: 1.0.0b1 - - Restored the import for :class:`.Function` to the ``sqlalchemy.sql.expression`` - import namespace, which was removed at the beginning of 0.9. - - .. change:: - :tags: bug, orm, sql - :tickets: 3013 - :versions: 1.0.0b1 - - Fixes to the newly enhanced boolean coercion in :ticket:`2804` where - the new rules for "where" and "having" wouldn't take effect for the - "whereclause" and "having" kw arguments of the :func:`_expression.select` construct, - which is also what :class:`_query.Query` uses so wasn't working in the - ORM either. - - .. change:: - :tags: feature, sql - :tickets: 2990 - :versions: 1.0.0b1 - - Added new flag :paramref:`.expression.between.symmetric`, when set to True - renders "BETWEEN SYMMETRIC". Also added a new negation operator - "notbetween_op", which now allows an expression like ``~col.between(x, y)`` - to render as "col NOT BETWEEN x AND y", rather than a parenthesized NOT - string. - -.. changelog:: - :version: 0.9.4 - :released: March 28, 2014 - - .. change:: - :tags: feature, orm - :tickets: 3007 - - Added new parameter :paramref:`.orm.mapper.confirm_deleted_rows`. Defaults - to True, indicates that a series of DELETE statements should confirm - that the cursor rowcount matches the number of primary keys that should - have matched; this behavior had been taken off in most cases - (except when version_id is used) to support the unusual edge case of - self-referential ON DELETE CASCADE; to accommodate this, the message - is now just a warning, not an exception, and the flag can be used - to indicate a mapping that expects self-referential cascaded - deletes of this nature. See also :ticket:`2403` for background on the - original change. - - .. change:: - :tags: bug, ext, automap - :tickets: 3004 - - Added support to automap for the case where a relationship should - not be created between two classes that are in a joined inheritance - relationship, for those foreign keys that link the subclass back to - the superclass. - - .. change:: - :tags: bug, orm - :tickets: 2948 - - Fixed a very old behavior where the lazy load emitted for a one-to-many - could inappropriately pull in the parent table, and also return results - inconsistent based on what's in the parent table, when the primaryjoin - includes some kind of discriminator against the parent table, such - as ``and_(parent.id == child.parent_id, parent.deleted == False)``. - While this primaryjoin doesn't make that much sense for a one-to-many, - it is slightly more common when applied to the many-to-one side, and - the one-to-many comes as a result of a backref. - Loading rows from ``child`` in this case would keep ``parent.deleted == False`` - as is within the query, thereby yanking it into the FROM clause - and doing a cartesian product. The new behavior will now substitute - the value of the local "parent.deleted" for that parameter as is - appropriate. Though typically, a real-world app probably wants to use a - different primaryjoin for the o2m side in any case. - - .. change:: - :tags: bug, orm - :tickets: 2965 - - Improved the check for "how to join from A to B" such that when - a table has multiple, composite foreign keys targeting a parent table, - the :paramref:`_orm.relationship.foreign_keys` argument will be properly - interpreted in order to resolve the ambiguity; previously this condition - would raise that there were multiple FK paths when in fact the - foreign_keys argument should be establishing which one is expected. - - .. change:: - :tags: bug, mysql - - Tweaked the settings for mysql-connector-python; in Py2K, the - "supports unicode statements" flag is now False, so that SQLAlchemy - will encode the *SQL string* (note: *not* the parameters) - to bytes before sending to the database. This seems to allow - all unicode-related tests to pass for mysql-connector, including those - that use non-ascii table/column names, as well as some tests for the - TEXT type using unicode under cursor.executemany(). - - .. change:: - :tags: feature, engine - - Added some new event mechanics for dialect-level events; the initial - implementation allows an event handler to redefine the specific mechanics - by which an arbitrary dialect invokes execute() or executemany() on a - DBAPI cursor. The new events, at this point semi-public and experimental, - are in support of some upcoming transaction-related extensions. - - .. change:: - :tags: feature, engine - :tickets: 2978 - - An event listener can now be associated with a :class:`_engine.Engine`, - after one or more :class:`_engine.Connection` objects have been created - (such as by an orm :class:`.Session` or via explicit connect) - and the listener will pick up events from those connections. - Previously, performance concerns pushed the event transfer from - :class:`_engine.Engine` to :class:`_engine.Connection` at init-time only, but - we've inlined a bunch of conditional checks to make this possible - without any additional function calls. - - .. change:: - :tags: bug, tests - :tickets: 2980 - - Fixed a few errant ``u''`` strings that would prevent tests from passing - in Py3.2. Patch courtesy Arfrever Frehtes Taifersar Arahesis. - - .. change:: - :tags: bug, engine - :tickets: 2985 - - A major improvement made to the mechanics by which the :class:`_engine.Engine` - recycles the connection pool when a "disconnect" condition is detected; - instead of discarding the pool and explicitly closing out connections, - the pool is retained and a "generational" timestamp is updated to - reflect the current time, thereby causing all existing connections - to be recycled when they are next checked out. This greatly simplifies - the recycle process, removes the need for "waking up" connect attempts - waiting on the old pool and eliminates the race condition that many - immediately-discarded "pool" objects could be created during the - recycle operation. - - .. change:: - :tags: bug, oracle - :tickets: 2987 - - Added new datatype :class:`_oracle.DATE`, which is a subclass of - :class:`.DateTime`. As Oracle has no "datetime" type per se, - it instead has only ``DATE``, it is appropriate here that the - ``DATE`` type as present in the Oracle dialect be an instance of - :class:`.DateTime`. This issue doesn't change anything as far as - the behavior of the type, as data conversion is handled by the - DBAPI in any case, however the improved subclass layout will help - the use cases of inspecting types for cross-database compatibility. - Also removed uppercase ``DATETIME`` from the Oracle dialect as this - type isn't functional in that context. - - .. change:: - :tags: bug, sql - :tickets: 2988 - - Fixed an 0.9 regression where a :class:`_schema.Table` that failed to - reflect correctly wouldn't be removed from the parent - :class:`_schema.MetaData`, even though in an invalid state. Pullreq - courtesy Roman Podoliaka. - - .. change:: - :tags: bug, engine - - The :meth:`_events.ConnectionEvents.after_cursor_execute` event is now - emitted for the "_cursor_execute()" method of :class:`_engine.Connection`; - this is the "quick" executor that is used for things like - when a sequence is executed ahead of an INSERT statement, as well as - for dialect startup checks like unicode returns, charset, etc. - the :meth:`_events.ConnectionEvents.before_cursor_execute` event was already - invoked here. The "executemany" flag is now always set to False - here, as this event always corresponds to a single execution. - Previously the flag could be True if we were acting on behalf of - an executemany INSERT statement. - - .. change:: - :tags: bug, orm - - Added support for the not-quite-yet-documented ``insert=True`` - flag for :func:`.event.listen` to work with mapper / instance events. - - .. change:: - :tags: feature, sql - - Added support for literal rendering of boolean values, e.g. - "true" / "false" or "1" / "0". - - .. change:: - :tags: feature, sql - - Added a new feature :func:`_schema.conv`, the purpose of which is to - mark a constraint name as already having had a naming convention applied. - This token will be used by Alembic migrations as of Alembic 0.6.4 - in order to render constraints in migration scripts with names marked - as already having been subject to a naming convention. - - .. change:: - :tags: bug, sql - - :paramref:`_schema.MetaData.naming_convention` feature will now also - apply to :class:`.CheckConstraint` objects that are associated - directly with a :class:`_schema.Column` instead of just on the - :class:`_schema.Table`. - - .. change:: - :tags: bug, sql - :tickets: 2991 - - Fixed bug in new :paramref:`_schema.MetaData.naming_convention` feature - where the name of a check constraint making use of the - `"%(constraint_name)s"` token would get doubled up for the - constraint generated by a boolean or enum type, and overall - duplicate events would cause the `"%(constraint_name)s"` token - to keep compounding itself. - - .. change:: - :tags: feature, orm - - A warning is emitted if the :meth:`.MapperEvents.before_configured` - or :meth:`.MapperEvents.after_configured` events are applied to a - specific mapper or mapped class, as the events are only invoked - for the :class:`_orm.Mapper` target at the general level. - - .. change:: - :tags: feature, orm - - Added a new keyword argument ``once=True`` to :func:`.event.listen` - and :func:`.event.listens_for`. This is a convenience feature which - will wrap the given listener such that it is only invoked once. - - .. change:: - :tags: feature, oracle - :tickets: 2911 - - Added a new engine option ``coerce_to_unicode=True`` to the - cx_Oracle dialect, which restores the cx_Oracle outputtypehandler - approach to Python unicode conversion under Python 2, which was - removed in 0.9.2 as a result of :ticket:`2911`. Some use cases would - prefer that unicode coercion is unconditional for all string values, - despite performance concerns. Pull request courtesy - Christoph Zwerschke. - - .. change:: - :tags: bug, pool - - Fixed small issue in :class:`.SingletonThreadPool` where the current - connection to be returned might get inadvertently cleaned out during - the "cleanup" process. Patch courtesy jd23. - - .. change:: - :tags: bug, ext, py3k - - Fixed bug in association proxy where assigning an empty slice - (e.g. ``x[:] = [...]``) would fail on Py3k. - - .. change:: - :tags: bug, general - :tickets: 2979 - - Fixed some test/feature failures occurring in Python 3.4, - in particular the logic used to wrap "column default" callables - wouldn't work properly for Python built-ins. - - .. change:: - :tags: feature, general - - Support has been added for pytest to run tests. This runner - is currently being supported in addition to nose, and will likely - be preferred to nose going forward. The nose plugin system used - by SQLAlchemy has been split out so that it works under pytest as - well. There are no plans to drop support for nose at the moment - and we hope that the test suite itself can continue to remain as - agnostic of testing platform as possible. See the file - README.unittests.rst for updated information on running tests - with pytest. - - The test plugin system has also been enhanced to support running - tests against multiple database URLs at once, by specifying the ``--db`` - and/or ``--dburi`` flags multiple times. This does not run the entire test - suite for each database, but instead allows test cases that are specific - to certain backends make use of that backend as the test is run. - When using pytest as the test runner, the system will also run - specific test suites multiple times, once for each database, particularly - those tests within the "dialect suite". The plan is that the enhanced - system will also be used by Alembic, and allow Alembic to run - migration operation tests against multiple backends in one run, including - third-party backends not included within Alembic itself. - Third party dialects and extensions are also encouraged to standardize - on SQLAlchemy's test suite as a basis; see the file README.dialects.rst - for background on building out from SQLAlchemy's test platform. - - .. change:: - :tags: feature, orm - :tickets: 2976 - - Added a new option to :paramref:`_orm.relationship.innerjoin` which is - to specify the string ``"nested"``. When set to ``"nested"`` as opposed - to ``True``, the "chaining" of joins will parenthesize the inner join on the - right side of an existing outer join, instead of chaining as a string - of outer joins. This possibly should have been the default behavior - when 0.9 was released, as we introduced the feature of right-nested - joins in the ORM, however we are keeping it as a non-default for now - to avoid further surprises. - - .. seealso:: - - :ref:`feature_2976` - - .. change:: - :tags: bug, ext - :tickets: 2810 - - Fixed a regression in association proxy caused by :ticket:`2810` which - caused a user-provided "getter" to no longer receive values of ``None`` - when fetching scalar values from a target that is non-present. The - check for None introduced by this change is now moved into the default - getter, so a user-provided getter will also again receive values of - None. - - .. change:: - :tags: bug, sql - :tickets: 2974 - - Adjusted the logic which applies names to the .c collection when - a no-name :class:`.BindParameter` is received, e.g. via :func:`_expression.literal` - or similar; the "key" of the bind param is used as the key within - .c. rather than the rendered name. Since these binds have "anonymous" - names in any case, this allows individual bound parameters to - have their own name within a selectable if they are otherwise unlabeled. - - .. change:: - :tags: bug, sql - :tickets: 2974 - - Some changes to how the :attr:`_expression.FromClause.c` collection behaves - when presented with duplicate columns. The behavior of emitting a - warning and replacing the old column with the same name still - remains to some degree; the replacement in particular is to maintain - backwards compatibility. However, the replaced column still remains - associated with the ``c`` collection now in a collection ``._all_columns``, - which is used by constructs such as aliases and unions, to deal with - the set of columns in ``c`` more towards what is actually in the - list of columns rather than the unique set of key names. This helps - with situations where SELECT statements with same-named columns - are used in unions and such, so that the union can match the columns - up positionally and also there's some chance of :meth:`_expression.FromClause.corresponding_column` - still being usable here (it can now return a column that is only - in selectable.c._all_columns and not otherwise named). - The new collection is underscored as we still need to decide where this - list might end up. Theoretically it - would become the result of iter(selectable.c), however this would mean - that the length of the iteration would no longer match the length of - keys(), and that behavior needs to be checked out. - - .. change:: - :tags: bug, sql - - Fixed issue in new :meth:`_expression.TextClause.columns` method where the ordering - of columns given positionally would not be preserved. This could - have potential impact in positional situations such as applying the - resulting :class:`.TextAsFrom` object to a union. - - .. change:: - :tags: feature, sql - :tickets: 2962, 2866 - - The new dialect-level keyword argument system for schema-level - constructs has been enhanced in order to assist with existing - schemes that rely upon addition of ad-hoc keyword arguments to - constructs. - - E.g., a construct such as :class:`.Index` will again accept - ad-hoc keyword arguments within the :attr:`.Index.kwargs` collection, - after construction:: - - idx = Index("a", "b") - idx.kwargs["mysql_someargument"] = True - - To suit the use case of allowing custom arguments at construction time, - the :meth:`.DialectKWArgs.argument_for` method now allows this registration:: - - Index.argument_for("mysql", "someargument", False) - - idx = Index("a", "b", mysql_someargument=True) - - .. seealso:: - - :meth:`.DialectKWArgs.argument_for` - - .. change:: - :tags: bug, orm, engine - :tickets: 2973 - - Fixed bug where events set to listen at the class - level (e.g. on the :class:`_orm.Mapper` or :class:`.ClassManager` - level, as opposed to on an individual mapped class, and also on - :class:`_engine.Connection`) that also made use of internal argument conversion - (which is most within those categories) would fail to be removable. - - .. change:: - :tags: bug, orm - - Fixed regression from 0.8 where using an option like - :func:`_orm.lazyload` with the "wildcard" expression, e.g. ``"*"``, - would raise an assertion error in the case where the query didn't - contain any actual entities. This assertion is meant for other cases - and was catching this one inadvertently. - - .. change:: - :tags: bug, examples - - Fixed bug in the versioned_history example where column-level INSERT - defaults would prevent history values of NULL from being written. - - .. change:: - :tags: orm, bug, sqlite - :tickets: 2969 - - More fixes to SQLite "join rewriting"; the fix from :ticket:`2967` - implemented right before the release of 0.9.3 affected the case where - a UNION contained nested joins in it. "Join rewriting" is a feature - with a wide range of possibilities and is the first intricate - "SQL rewriting" feature we've introduced in years, so we're sort of - going through a lot of iterations with it (not unlike eager loading - back in the 0.2/0.3 series, polymorphic loading in 0.4/0.5). We should - be there soon so thanks for bearing with us :). - - -.. changelog:: - :version: 0.9.3 - :released: February 19, 2014 - - .. change:: - :tags: orm, bug, sqlite - :tickets: 2967 - - Fixed bug in SQLite "join rewriting" where usage of an exists() construct - would fail to be rewritten properly, such as when the exists is - mapped to a column_property in an intricate nested-join scenario. - Also fixed a somewhat related issue where join rewriting would fail - on the columns clause of the SELECT statement if the targets were - aliased tables, as opposed to individual aliased columns. - - .. change:: - :tags: sqlite, bug - - The SQLite dialect will now skip unsupported arguments when reflecting - types; such as if it encounters a string like ``INTEGER(5)``, the - :class:`_types.INTEGER` type will be instantiated without the "5" being included, - based on detecting a ``TypeError`` on the first attempt. - - .. change:: - :tags: sqlite, bug - - Support has been added to SQLite type reflection to fully support - the "type affinity" contract specified at https://www.sqlite.org/datatype3.html. - In this scheme, keywords like ``INT``, ``CHAR``, ``BLOB`` or - ``REAL`` located in the type name generically associate the type with - one of five affinities. Pull request courtesy Erich Blume. - - .. seealso:: - - :ref:`sqlite_type_reflection` - - .. change:: - :tags: postgresql, feature - - Added the :attr:`.TypeEngine.python_type` convenience accessor onto the - :class:`_postgresql.ARRAY` type. Pull request courtesy Alexey Terentev. - - .. change:: - :tags: examples, feature - - Added optional "changed" column to the versioned rows example, as well - as support for when the versioned :class:`_schema.Table` has an explicit - :paramref:`_schema.Table.schema` argument. Pull request - courtesy jplaverdure. - - .. change:: - :tags: bug, postgresql - :tickets: 2946 - - Added server version detection to the newly added dialect startup - query for "show standard_conforming_strings"; as this variable was - added as of PG 8.2, we skip the query for PG versions who report a - version string earlier than that. - - .. change:: - :tags: bug, orm, declarative - :tickets: 2950 - - Fixed bug where :class:`.AbstractConcreteBase` would fail to be - fully usable within declarative relationship configuration, as its - string classname would not be available in the registry of classnames - at mapper configuration time. The class now explicitly adds itself - to the class registry, and additionally both :class:`.AbstractConcreteBase` - as well as :class:`.ConcreteBase` set themselves up *before* mappers - are configured within the :func:`.configure_mappers` setup, using - the new :meth:`.MapperEvents.before_configured` event. - - .. change:: - :tags: feature, orm - - Added new :meth:`.MapperEvents.before_configured` event which allows - an event at the start of :func:`.configure_mappers`, as well - as ``__declare_first__()`` hook within declarative to complement - ``__declare_last__()``. - - .. change:: - :tags: bug, mysql, cymysql - :tickets: 2934 - - Fixed bug in cymysql dialect where a version string such as - ``'33a-MariaDB'`` would fail to parse properly. Pull request - courtesy Matt Schmidt. - - .. change:: - :tags: bug, orm - :tickets: 2949 - - Fixed an 0.9 regression where ORM instance or mapper events applied - to a base class such as a declarative base with the propagate=True - flag would fail to apply to existing mapped classes which also - used inheritance due to an assertion. Additionally, repaired an - attribute error which could occur during removal of such an event, - depending on how it was first assigned. - - .. change:: - :tags: bug, ext - - Fixed bug where the :class:`.AutomapBase` class of the - new automap extension would fail if classes - were pre-arranged in single or potentially joined inheritance patterns. - The repaired joined inheritance issue could also potentially apply when - using :class:`.DeferredReflection` as well. - - - .. change:: - :tags: bug, sql - - Fixed regression in new "naming convention" feature where conventions - would fail if the referred table in a foreign key contained a schema - name. Pull request courtesy Thomas Farvour. - - .. change:: - :tags: bug, sql - - Fixed bug where so-called "literal render" of :func:`.bindparam` - constructs would fail if the bind were constructed with a callable, - rather than a direct value. This prevented ORM expressions - from being rendered with the "literal_binds" compiler flag. - - .. change:: - :tags: bug, orm - :tickets: 2935 - - Improved the initialization logic of composite attributes such that - calling ``MyClass.attribute`` will not require that the configure - mappers step has occurred, e.g. it will just work without throwing - any error. - - .. change:: - :tags: bug, orm - :tickets: 2932 - - More issues with [ticket:2932] first resolved in 0.9.2 where - using a column key of the form ``_`` - matching that of an aliased column in the text would still not - match at the ORM level, which is ultimately due to a core - column-matching issue. Additional rules have been added so that the - column ``_label`` is taken into account when working with a - :class:`.TextAsFrom` construct or with literal columns. - -.. changelog:: - :version: 0.9.2 - :released: February 2, 2014 - - .. change:: - :tags: bug, examples - - Added a tweak to the "history_meta" example where the check for - "history" on a relationship-bound attribute will now no longer emit - any SQL if the relationship is unloaded. - - .. change:: - :tags: feature, sql - - Added :paramref:`.MetaData.reflect.dialect_kwargs` - to support dialect-level reflection options for all :class:`_schema.Table` - objects reflected. - - .. change:: - :tags: feature, postgresql - :tickets: 2922 - - Added a new dialect-level argument ``postgresql_ignore_search_path``; - this argument is accepted by both the :class:`_schema.Table` constructor - as well as by the :meth:`_schema.MetaData.reflect` method. When in use - against PostgreSQL, a foreign-key referenced table which specifies - a remote schema name will retain that schema name even if the name - is present in the ``search_path``; the default behavior since 0.7.3 - has been that schemas present in ``search_path`` would not be copied - to reflected :class:`_schema.ForeignKey` objects. The documentation has been - updated to describe in detail the behavior of the ``pg_get_constraintdef()`` - function and how the ``postgresql_ignore_search_path`` feature essentially - determines if we will honor the schema qualification reported by - this function or not. - - .. seealso:: - - :ref:`postgresql_schema_reflection` - - .. change:: - :tags: bug, sql - :tickets: 2913 - - The behavior of :meth:`_schema.Table.tometadata` has been adjusted such that - the schema target of a :class:`_schema.ForeignKey` will not be changed unless - that schema matches that of the parent table. That is, if - a table "schema_a.user" has a foreign key to "schema_b.order.id", - the "schema_b" target will be maintained whether or not the - "schema" argument is passed to :meth:`_schema.Table.tometadata`. However - if a table "schema_a.user" refers to "schema_a.order.id", the presence - of "schema_a" will be updated on both the parent and referred tables. - This is a behavioral change hence isn't likely to be backported to - 0.8; it is assumed that the previous behavior is pretty buggy - however and that it's unlikely anyone was relying upon it. - - Additionally, a new parameter has been added - :paramref:`.Table.tometadata.referred_schema_fn`. This refers to a - callable function which will be used to determine the new referred - schema for any :class:`_schema.ForeignKeyConstraint` encountered in the - tometadata operation. This callable can be used to revert to the - previous behavior or to customize how referred schemas are treated - on a per-constraint basis. - - .. change:: - :tags: bug, orm - :tickets: 2932 - - Fixed bug in new :class:`.TextAsFrom` construct where :class:`_schema.Column`- - oriented row lookups were not matching up to the ad-hoc :class:`.ColumnClause` - objects that :class:`.TextAsFrom` generates, thereby making it not - usable as a target in :meth:`_query.Query.from_statement`. Also fixed - :meth:`_query.Query.from_statement` mechanics to not mistake a :class:`.TextAsFrom` - for a :class:`_expression.Select` construct. This bug is also an 0.9 regression - as the :meth:`_expression.TextClause.columns` method is called to accommodate the - :paramref:`_expression.text.typemap` argument. - - .. change:: - :tags: feature, sql - :tickets: 2923 - - Added a new feature which allows automated naming conventions to be - applied to :class:`.Constraint` and :class:`.Index` objects. Based - on a recipe in the wiki, the new feature uses schema-events to set up - names as various schema objects are associated with each other. The - events then expose a configuration system through a new argument - :paramref:`_schema.MetaData.naming_convention`. This system allows production - of both simple and custom naming schemes for constraints and indexes - on a per-:class:`_schema.MetaData` basis. - - .. seealso:: - - :ref:`constraint_naming_conventions` - - .. change:: - :tags: bug, orm - :tickets: 2921 - - Added a new directive used within the scope of an attribute "set" operation - to disable autoflush, in the case that the attribute needs to lazy-load - the "old" value, as in when replacing one-to-one values or some - kinds of many-to-one. A flush at this point otherwise occurs - at the point that the attribute is None and can cause NULL violations. - - .. change:: - :tags: feature, orm - - Added a new parameter :paramref:`.Operators.op.is_comparison`. This - flag allows a custom op from :meth:`.Operators.op` to be considered - as a "comparison" operator, thus usable for custom - :paramref:`_orm.relationship.primaryjoin` conditions. - - .. seealso:: - - :ref:`relationship_custom_operator` - - .. change:: - :tags: bug, sqlite - - Fixed bug whereby SQLite compiler failed to propagate compiler arguments - such as "literal binds" into a CAST expression. - - .. change:: - :tags: bug, sql - - Fixed bug whereby binary type would fail in some cases - if used with a "test" dialect, such as a DefaultDialect or other - dialect with no DBAPI. - - .. change:: - :tags: bug, sql, py3k - - Fixed bug where "literal binds" wouldn't work with a bound parameter - that's a binary type. A similar, but different, issue is fixed - in 0.8. - - .. change:: - :tags: bug, sql - :tickets: 2927 - - Fixed regression whereby the "annotation" system used by the ORM was leaking - into the names used by standard functions in :mod:`sqlalchemy.sql.functions`, - such as ``func.coalesce()`` and ``func.max()``. Using these functions - in ORM attributes and thus producing annotated versions of them could - corrupt the actual function name rendered in the SQL. - - .. change:: - :tags: bug, sql - :tickets: 2924, 2848 - - Fixed 0.9 regression where the new sortable support for :class:`.RowProxy` - would lead to ``TypeError`` when compared to non-tuple types as it attempted - to apply tuple() to the "other" object unconditionally. The - full range of Python comparison operators have now been implemented on - :class:`.RowProxy`, using an approach that guarantees a comparison - system that is equivalent to that of a tuple, and the "other" object - is only coerced if it's an instance of RowProxy. - - .. change:: - :tags: bug, orm - :tickets: 2918 - - Fixed an 0.9 regression where the automatic aliasing applied by - :class:`_query.Query` and in other situations where selects or joins - were aliased (such as joined table inheritance) could fail if a - user-defined :class:`_schema.Column` subclass were used in the expression. - In this case, the subclass would fail to propagate ORM-specific - "annotations" along needed by the adaptation. The "expression - annotations" system has been corrected to account for this case. - - .. change:: - :tags: feature, orm - - Support is improved for supplying a :func:`_expression.join` construct as the - target of :paramref:`_orm.relationship.secondary` for the purposes - of creating very complex :func:`_orm.relationship` join conditions. - The change includes adjustments to query joining, joined eager loading - to not render a SELECT subquery, changes to lazy loading such that - the "secondary" target is properly included in the SELECT, and - changes to declarative to better support specification of a - join() object with classes as targets. - - The new use case is somewhat experimental, but a new documentation section - has been added. - - .. seealso:: - - :ref:`composite_secondary_join` - - .. change:: - :tags: bug, mysql, sql - :tickets: 2917 - - Added new test coverage for so-called "down adaptions" of SQL types, - where a more specific type is adapted to a more generic one - this - use case is needed by some third party tools such as ``sqlacodegen``. - The specific cases that needed repair within this test suite were that - of :class:`.mysql.ENUM` being downcast into a :class:`_types.Enum`, - and that of SQLite date types being cast into generic date types. - The ``adapt()`` method needed to become more specific here to counteract - the removal of a "catch all" ``**kwargs`` collection on the base - :class:`.TypeEngine` class that was removed in 0.9. - - .. change:: - :tags: feature, sql - :tickets: 2910 - - Options can now be specified on a :class:`.PrimaryKeyConstraint` object - independently of the specification of columns in the table with - the ``primary_key=True`` flag; use a :class:`.PrimaryKeyConstraint` - object with no columns in it to achieve this result. - - Previously, an explicit :class:`.PrimaryKeyConstraint` would have the - effect of those columns marked as ``primary_key=True`` being ignored; - since this is no longer the case, the :class:`.PrimaryKeyConstraint` - will now assert that either one style or the other is used to specify - the columns, or if both are present, that the column lists match - exactly. If an inconsistent set of columns in the - :class:`.PrimaryKeyConstraint` - and within the :class:`_schema.Table` marked as ``primary_key=True`` are - present, a warning is emitted, and the list of columns is taken - only from the :class:`.PrimaryKeyConstraint` alone as was the case - in previous releases. - - - - .. seealso:: - - :class:`.PrimaryKeyConstraint` - - .. change:: - :tags: feature, sql - :tickets: 2866 - - The system by which schema constructs and certain SQL constructs - accept dialect-specific keyword arguments has been enhanced. This - system includes commonly the :class:`_schema.Table` and :class:`.Index` constructs, - which accept a wide variety of dialect-specific arguments such as - ``mysql_engine`` and ``postgresql_where``, as well as the constructs - :class:`.PrimaryKeyConstraint`, :class:`.UniqueConstraint`, - :class:`_expression.Update`, :class:`_expression.Insert` and :class:`_expression.Delete`, and also - newly added kwarg capability to :class:`_schema.ForeignKeyConstraint` - and :class:`_schema.ForeignKey`. The change is that participating dialects - can now specify acceptable argument lists for these constructs, allowing - an argument error to be raised if an invalid keyword is specified for - a particular dialect. If the dialect portion of the keyword is unrecognized, - a warning is emitted only; while the system will actually make use - of setuptools entrypoints in order to locate non-local dialects, - the use case where certain dialect-specific arguments are used - in an environment where that third-party dialect is uninstalled remains - supported. Dialects also have to explicitly opt-in to this system, - so that external dialects which aren't making use of this system - will remain unaffected. - - .. change:: - :tags: bug, sql - - A :class:`.UniqueConstraint` created inline with a :class:`_schema.Table` - that has no columns within it will be skipped. Pullreq courtesy - Derek Harland. - - .. change:: - :tags: feature, mssql - - Added an option ``mssql_clustered`` to the :class:`.UniqueConstraint` - and :class:`.PrimaryKeyConstraint` constructs; on SQL Server, this adds - the ``CLUSTERED`` keyword to the constraint construct within DDL. - Pullreq courtesy Derek Harland. - - .. change:: - :tags: bug, sql, orm - :tickets: 2912 - - Fixed the multiple-table "UPDATE..FROM" construct, only usable on - MySQL, to correctly render the SET clause among multiple columns - with the same name across tables. This also changes the name used for - the bound parameter in the SET clause to "_" for - the non-primary table only; as this parameter is typically specified - using the :class:`_schema.Column` object directly this should not have an - impact on applications. The fix takes effect for both - :meth:`_schema.Table.update` as well as :meth:`_query.Query.update` in the ORM. - - .. change:: - :tags: bug, oracle - :tickets: 2911 - - It's been observed that the usage of a cx_Oracle "outputtypehandler" - in Python 2.xx in order to coerce string values to Unicode is inordinately - expensive; even though cx_Oracle is written in C, when you pass the - Python ``unicode`` primitive to cursor.var() and associate with an output - handler, the library counts every conversion as a Python function call - with all the requisite overhead being recorded; this *despite* the fact - when running in Python 3, all strings are also unconditionally coerced - to unicode but it does *not* incur this overhead, - meaning that cx_Oracle is failing to use performant techniques in Py2K. - As SQLAlchemy cannot easily select for this style of type handler on a - per-column basis, the handler was assembled unconditionally thereby - adding the overhead to all string access. - - So this logic has been replaced with SQLAlchemy's own unicode - conversion system, which now - only takes effect in Py2K for columns that are requested as unicode. - When C extensions are used, SQLAlchemy's system appears to be 2-3x faster than - cx_Oracle's. Additionally, SQLAlchemy's unicode conversion has been - enhanced such that when the "conditional" converter is required - (now needed for the Oracle backend), the check for "already unicode" is now - performed in C and no longer introduces significant overhead. - - This change has two impacts on the cx_Oracle backend. One is that - string values in Py2K which aren't specifically requested with the - Unicode type or convert_unicode=True will now come back as ``str``, - not ``unicode`` - this behavior is similar to a backend such as - MySQL. Additionally, when unicode values are requested with the cx_Oracle - backend, if the C extensions are *not* used, there is now an additional - overhead of an isinstance() check per column. This tradeoff has been - made as it can be worked around and no longer places a performance burden - on the likely majority of Oracle result columns that are non-unicode - strings. - - .. change:: - :tags: bug, orm - :tickets: 2908 - - Fixed a bug involving the new flattened JOIN structures which - are used with :func:`_orm.joinedload()` (thereby causing a regression - in joined eager loading) as well as :func:`.aliased` - in conjunction with the ``flat=True`` flag and joined-table inheritance; - basically multiple joins across a "parent JOIN sub" entity using different - paths to get to a target class wouldn't form the correct ON conditions. - An adjustment / simplification made in the mechanics of figuring - out the "left side" of the join in the case of an aliased, joined-inh - class repairs the issue. - - .. change:: - :tags: bug, mysql - - The MySQL CAST compilation now takes into account aspects of a string - type such as "charset" and "collation". While MySQL wants all character- - based CAST calls to use the CHAR type, we now create a real CHAR - object at CAST time and copy over all the parameters it has, so that - an expression like ``cast(x, mysql.TEXT(charset='utf8'))`` will - render ``CAST(t.col AS CHAR CHARACTER SET utf8)``. - - .. change:: - :tags: bug, mysql - :tickets: 2906 - - Added new "unicode returns" detection to the MySQL dialect and - to the default dialect system overall, such that any dialect - can add extra "tests" to the on-first-connect "does this DBAPI - return unicode directly?" detection. In this case, we are - adding a check specifically against the "utf8" encoding with - an explicit "utf8_bin" collation type (after checking that - this collation is available) to test for some buggy unicode - behavior observed with MySQLdb version 1.2.3. While MySQLdb - has resolved this issue as of 1.2.4, the check here should - guard against regressions. The change also allows the "unicode" - checks to log in the engine logs, which was not previously - the case. - - .. change:: - :tags: bug, mysql, pool, engine - :tickets: 2907 - - :class:`_engine.Connection` now associates a new - :class:`.RootTransaction` or :class:`.TwoPhaseTransaction` - with its immediate :class:`._ConnectionFairy` as a "reset handler" - for the span of that transaction, which takes over the task - of calling commit() or rollback() for the "reset on return" behavior - of :class:`_pool.Pool` if the transaction was not otherwise completed. - This resolves the issue that a picky transaction - like that of MySQL two-phase will be - properly closed out when the connection is closed without an - explicit rollback or commit (e.g. no longer raises "XAER_RMFAIL" - in this case - note this only shows up in logging as the exception - is not propagated within pool reset). - This issue would arise e.g. when using an orm - :class:`.Session` with ``twophase`` set, and then - :meth:`.Session.close` is called without an explicit rollback or - commit. The change also has the effect that you will now see - an explicit "ROLLBACK" in the logs when using a :class:`.Session` - object in non-autocommit mode regardless of how that session was - discarded. Thanks to Jeff Dairiki and Laurence Rowe for isolating - the issue here. - - .. change:: - :tags: feature, pool, engine - - Added a new pool event :meth:`_events.PoolEvents.invalidate`. Called when - a DBAPI connection is to be marked as "invalidated" and discarded - from the pool. - - .. change:: - :tags: bug, pool - - The argument names for the :meth:`_events.PoolEvents.reset` event have been - renamed to ``dbapi_connection`` and ``connection_record`` in order - to maintain consistency with all the other pool events. It is expected - that any existing listeners for this relatively new and - seldom-used event are using positional style to receive arguments in - any case. - - .. change:: - :tags: bug, py3k, cextensions - - Fixed an issue where the C extensions in Py3K are using the wrong API - to specify the top-level module function, which breaks - in Python 3.4b2. Py3.4b2 changes PyMODINIT_FUNC to return - "void" instead of ``PyObject *``, so we now make sure to use - "PyMODINIT_FUNC" instead of ``PyObject *`` directly. Pull request - courtesy cgohlke. - - .. change:: - :tags: bug, schema - - Restored :class:`sqlalchemy.schema.SchemaVisitor` to the ``.schema`` - module. Pullreq courtesy Sean Dague. - -.. changelog:: - :version: 0.9.1 - :released: January 5, 2014 - - .. change:: - :tags: bug, orm, events - :tickets: 2905 - - Fixed regression where using a ``functools.partial()`` with the event - system would cause a recursion overflow due to usage of inspect.getargspec() - on it in order to detect a legacy calling signature for certain events, - and apparently there's no way to do this with a partial object. Instead - we skip the legacy check and assume the modern style; the check itself - now only occurs for the SessionEvents.after_bulk_update and - SessionEvents.after_bulk_delete events. Those two events will require - the new signature style if assigned to a "partial" event listener. - - .. change:: - :tags: feature, orm, extensions - - A new, **experimental** extension :mod:`sqlalchemy.ext.automap` is added. - This extension expands upon the functionality of Declarative as well as - the :class:`.DeferredReflection` class to produce a base class which - automatically generates mapped classes *and relationships* based on - table metadata. - - .. seealso:: - - :ref:`feature_automap` - - :ref:`automap_toplevel` - - .. change:: - :tags: feature, sql - - Conjunctions like :func:`.and_` and :func:`.or_` can now accept - Python generators as a single argument, e.g.:: - - and_(x == y for x, y in tuples) - - The logic here looks for a single argument ``*args`` where the first - element is an instance of ``types.GeneratorType``. - - .. change:: - :tags: feature, schema - - The :paramref:`_schema.Table.extend_existing` and :paramref:`_schema.Table.autoload_replace` - parameters are now available on the :meth:`_schema.MetaData.reflect` - method. - - .. change:: - :tags: bug, orm, declarative - - Fixed an extremely unlikely memory issue where when using - :class:`.DeferredReflection` - to define classes pending for reflection, if some subset of those - classes were discarded before the :meth:`.DeferredReflection.prepare` - method were called to reflect and map the class, a strong reference - to the class would remain held within the declarative internals. - This internal collection of "classes to map" now uses weak - references against the classes themselves. - - .. change:: - :tags: bug, orm - - Fixed bug where using new :attr:`.Session.info` attribute would fail - if the ``.info`` argument were only passed to the :class:`.sessionmaker` - creation call but not to the object itself. Courtesy Robin Schoonover. - - .. change:: - :tags: bug, orm - :tickets: 2901 - - Fixed regression where we don't check the given name against the - correct string class when setting up a backref based on a name, - therefore causing the error "too many values to unpack". This was - related to the Py3k conversion. - - .. change:: - :tags: bug, orm, declarative - :tickets: 2900 - - A quasi-regression where apparently in 0.8 you can set a class-level - attribute on declarative to simply refer directly to an :class:`.InstrumentedAttribute` - on a superclass or on the class itself, and it - acts more or less like a synonym; in 0.9, this fails to set up enough - bookkeeping to keep up with the more liberalized backref logic - from :ticket:`2789`. Even though this use case was never directly - considered, it is now detected by declarative at the "setattr()" level - as well as when setting up a subclass, and the mirrored/renamed attribute - is now set up as a :func:`.synonym` instead. - - .. change:: - :tags: bug, orm - :tickets: 2903 - - Fixed regression where we apparently still create an implicit - alias when saying query(B).join(B.cs), where "C" is a joined inh - class; however, this implicit alias was created only considering - the immediate left side, and not a longer chain of joins along different - joined-inh subclasses of the same base. As long as we're still - implicitly aliasing in this case, the behavior is dialed back a bit - so that it will alias the right side in a wider variety of cases. - -.. changelog:: - :version: 0.9.0 - :released: December 30, 2013 - - .. change:: - :tags: bug, orm, declarative - :tickets: 2828 - - Declarative does an extra check to detect if the same - :class:`_schema.Column` is mapped multiple times under different properties - (which typically should be a :func:`.synonym` instead) or if two - or more :class:`_schema.Column` objects are given the same name, raising - a warning if this condition is detected. - - .. change:: - :tags: bug, firebird - :tickets: 2898 - - Changed the queries used by Firebird to list table and view names - to query from the ``rdb$relations`` view instead of the - ``rdb$relation_fields`` and ``rdb$view_relations`` views. - Variants of both the old and new queries are mentioned on many - FAQ and blogs, however the new queries are taken straight from - the "Firebird FAQ" which appears to be the most official source - of info. - - .. change:: - :tags: bug, mysql - :tickets: 2893 - - Improvements to the system by which SQL types generate within - ``__repr__()``, particularly with regards to the MySQL integer/numeric/ - character types which feature a wide variety of keyword arguments. - The ``__repr__()`` is important for use with Alembic autogenerate - for when Python code is rendered in a migration script. - - .. change:: - :tags: feature, postgresql - :tickets: 2581 - - Support for PostgreSQL JSON has been added, using the new - :class:`_types.JSON` type. Huge thanks to Nathan Rice for - implementing and testing this. - - .. change:: - :tags: bug, sql - - The :func:`.cast` function, when given a plain literal value, - will now apply the given type to the given literal value on the - bind parameter side according to the type given to the cast, - in the same manner as that of the :func:`.type_coerce` function. - However unlike :func:`.type_coerce`, this only takes effect if a - non-clauseelement value is passed to :func:`.cast`; an existing typed - construct will retain its type. - - .. change:: - :tags: bug, postgresql - - Now using psycopg2 UNICODEARRAY extension for handling unicode arrays - with psycopg2 + normal "native unicode" mode, in the same way the - UNICODE extension is used. - - .. change:: - :tags: bug, sql - :tickets: 2883 - - The :class:`_schema.ForeignKey` class more aggressively checks the given - column argument. If not a string, it checks that the object is - at least a :class:`.ColumnClause`, or an object that resolves to one, - and that the ``.table`` attribute, if present, refers to a - :class:`_expression.TableClause` or subclass, and not something like an - :class:`_expression.Alias`. Otherwise, a :class:`.ArgumentError` is raised. - - - .. change:: - :tags: feature, orm - - The :class:`.exc.StatementError` or DBAPI-related subclass - now can accommodate additional information about the "reason" for - the exception; the :class:`.Session` now adds some detail to it - when the exception occurs within an autoflush. This approach - is taken as opposed to combining :class:`.FlushError` with - a Python 3 style "chained exception" approach so as to maintain - compatibility both with Py2K code as well as code that already - catches ``IntegrityError`` or similar. - - .. change:: - :tags: feature, postgresql - - Added support for PostgreSQL TSVECTOR via the - :class:`_postgresql.TSVECTOR` type. Pull request courtesy - Noufal Ibrahim. - - .. change:: - :tags: feature, engine - :tickets: 2875 - - The :func:`.engine_from_config` function has been improved so that - we will be able to parse dialect-specific arguments from string - configuration dictionaries. Dialect classes can now provide their - own list of parameter types and string-conversion routines. - The feature is not yet used by the built-in dialects, however. - - .. change:: - :tags: bug, sql - :tickets: 2879 - - The precedence rules for the :meth:`.ColumnOperators.collate` operator - have been modified, such that the COLLATE operator is now of lower - precedence than the comparison operators. This has the effect that - a COLLATE applied to a comparison will not render parenthesis - around the comparison, which is not parsed by backends such as - MSSQL. The change is backwards incompatible for those setups that - were working around the issue by applying :meth:`.Operators.collate` - to an individual element of the comparison expression, - rather than the comparison expression as a whole. - - .. seealso:: - - :ref:`migration_2879` - - .. change:: - :tags: bug, orm, declarative - :tickets: 2865 - - The :class:`.DeferredReflection` class has been enhanced to provide - automatic reflection support for the "secondary" table referred - to by a :func:`_orm.relationship`. "secondary", when specified - either as a string table name, or as a :class:`_schema.Table` object with - only a name and :class:`_schema.MetaData` object will also be included - in the reflection process when :meth:`.DeferredReflection.prepare` - is called. - - .. change:: - :tags: feature, orm, backrefs - :tickets: 1535 - - Added new argument ``include_backrefs=True`` to the - :func:`.validates` function; when set to False, a validation event - will not be triggered if the event was initiated as a backref to - an attribute operation from the other side. - - .. seealso:: - - :ref:`feature_1535` - - .. change:: - :tags: bug, orm, collections, py3k - - Added support for the Python 3 method ``list.clear()`` within - the ORM collection instrumentation system; pull request - courtesy Eduardo Schettino. - - .. change:: - :tags: bug, postgresql - :tickets: 2878 - - Fixed bug where values within an ENUM weren't escaped for single - quote signs. Note that this is backwards-incompatible for existing - workarounds that manually escape the single quotes. - - .. seealso:: - - :ref:`migration_2878` - - .. change:: - :tags: bug, orm, declarative - - Fixed bug where in Py2K a unicode literal would not be accepted - as the string name of a class or other argument within - declarative using :func:`_orm.relationship`. - - .. change:: - :tags: feature, sql - :tickets: 2877, 2882 - - New improvements to the :func:`_expression.text` construct, including - more flexible ways to set up bound parameters and return types; - in particular, a :func:`_expression.text` can now be turned into a full - FROM-object, embeddable in other statements as an alias or CTE - using the new method :meth:`_expression.TextClause.columns`. The :func:`_expression.text` - construct can also render "inline" bound parameters when the construct - is compiled in a "literal bound" context. - - .. seealso:: - - :ref:`feature_2877` - - .. change:: - :tags: feature, sql - - A new API for specifying the ``FOR UPDATE`` clause of a ``SELECT`` - is added with the new :meth:`_expression.GenerativeSelect.with_for_update` method. - This method supports a more straightforward system of setting - dialect-specific options compared to the ``for_update`` keyword - argument of :func:`_expression.select`, and also includes support for the - SQL standard ``FOR UPDATE OF`` clause. The ORM also includes - a new corresponding method :meth:`_query.Query.with_for_update`. - Pull request courtesy Mario Lassnig. - - .. seealso:: - - :ref:`feature_github_42` - - .. change:: - :tags: feature, orm - - A new API for specifying the ``FOR UPDATE`` clause of a ``SELECT`` - is added with the new :meth:`_query.Query.with_for_update` method, - to complement the new :meth:`_expression.GenerativeSelect.with_for_update` method. - Pull request courtesy Mario Lassnig. - - .. seealso:: - - :ref:`feature_github_42` - - .. change:: - :tags: bug, engine - :tickets: 2873 - - The :func:`_sa.create_engine` routine and the related :func:`.make_url` - function no longer considers the ``+`` sign to be a space within the - password field. The parsing in this area has been adjusted to match - more closely to how RFC 1738 handles these tokens, in that both - ``username`` and ``password`` expect only ``:``, ``@``, and ``/`` to be - encoded. - - .. seealso:: - - :ref:`migration_2873` - - - .. change:: - :tags: bug, orm - :tickets: 2872 - - Some refinements to the :class:`.AliasedClass` construct with regards - to descriptors, like hybrids, synonyms, composites, user-defined - descriptors, etc. The attribute - adaptation which goes on has been made more robust, such that if a descriptor - returns another instrumented attribute, rather than a compound SQL - expression element, the operation will still proceed. - Additionally, the "adapted" operator will retain its class; previously, - a change in class from ``InstrumentedAttribute`` to ``QueryableAttribute`` - (a superclass) would interact with Python's operator system such that - an expression like ``aliased(MyClass.x) > MyClass.x`` would reverse itself - to read ``myclass.x < myclass_1.x``. The adapted attribute will also - refer to the new :class:`.AliasedClass` as its parent which was not - always the case before. - - .. change:: - :tags: feature, sql - :tickets: 2867 - - The precision used when coercing a returned floating point value to - Python ``Decimal`` via string is now configurable. The - flag ``decimal_return_scale`` is now supported by all :class:`.Numeric` - and :class:`.Float` types, which will ensure this many digits are taken - from the native floating point value when it is converted to string. - If not present, the type will make use of the value of ``.scale``, if - the type supports this setting and it is non-None. Otherwise the original - default length of 10 is used. - - .. seealso:: - - :ref:`feature_2867` - - .. change:: - :tags: bug, schema - :tickets: 2868 - - Fixed a regression caused by :ticket:`2812` where the repr() for - table and column names would fail if the name contained non-ascii - characters. - - .. change:: - :tags: bug, engine - :tickets: 2848 - - The :class:`.RowProxy` object is now sortable in Python as a regular - tuple is; this is accomplished via ensuring tuple() conversion on - both sides within the ``__eq__()`` method as well as - the addition of a ``__lt__()`` method. - - .. seealso:: - - :ref:`migration_2848` - - .. change:: - :tags: bug, orm - :tickets: 2833 - - The ``viewonly`` flag on :func:`_orm.relationship` will now prevent - attribute history from being written on behalf of the target attribute. - This has the effect of the object not being written to the - Session.dirty list if it is mutated. Previously, the object would - be present in Session.dirty, but no change would take place on behalf - of the modified attribute during flush. The attribute still emits - events such as backref events and user-defined events and will still - receive mutations from backrefs. - - .. seealso:: - - :ref:`migration_2833` - - .. change:: - :tags: bug, orm - - Added support for new :attr:`.Session.info` attribute to - :class:`.scoped_session`. - - .. change:: - :tags: removed - - The "informix" and "informixdb" dialects have been removed; the code - is now available as a separate repository on Bitbucket. The IBM-DB - project has provided production-level Informix support since the - informixdb dialect was first added. - - .. change:: - :tags: bug, orm - - Fixed bug where usage of new :class:`.Bundle` object would cause - the :attr:`_query.Query.column_descriptions` attribute to fail. - - .. change:: - :tags: bug, examples - - Fixed bug which prevented history_meta recipe from working with - joined inheritance schemes more than one level deep. - - .. change:: - :tags: bug, orm, sql, sqlite - :tickets: 2858 - - Fixed a regression introduced by the join rewriting feature of - :ticket:`2369` and :ticket:`2587` where a nested join with one side - already an aliased select would fail to translate the ON clause on the - outside correctly; in the ORM this could be seen when using a - SELECT statement as a "secondary" table. - -.. changelog:: - :version: 0.9.0b1 - :released: October 26, 2013 - - .. change:: - :tags: feature, orm - :tickets: 2810 - - The association proxy now returns ``None`` when fetching a scalar - attribute off of a scalar relationship, where the scalar relationship - itself points to ``None``, instead of raising an ``AttributeError``. - - .. seealso:: - - :ref:`migration_2810` - - .. change:: - :tags: feature, sql, postgresql, mysql - :tickets: 2183 - - The PostgreSQL and MySQL dialects now support reflection/inspection - of foreign key options, including ON UPDATE, ON DELETE. PostgreSQL - also reflects MATCH, DEFERRABLE, and INITIALLY. Courtesy ijl. - - .. change:: - :tags: bug, mysql - :tickets: 2839 - - Fix and test parsing of MySQL foreign key options within reflection; - this complements the work in :ticket:`2183` where we begin to support - reflection of foreign key options such as ON UPDATE/ON DELETE - cascade. - - .. change:: - :tags: bug, orm - :tickets: 2787 - - :func:`.attributes.get_history()` when used with a scalar column-mapped - attribute will now honor the "passive" flag - passed to it; as this defaults to ``PASSIVE_OFF``, the function will - by default query the database if the value is not present. - This is a behavioral change vs. 0.8. - - .. seealso:: - - :ref:`change_2787` - - .. change:: - :tags: feature, orm - :tickets: 2787 - - Added new method :meth:`.AttributeState.load_history`, works like - :attr:`.AttributeState.history` but also fires loader callables. - - .. seealso:: - - :ref:`change_2787` - - - .. change:: - :tags: feature, sql - :tickets: 2850 - - A :func:`.bindparam` construct with a "null" type (e.g. no type - specified) is now copied when used in a typed expression, and the - new copy is assigned the actual type of the compared column. Previously, - this logic would occur on the given :func:`.bindparam` in place. - Additionally, a similar process now occurs for :func:`.bindparam` constructs - passed to :meth:`.ValuesBase.values` for an :class:`_expression.Insert` or - :class:`_expression.Update` construct, within the compilation phase of the - construct. - - These are both subtle behavioral changes which may impact some - usages. - - .. seealso:: - - :ref:`migration_2850` - - .. change:: - :tags: feature, sql - :tickets: 2804, 2823, 2734 - - An overhaul of expression handling for special symbols particularly - with conjunctions, e.g. - ``None`` :func:`_expression.null` :func:`_expression.true` - :func:`_expression.false`, including consistency in rendering NULL - in conjunctions, "short-circuiting" of :func:`.and_` and :func:`.or_` - expressions which contain boolean constants, and rendering of - boolean constants and expressions as compared to "1" or "0" for backends - that don't feature ``true``/``false`` constants. - - .. seealso:: - - :ref:`migration_2804` - - .. change:: - :tags: feature, sql - :tickets: 2838 - - The typing system now handles the task of rendering "literal bind" values, - e.g. values that are normally bound parameters but due to context must - be rendered as strings, typically within DDL constructs such as - CHECK constraints and indexes (note that "literal bind" values - become used by DDL as of :ticket:`2742`). A new method - :meth:`.TypeEngine.literal_processor` serves as the base, and - :meth:`.TypeDecorator.process_literal_param` is added to allow wrapping - of a native literal rendering method. - - .. seealso:: - - :ref:`change_2838` - - .. change:: - :tags: feature, sql - :tickets: 2716 - - The :meth:`_schema.Table.tometadata` method now produces copies of - all :attr:`.SchemaItem.info` dictionaries from all :class:`.SchemaItem` - objects within the structure including columns, constraints, - foreign keys, etc. As these dictionaries - are copies, they are independent of the original dictionary. - Previously, only the ``.info`` dictionary of :class:`_schema.Column` was transferred - within this operation, and it was only linked in place, not copied. - - .. change:: - :tags: feature, postgresql - :tickets: 2840 - - Added support for rendering ``SMALLSERIAL`` when a :class:`.SmallInteger` - type is used on a primary key autoincrement column, based on server - version detection of PostgreSQL version 9.2 or greater. - - .. change:: - :tags: feature, mysql - :tickets: 2817 - - The MySQL :class:`.mysql.SET` type now features the same auto-quoting - behavior as that of :class:`.mysql.ENUM`. Quotes are not required when - setting up the value, but quotes that are present will be auto-detected - along with a warning. This also helps with Alembic where - the SET type doesn't render with quotes. - - .. change:: - :tags: feature, sql - - The ``default`` argument of :class:`_schema.Column` now accepts a class - or object method as an argument, in addition to a standalone function; - will properly detect if the "context" argument is accepted or not. - - .. change:: - :tags: bug, sql - :tickets: 2835 - - The "name" attribute is set on :class:`.Index` before the "attach" - events are called, so that attachment events can be used to dynamically - generate a name for the index based on the parent table and/or - columns. - - .. change:: - :tags: bug, engine - :tickets: 2748 - - The method signature of :meth:`.Dialect.reflecttable`, which in - all known cases is provided by :class:`.DefaultDialect`, has been - tightened to expect ``include_columns`` and ``exclude_columns`` - arguments without any kw option, reducing ambiguity - previously - ``exclude_columns`` was missing. - - .. change:: - :tags: bug, sql - :tickets: 2831 - - The erroneous kw arg "schema" has been removed from the :class:`_schema.ForeignKey` - object. this was an accidental commit that did nothing; a warning is raised - in 0.8.3 when this kw arg is used. - - .. change:: - :tags: feature, orm - :tickets: 1418 - - Added a new load option :func:`_orm.load_only`. This allows a series - of column names to be specified as loading "only" those attributes, - deferring the rest. - - .. change:: - :tags: feature, orm - :tickets: 1418 - - The system of loader options has been entirely rearchitected to build - upon a much more comprehensive base, the :class:`_orm.Load` object. This - base allows any common loader option like :func:`_orm.joinedload`, - :func:`.defer`, etc. to be used in a "chained" style for the purpose - of specifying options down a path, such as ``joinedload("foo").subqueryload("bar")``. - The new system supersedes the usage of dot-separated path names, - multiple attributes within options, and the usage of ``_all()`` options. - - .. seealso:: - - :ref:`feature_1418` - - .. change:: - :tags: feature, orm - :tickets: 2824 - - The :func:`.composite` construct now maintains the return object - when used in a column-oriented :class:`_query.Query`, rather than expanding - out into individual columns. This makes use of the new :class:`.Bundle` - feature internally. This behavior is backwards incompatible; to - select from a composite column which will expand out, use - ``MyClass.some_composite.clauses``. - - .. seealso:: - - :ref:`migration_2824` - - .. change:: - :tags: feature, orm - :tickets: 2824 - - A new construct :class:`.Bundle` is added, which allows for specification - of groups of column expressions to a :class:`_query.Query` construct. - The group of columns are returned as a single tuple by default. The - behavior of :class:`.Bundle` can be overridden however to provide - any sort of result processing to the returned row. The behavior - of :class:`.Bundle` is also embedded into composite attributes now - when they are used in a column-oriented :class:`_query.Query`. - - .. seealso:: - - :ref:`change_2824` - - :ref:`migration_2824` - - .. change:: - :tags: bug, sql - :tickets: 2812 - - A rework to the way that "quoted" identifiers are handled, in that - instead of relying upon various ``quote=True`` flags being passed around, - these flags are converted into rich string objects with quoting information - included at the point at which they are passed to common schema constructs - like :class:`_schema.Table`, :class:`_schema.Column`, etc. This solves the issue - of various methods that don't correctly honor the "quote" flag such - as :meth:`_engine.Engine.has_table` and related methods. The :class:`.quoted_name` - object is a string subclass that can also be used explicitly if needed; - the object will hold onto the quoting preferences passed and will - also bypass the "name normalization" performed by dialects that - standardize on uppercase symbols, such as Oracle, Firebird and DB2. - The upshot is that the "uppercase" backends can now work with force-quoted - names, such as lowercase-quoted names and new reserved words. - - .. seealso:: - - :ref:`change_2812` - - .. change:: - :tags: feature, orm - :tickets: 2793 - - The ``version_id_generator`` parameter of ``Mapper`` can now be specified - to rely upon server generated version identifiers, using triggers - or other database-provided versioning features, or via an optional programmatic - value, by setting ``version_id_generator=False``. - When using a server-generated version identifier, the ORM will use RETURNING when - available to immediately - load the new version value, else it will emit a second SELECT. - - .. change:: - :tags: feature, orm - :tickets: 2793 - - The ``eager_defaults`` flag of :class:`_orm.Mapper` will now allow the - newly generated default values to be fetched using an inline - RETURNING clause, rather than a second SELECT statement, for backends - that support RETURNING. - - .. change:: - :tags: feature, core - :tickets: 2793 - - Added a new variant to :meth:`.UpdateBase.returning` called - :meth:`.ValuesBase.return_defaults`; this allows arbitrary columns - to be added to the RETURNING clause of the statement without interfering - with the compilers usual "implicit returning" feature, which is used to - efficiently fetch newly generated primary key values. For supporting - backends, a dictionary of all fetched values is present at - :attr:`_engine.ResultProxy.returned_defaults`. - - .. change:: - :tags: bug, mysql - - Improved support for the cymysql driver, supporting version 0.6.5, - courtesy Hajime Nakagami. - - .. change:: - :tags: general - - A large refactoring of packages has reorganized - the import structure of many Core modules as well as some aspects - of the ORM modules. In particular ``sqlalchemy.sql`` has been broken - out into several more modules than before so that the very large size - of ``sqlalchemy.sql.expression`` is now pared down. The effort - has focused on a large reduction in import cycles. Additionally, - the system of API functions in ``sqlalchemy.sql.expression`` and - ``sqlalchemy.orm`` has been reorganized to eliminate redundancy - in documentation between the functions vs. the objects they produce. - - .. change:: - :tags: orm, feature, orm - - Added a new attribute :attr:`.Session.info` to :class:`.Session`; - this is a dictionary where applications can store arbitrary - data local to a :class:`.Session`. - The contents of :attr:`.Session.info` can be also be initialized - using the ``info`` argument of :class:`.Session` or - :class:`.sessionmaker`. - - - .. change:: - :tags: feature, general, py3k - :tickets: 2161 - - The C extensions are ported to Python 3 and will build under - any supported CPython 2 or 3 environment. - - .. change:: - :tags: feature, orm - :tickets: 2268 - - Removal of event listeners is now implemented. The feature is - provided via the :func:`.event.remove` function. - - .. seealso:: - - :ref:`feature_2268` - - .. change:: - :tags: feature, orm - :tickets: 2789 - - The mechanism by which attribute events pass along an - :class:`.AttributeImpl` as an "initiator" token has been changed; - the object is now an event-specific object called :class:`.attributes.Event`. - Additionally, the attribute system no longer halts events based - on a matching "initiator" token; this logic has been moved to be - specific to ORM backref event handlers, which are the typical source - of the re-propagation of an attribute event onto subsequent append/set/remove - operations. End user code which emulates the behavior of backrefs - must now ensure that recursive event propagation schemes are halted, - if the scheme does not use the backref handlers. Using this new system, - backref handlers can now perform a - "two-hop" operation when an object is appended to a collection, - associated with a new many-to-one, de-associated with the previous - many-to-one, and then removed from a previous collection. Before this - change, the last step of removal from the previous collection would - not occur. - - .. seealso:: - - :ref:`migration_2789` - - .. change:: - :tags: feature, sql - :tickets: 722 - - Added new method to the :func:`_expression.insert` construct - :meth:`_expression.Insert.from_select`. Given a list of columns and - a selectable, renders ``INSERT INTO (table) (columns) SELECT ..``. - While this feature is highlighted as part of 0.9 it is also - backported to 0.8.3. - - .. seealso:: - - :ref:`feature_722` - - .. change:: - :tags: feature, engine - :tickets: 2770 - - New events added to :class:`_events.ConnectionEvents`: - - * :meth:`_events.ConnectionEvents.engine_connect` - * :meth:`_events.ConnectionEvents.set_connection_execution_options` - * :meth:`_events.ConnectionEvents.set_engine_execution_options` - - .. change:: - :tags: bug, sql - :tickets: 1765 - - The resolution of :class:`_schema.ForeignKey` objects to their - target :class:`_schema.Column` has been reworked to be as - immediate as possible, based on the moment that the - target :class:`_schema.Column` is associated with the same - :class:`_schema.MetaData` as this :class:`_schema.ForeignKey`, rather - than waiting for the first time a join is constructed, - or similar. This along with other improvements allows - earlier detection of some foreign key configuration - issues. Also included here is a rework of the - type-propagation system, so that - it should be reliable now to set the type as ``None`` - on any :class:`_schema.Column` that refers to another via - :class:`_schema.ForeignKey` - the type will be copied from the - target column as soon as that other column is associated, - and now works for composite foreign keys as well. - - .. seealso:: - - :ref:`migration_1765` - - .. change:: - :tags: feature, sql - :tickets: 2744, 2734 - - Provided a new attribute for :class:`.TypeDecorator` - called :attr:`.TypeDecorator.coerce_to_is_types`, - to make it easier to control how comparisons using - ``==`` or ``!=`` to ``None`` and boolean types goes - about producing an ``IS`` expression, or a plain - equality expression with a bound parameter. - - .. change:: - :tags: feature, pool - :tickets: 2752 - - Added pool logging for "rollback-on-return" and the less used - "commit-on-return". This is enabled with the rest of pool - "debug" logging. - - .. change:: - :tags: bug, orm, associationproxy - :tickets: 2751 - - Added additional criterion to the ==, != comparators, used with - scalar values, for comparisons to None to also take into account - the association record itself being non-present, in addition to the - existing test for the scalar endpoint on the association record - being NULL. Previously, comparing ``Cls.scalar == None`` would return - records for which ``Cls.associated`` were present and - ``Cls.associated.scalar`` is None, but not rows for which - ``Cls.associated`` is non-present. More significantly, the - inverse operation ``Cls.scalar != None`` *would* return ``Cls`` - rows for which ``Cls.associated`` was non-present. - - The case for ``Cls.scalar != 'somevalue'`` is also modified - to act more like a direct SQL comparison; only rows for - which ``Cls.associated`` is present and ``Associated.scalar`` - is non-NULL and not equal to ``'somevalue'`` are returned. - Previously, this would be a simple ``NOT EXISTS``. - - Also added a special use case where you - can call ``Cls.scalar.has()`` with no arguments, - when ``Cls.scalar`` is a column-based value - this returns whether or - not ``Cls.associated`` has any rows present, regardless of whether - or not ``Cls.associated.scalar`` is NULL or not. - - .. seealso:: - - :ref:`migration_2751` - - - .. change:: - :tags: feature, orm - :tickets: 2587 - - A major change regarding how the ORM constructs joins where - the right side is itself a join or left outer join. The ORM - is now configured to allow simple nesting of joins of - the form ``a JOIN (b JOIN c ON b.id=c.id) ON a.id=b.id``, - rather than forcing the right side into a ``SELECT`` subquery. - This should allow significant performance improvements on most - backends, most particularly MySQL. The one database backend - that has for many years held back this change, SQLite, is now addressed by - moving the production of the ``SELECT`` subquery from the - ORM to the SQL compiler; so that a right-nested join on SQLite will still - ultimately render with a ``SELECT``, while all other backends - are no longer impacted by this workaround. - - As part of this change, a new argument ``flat=True`` has been added - to the :func:`_orm.aliased`, :meth:`_expression.Join.alias`, and - :func:`_orm.with_polymorphic` functions, which allows an "alias" of a - JOIN to be produced which applies an anonymous alias to each component - table within the join, rather than producing a subquery. - - .. seealso:: - - :ref:`feature_joins_09` - - - .. change:: - :tags: bug, orm - :tickets: 2369 - - Fixed an obscure bug where the wrong results would be - fetched when joining/joinedloading across a many-to-many - relationship to a single-table-inheriting - subclass with a specific discriminator value, due to "secondary" - rows that would come back. The "secondary" and right-side - tables are now inner joined inside of parenthesis for all - ORM joins on many-to-many relationships so that the left->right - join can accurately filtered. This change was made possible - by finally addressing the issue with right-nested joins - outlined in :ticket:`2587`. - - .. seealso:: - - :ref:`feature_joins_09` - - .. change:: - :tags: bug, mssql, pyodbc - :tickets: 2355 - - Fixes to MSSQL with Python 3 + pyodbc, including that statements - are passed correctly. - - .. change:: - :tags: feature, sql - :tickets: 1068 - - A :func:`~sqlalchemy.sql.expression.label` construct will now render as its name alone - in an ``ORDER BY`` clause, if that label is also referred to - in the columns clause of the select, instead of rewriting the - full expression. This gives the database a better chance to - optimize the evaluation of the same expression in two different - contexts. - - .. seealso:: - - :ref:`migration_1068` - - .. change:: - :tags: feature, firebird - :tickets: 2504 - - The ``fdb`` dialect is now the default dialect when - specified without a dialect qualifier, i.e. ``firebird://``, - per the Firebird project publishing ``fdb`` as their - official Python driver. - - .. change:: - :tags: feature, general, py3k - :tickets: 2671 - - The codebase is now "in-place" for Python - 2 and 3, the need to run 2to3 has been removed. - Compatibility is now against Python 2.6 on forward. - - .. change:: - :tags: feature, oracle, py3k - - The Oracle unit tests with cx_oracle now pass - fully under Python 3. - - .. change:: - :tags: bug, orm - :tickets: 2736 - - The "auto-aliasing" behavior of the :meth:`_query.Query.select_from` - method has been turned off. The specific behavior is now - available via a new method :meth:`_query.Query.select_entity_from`. - The auto-aliasing behavior here was never well documented and - is generally not what's desired, as :meth:`_query.Query.select_from` - has become more oriented towards controlling how a JOIN is - rendered. :meth:`_query.Query.select_entity_from` will also be made - available in 0.8 so that applications which rely on the auto-aliasing - can shift their applications to use this method. - - .. seealso:: - - :ref:`migration_2736` diff --git a/doc/build/changelog/changelog_10.rst b/doc/build/changelog/changelog_10.rst deleted file mode 100644 index 1db674078fe..00000000000 --- a/doc/build/changelog/changelog_10.rst +++ /dev/null @@ -1,3404 +0,0 @@ -============= -1.0 Changelog -============= - -.. changelog_imports:: - - .. include:: changelog_09.rst - :start-line: 5 - - - .. include:: changelog_08.rst - :start-line: 5 - - - .. include:: changelog_07.rst - :start-line: 5 - - - -.. changelog:: - :version: 1.0.19 - :released: August 3, 2017 - - .. change:: - :tags: bug, oracle, performance, py2k - :tickets: 4035 - :versions: 1.0.19, 1.1.13, 1.2.0b3 - - Fixed performance regression caused by the fix for :ticket:`3937` where - cx_Oracle as of version 5.3 dropped the ``.UNICODE`` symbol from its - namespace, which was interpreted as cx_Oracle's "WITH_UNICODE" mode being - turned on unconditionally, which invokes functions on the SQLAlchemy - side which convert all strings to unicode unconditionally and causing - a performance impact. In fact, per cx_Oracle's author the - "WITH_UNICODE" mode has been removed entirely as of 5.1, so the expensive unicode - conversion functions are no longer necessary and are disabled if - cx_Oracle 5.1 or greater is detected under Python 2. The warning against - "WITH_UNICODE" mode that was removed under :ticket:`3937` is also restored. - -.. changelog:: - :version: 1.0.18 - :released: July 24, 2017 - - .. change:: - :tags: bug, tests, py3k - :tickets: 4034 - :versions: 1.0.18, 1.1.12, 1.2.0b2 - - Fixed issue in testing fixtures which was incompatible with a change - made as of Python 3.6.2 involving context managers. - - .. change:: 3937 - :tags: bug, oracle - :tickets: 3937 - :versions: 1.1.7 - - A fix to cx_Oracle's WITH_UNICODE mode which was uncovered by the - fact that cx_Oracle 5.3 now seems to hardcode this flag on in - the build; an internal method that uses this mode wasn't using - the correct signature. - - -.. changelog:: - :version: 1.0.17 - :released: January 17, 2017 - - .. change:: - :tags: bug, py3k - :tickets: 3886 - :versions: 1.1.5 - - Fixed Python 3.6 DeprecationWarnings related to escaped strings without - the 'r' modifier, and added test coverage for Python 3.6. - - .. change:: - :tags: bug, orm - :tickets: 3884 - :versions: 1.1.5 - - Fixed bug involving joined eager loading against multiple entities - when polymorphic inheritance is also in use which would throw - "'NoneType' object has no attribute 'isa'". The issue was introduced - by the fix for :ticket:`3611`. - -.. changelog:: - :version: 1.0.16 - :released: November 15, 2016 - - .. change:: - :tags: bug, orm - :tickets: 3849 - :versions: 1.1.4 - - Fixed bug in :meth:`.Session.bulk_update_mappings` where an alternate-named - primary key attribute would not track properly into the UPDATE statement. - - .. change:: - :tags: bug, mssql - :tickets: 3810 - :versions: 1.1.0 - - Changed the query used to get "default schema name", from one that - queries the database principals table to using the - "schema_name()" function, as issues have been reported that the - former system was unavailable on the Azure Data Warehouse edition. - It is hoped that this will finally work across all SQL Server - versions and authentication styles. - - .. change:: - :tags: bug, mssql - :tickets: 3814 - :versions: 1.1.0 - - Updated the server version info scheme for pyodbc to use SQL Server - SERVERPROPERTY(), rather than relying upon pyodbc.SQL_DBMS_VER, which - continues to be unreliable particularly with FreeTDS. - - .. change:: - :tags: bug, orm - :tickets: 3800 - :versions: 1.1.0 - - Fixed bug where joined eager loading would fail for a polymorphically- - loaded mapper, where the polymorphic_on was set to an un-mapped - expression such as a CASE expression. - - .. change:: - :tags: bug, orm - :tickets: 3798 - :versions: 1.1.0 - - Fixed bug where the ArgumentError raised for an invalid bind - sent to a Session via :meth:`.Session.bind_mapper`, - :meth:`.Session.bind_table`, - or the constructor would fail to be correctly raised. - - .. change:: - :tags: bug, mssql - :tickets: 3791 - :versions: 1.1.0 - - Added error code 20017 "unexpected EOF from the server" to the list of - disconnect exceptions that result in a connection pool reset. Pull - request courtesy Ken Robbins. - - .. change:: - :tags: bug, orm.declarative - :tickets: 3797 - :versions: 1.1.0 - - Fixed bug where setting up a single-table inh subclass of a joined-table - subclass which included an extra column would corrupt the foreign keys - collection of the mapped table, thereby interfering with the - initialization of relationships. - - .. change:: - :tags: bug, orm - :tickets: 3781 - :versions: 1.1.4 - - Fixed bug in :meth:`.Session.bulk_save` where an UPDATE would - not function correctly in conjunction with a mapping that - implements a version id counter. - - .. 3778 - - .. change:: - :tags: bug, orm - :tickets: 3778 - :versions: 1.1.4 - - Fixed bug where the :attr:`_orm.Mapper.attrs`, - :attr:`_orm.Mapper.all_orm_descriptors` and other derived attributes would - fail to refresh when mapper properties or other ORM constructs were - added to the mapper/class after these accessors were first called. - - .. change:: 3762 - :tags: bug, mssql - :tickets: 3762 - :versions: 1.1.4 - - Fixed bug in pyodbc dialect (as well as in the mostly non-working - adodbapi dialect) whereby a semicolon present in the password - or username fields could be interpreted as a separator for another - token; the values are now quoted when semicolons are present. - -.. changelog:: - :version: 1.0.15 - :released: September 1, 2016 - - .. change:: - :tags: bug, mysql - :tickets: 3787 - :versions: 1.1.0 - - Added support for parsing MySQL/Connector boolean and integer - arguments within the URL query string: connection_timeout, - connect_timeout, pool_size, get_warnings, - raise_on_warnings, raw, consume_results, ssl_verify_cert, force_ipv6, - pool_reset_session, compress, allow_local_infile, use_pure. - - .. change:: - :tags: bug, orm - :tickets: 3773, 3774 - :versions: 1.1.0 - - Fixed bug in subquery eager loading where a subqueryload - of an "of_type()" object linked to a second subqueryload of a plain - mapped class, or a longer chain of several "of_type()" attributes, - would fail to link the joins correctly. - - .. change:: - :tags: bug, sql - :tickets: 3755 - :versions: 1.1.0 - - Fixed bug in :class:`_schema.Table` where the internal method - ``_reset_exported()`` would corrupt the state of the object. This - method is intended for selectable objects and is called by the ORM - in some cases; an erroneous mapper configuration would could lead the - ORM to call this on a :class:`_schema.Table` object. - - .. change:: - :tags: bug, ext - :tickets: 3743 - :versions: 1.1.0b3 - - Fixed bug in ``sqlalchemy.ext.baked`` where the unbaking of a - subquery eager loader query would fail due to a variable scoping - issue, when multiple subquery loaders were involved. Pull request - courtesy Mark Hahnenberg. - -.. changelog:: - :version: 1.0.14 - :released: July 6, 2016 - - .. change:: - :tags: bug, postgresql - :tickets: 3739 - :versions: 1.1.0b3 - - Fixed bug whereby :class:`.TypeDecorator` and :class:`.Variant` - types were not deeply inspected enough by the PostgreSQL dialect - to determine if SMALLSERIAL or BIGSERIAL needed to be rendered - rather than SERIAL. - - .. change:: - :tags: bug, oracle - :tickets: 3741 - :versions: 1.1.0b3 - - Fixed bug in :paramref:`.Select.with_for_update.of`, where the Oracle - "rownum" approach to LIMIT/OFFSET would fail to accommodate for the - expressions inside the "OF" clause, which must be stated at the topmost - level referring to expression within the subquery. The expressions are - now added to the subquery if needed. - - .. change:: - :tags: bug, sql - :tickets: 3735 - :versions: 1.1.0b2 - - Fixed issue in SQL math negation operator where the type of the - expression would no longer be the numeric type of the original. - This would cause issues where the type determined result set - behaviors. - - .. change:: - :tags: bug, sql - :tickets: 3728 - :versions: 1.1.0b2 - - Fixed bug whereby the ``__getstate__`` / ``__setstate__`` - methods for sqlalchemy.util.Properties were - non-working due to the transition in the 1.0 series to ``__slots__``. - The issue potentially impacted some third-party applications. - Pull request courtesy Pieter Mulder. - - .. change:: - :tags: bug, sql - :tickets: 3724 - - :meth:`_expression.FromClause.count` is pending deprecation for 1.1. This function - makes use of an arbitrary column in the table and is not reliable; - for Core use, ``func.count()`` should be preferred. - - .. change:: - :tags: bug, sql - :tickets: 3722 - - Fixed bug in :class:`_expression.CTE` structure which would cause it to not - clone properly when a union was used, as is common in a recursive - CTE. The improper cloning would cause errors when the CTE is used - in various ORM contexts such as that of a :func:`.column_property`. - - .. change:: - :tags: bug, sql - :tickets: 3721 - - Fixed bug whereby :meth:`_schema.Table.tometadata` would make a duplicate - :class:`.UniqueConstraint` for each :class:`_schema.Column` object that - featured the ``unique=True`` parameter. - - .. change:: - :tags: bug, engine, postgresql - :tickets: 3716 - - Fixed bug in cross-schema foreign key reflection in conjunction - with the :paramref:`_schema.MetaData.schema` argument, where a referenced - table that is present in the "default" schema would fail since there - would be no way to indicate a :class:`_schema.Table` that has "blank" for - a schema. The special symbol :attr:`_schema.BLANK_SCHEMA` has been - added as an available value for :paramref:`_schema.Table.schema` and - :paramref:`.Sequence.schema`, indicating that the schema name - should be forced to be ``None`` even if :paramref:`_schema.MetaData.schema` - is specified. - - .. change:: - :tags: bug, examples - :tickets: 3704 - - Fixed a regression that occurred in the - examples/vertical/dictlike-polymorphic.py example which prevented it - from running. - -.. changelog:: - :version: 1.0.13 - :released: May 16, 2016 - - .. change:: - :tags: bug, orm - :tickets: 3700 - - Fixed bug in "evaluate" strategy of :meth:`_query.Query.update` and - :meth:`_query.Query.delete` which would fail to accommodate a bound - parameter with a "callable" value, as which occurs when filtering - by a many-to-one equality expression along a relationship. - - .. change:: - :tags: bug, postgresql - :tickets: 3715 - - Added disconnect detection support for the error string - "SSL error: decryption failed or bad record mac". Pull - request courtesy Iuri de Silvio. - - .. change:: - :tags: bug, mssql - :tickets: 3711 - - Fixed bug where by ROW_NUMBER OVER clause applied for OFFSET - selects in SQL Server would inappropriately substitute a plain column - from the local statement that overlaps with a label name used by - the ORDER BY criteria of the statement. - - .. change:: - :tags: bug, orm - :tickets: 3710 - - Fixed bug whereby the event listeners used for backrefs could - be inadvertently applied multiple times, when using a deep class - inheritance hierarchy in conjunction with multiple mapper configuration - steps. - - .. change:: - :tags: bug, orm - :tickets: 3706 - - Fixed bug whereby passing a :func:`_expression.text` construct to the - :meth:`_query.Query.group_by` method would raise an error, instead - of interpreting the object as a SQL fragment. - - .. change:: - :tags: bug, oracle - :tickets: 3705 - - Fixed a bug in the cx_Oracle connect process that caused a TypeError - when the either the user, password or dsn was empty. This prevented - external authentication to Oracle databases, and prevented connecting - to the default dsn. The connect string oracle:// now logs into the - default dsn using the Operating System username, equivalent to - connecting using '/' with sqlplus. - - .. change:: - :tags: bug, oracle - :tickets: 3699 - - Fixed a bug in the result proxy used mainly by Oracle when binary and - other LOB types are in play, such that when query / statement caching - were used, the type-level result processors, notably that required by - the binary type itself but also any other processor, would become lost - after the first run of the statement due to it being removed from the - cached result metadata. - - .. change:: - :tags: bug, examples - :tickets: 3698 - - Changed the "directed graph" example to no longer consider - integer identifiers of nodes as significant; the "higher" / "lower" - references now allow mutual edges in both directions. - - .. change:: - :tags: bug, sql - :tickets: 3690 - - Fixed bug where when using ``case_sensitive=False`` with an - :class:`_engine.Engine`, the result set would fail to correctly accommodate - for duplicate column names in the result set, causing an error - when the statement is executed in 1.0, and preventing the - "ambiguous column" exception from functioning in 1.1. - - .. change:: - :tags: bug, sql - :tickets: 3682 - - Fixed bug where the negation of an EXISTS expression would not - be properly typed as boolean in the result, and also would fail to be - anonymously aliased in a SELECT list as is the case with a - non-negated EXISTS construct. - - .. change:: - :tags: bug, sql - :tickets: 3666 - - Fixed bug where "unconsumed column names" exception would fail to - be raised in the case where :meth:`_expression.Insert.values` were called - with a list of parameter mappings, instead of a single mapping - of parameters. Pull request courtesy Athena Yao. - - .. change:: - :tags: bug, orm - :tickets: 3663 - - Anonymous labeling is applied to a :attr:`.func` construct that is - passed to :func:`.column_property`, so that if the same attribute - is referred to as a column expression twice the names are de-duped, - thus avoiding "ambiguous column" errors. Previously, the - ``.label(None)`` would need to be applied in order for the name - to be de-anonymized. - - .. change:: - :tags: bug, py3k - :tickets: 3660 - - Fixed bug in "to_list" conversion where a single bytes object - would be turned into a list of individual characters. This would - impact among other things using the :meth:`_query.Query.get` method - on a primary key that's a bytes object. - - .. change:: - :tags: bug, orm - :tickets: 3658 - - Fixed regression appearing in the 1.0 series in ORM loading where the - exception raised for an expected column missing would incorrectly - be a ``NoneType`` error, rather than the expected - :class:`.NoSuchColumnError`. - - .. change:: - :tags: bug, mssql, oracle - :tickets: 3657 - - Fixed regression appearing in the 1.0 series which would cause the Oracle - and SQL Server dialects to incorrectly account for result set columns - when these dialects would wrap a SELECT in a subquery in order to - provide LIMIT/OFFSET behavior, and the original SELECT statement - referred to the same column multiple times, such as a column and - a label of that same column. This issue is related - to :ticket:`3658` in that when the error occurred, it would also - cause a ``NoneType`` error, rather than reporting that it couldn't - locate a column. - -.. changelog:: - :version: 1.0.12 - :released: February 15, 2016 - - .. change:: - :tags: bug, orm - :tickets: 3647 - - Fixed bug in :meth:`.Session.merge` where an object with a composite - primary key that has values for some but not all of the PK fields - would emit a SELECT statement leaking the internal NEVER_SET symbol - into the query, rather than detecting that this object does not have - a searchable primary key and no SELECT should be emitted. - - .. change:: - :tags: bug, postgresql - :tickets: 3644 - - Fixed bug in :func:`_expression.text` construct where a double-colon - expression would not escape properly, e.g. ``some\:\:expr``, as is most - commonly required when rendering PostgreSQL-style CAST expressions. - - .. change:: - :tags: bug, sql - :tickets: 3643 - - Fixed issue where the "literal_binds" flag was not propagated - for :func:`_expression.insert`, :func:`_expression.update` or - :func:`_expression.delete` constructs when compiled to string - SQL. Pull request courtesy Tim Tate. - - .. change:: - :tags: bug, oracle, jython - :tickets: 3621 - - Fixed a small issue in the Jython Oracle compiler involving the - rendering of "RETURNING" which allows this currently - unsupported/untested dialect to work rudimentarily with the 1.0 series. - Pull request courtesy Carlos Rivas. - - .. change:: - :tags: bug, sql - :tickets: 3642 - - Fixed issue where inadvertent use of the Python ``__contains__`` - override with a column expression (e.g. by using ``'x' in col``) - would cause an endless loop in the case of an ARRAY type, as Python - defers this to ``__getitem__`` access which never raises for this - type. Overall, all use of ``__contains__`` now raises - NotImplementedError. - - .. change:: - :tags: bug, engine, mysql - :tickets: 2696 - - Revisiting :ticket:`2696`, first released in 1.0.10, which attempts to - work around Python 2's lack of exception context reporting by emitting - a warning for an exception that was interrupted by a second exception - when attempting to roll back the already-failed transaction; this - issue continues to occur for MySQL backends in conjunction with a - savepoint that gets unexpectedly lost, which then causes a - "no such savepoint" error when the rollback is attempted, obscuring - what the original condition was. - - The approach has been generalized to the Core "safe - reraise" function which takes place across the ORM and Core in any - place that a transaction is being rolled back in response to an error - which occurred trying to commit, including the context managers - provided by :class:`.Session` and :class:`_engine.Connection`, and taking - place for operations such as a failure on "RELEASE SAVEPOINT". - Previously, the fix was only in place for a specific path within - the ORM flush/commit process; it now takes place for all transactional - context managers as well. - - .. change:: - :tags: bug, sql - :tickets: 3632 - - Fixed bug in :class:`_schema.Table` metadata construct which appeared - around the 0.9 series where adding columns to a :class:`_schema.Table` - that was unpickled would fail to correctly establish the - :class:`_schema.Column` within the 'c' collection, leading to issues in - areas such as ORM configuration. This could impact use cases such - as ``extend_existing`` and others. - - .. change:: - :tags: bug, py3k - :tickets: 3625 - - Fixed bug where some exception re-raise scenarios would attach - the exception to itself as the "cause"; while the Python 3 interpreter - is OK with this, it could cause endless loops in iPython. - - .. change:: - :tags: bug, mssql - :tickets: 3624 - - Fixed the syntax of the :func:`.extract` function when used on - MSSQL against a datetime value; the quotes around the keyword - are removed. Pull request courtesy Guillaume Doumenc. - - .. change:: - :tags: bug, orm - :tickets: 3623 - - Fixed regression since 0.9 where the 0.9 style loader options - system failed to accommodate for multiple :func:`.undefer_group` - loader options in a single query. Multiple :func:`.undefer_group` - options will now be taken into account even against the same - entity. - - .. change:: - :tags: bug, mssql, firebird - :tickets: 3622 - - Fixed 1.0 regression where the eager fetch of cursor.rowcount was - no longer called for an UPDATE or DELETE statement emitted via plain - text or via the :func:`_expression.text` construct, affecting those drivers - that erase cursor.rowcount once the cursor is closed such as SQL - Server ODBC and Firebird drivers. - - -.. changelog:: - :version: 1.0.11 - :released: December 22, 2015 - - .. change:: - :tags: bug, mysql - :tickets: 3613 - - An adjustment to the regular expression used to parse MySQL views, - such that we no longer assume the "ALGORITHM" keyword is present in - the reflected view source, as some users have reported this not being - present in some Amazon RDS environments. - - .. change:: - :tags: bug, mysql - - Added new reserved words for MySQL 5.7 to the MySQL dialect, - including 'generated', 'optimizer_costs', 'stored', 'virtual'. - Pull request courtesy Hanno Schlichting. - - .. change:: - :tags: bug, ext - :tickets: 3605 - - Further fixes to :ticket:`3605`, pop method on :class:`.MutableDict`, - where the "default" argument was not included. - - .. change:: - :tags: bug, ext - :tickets: 3612 - - Fixed bug in baked loader system where the systemwide monkeypatch - for setting up baked lazy loaders would interfere with other - loader strategies that rely on lazy loading as a fallback, e.g. - joined and subquery eager loaders, leading to ``IndexError`` - exceptions at mapper configuration time. - - .. change:: - :tags: bug, orm - :tickets: 3611 - - Fixed regression caused in 1.0.10 by the fix for :ticket:`3593` where - the check added for a polymorphic joinedload from a - poly_subclass->class->poly_baseclass connection would fail for the - scenario of class->poly_subclass->class. - - .. change:: - :tags: bug, orm - :tickets: 3610 - - Fixed bug where :meth:`.Session.bulk_update_mappings` and related - would not bump a version id counter when in use. The experience - here is still a little rough as the original version id is required - in the given dictionaries and there's not clean error reporting - on that yet. - - .. change:: - :tags: bug, sql - :tickets: 3609 - - Fixed bug in :meth:`_expression.Update.return_defaults` which would cause all - insert-default holding columns not otherwise included in the SET - clause (such as primary key cols) to get rendered into the RETURNING - even though this is an UPDATE. - - .. change:: - :tags: bug, orm - :tickets: 3609 - - Major fixes to the :paramref:`_orm.Mapper.eager_defaults` flag, this - flag would not be honored correctly in the case that multiple - UPDATE statements were to be emitted, either as part of a flush - or a bulk update operation. Additionally, RETURNING - would be emitted unnecessarily within update statements. - - .. change:: - :tags: bug, orm - :tickets: 3606 - - Fixed bug where use of the :meth:`_query.Query.select_from` method would - cause a subsequent call to the :meth:`_query.Query.with_parent` method to - fail. - -.. changelog:: - :version: 1.0.10 - :released: December 11, 2015 - - .. change:: - :tags: bug, ext - :tickets: 3605 - - Added support for the ``dict.pop()`` and ``dict.popitem()`` methods - to the :class:`.mutable.MutableDict` class. - - .. change:: - :tags: change, tests - - The ORM and Core tutorials, which have always been in doctest format, - are now exercised within the normal unit test suite in both Python - 2 and Python 3. - - .. change:: - :tags: bug, sql - :tickets: 3603 - - Fixed issue within the :meth:`_expression.Insert.from_select` construct whereby - the :class:`_expression.Select` construct would have its ``._raw_columns`` - collection mutated in-place when compiling the :class:`_expression.Insert` - construct, when the target :class:`_schema.Table` has Python-side defaults. - The :class:`_expression.Select` construct would compile standalone with the - erroneous column present subsequent to compilation of the - :class:`_expression.Insert`, and the :class:`_expression.Insert` statement itself would - fail on a second compile attempt due to duplicate bound parameters. - - .. change:: - :tags: bug, mysql - :tickets: 3602 - - Fixed bug in MySQL reflection where the "fractional sections portion" - of the :class:`.mysql.DATETIME`, :class:`.mysql.TIMESTAMP` and - :class:`.mysql.TIME` types would be incorrectly placed into the - ``timezone`` attribute, which is unused by MySQL, instead of the - ``fsp`` attribute. - - .. change:: - :tags: bug, orm - :tickets: 3599 - - Fixed issue where post_update on a many-to-one relationship would - fail to emit an UPDATE in the case where the attribute were set to - None and not previously loaded. - - .. change:: - :tags: bug, sql, postgresql - :tickets: 3598 - - Fixed bug where CREATE TABLE with a no-column table, but a constraint - such as a CHECK constraint would render an erroneous comma in the - definition; this scenario can occur such as with a PostgreSQL - INHERITS table that has no columns of its own. - - .. change:: - :tags: bug, mssql - :tickets: 3585 - - - Added the error "20006: Write to the server failed" to the list - of disconnect errors for the pymssql driver, as this has been observed - to render a connection unusable. - - .. change:: - :tags: bug, postgresql - :tickets: 3573 - - - Fixed issue where the "FOR UPDATE OF" PostgreSQL-specific SELECT - modifier would fail if the referred table had a schema qualifier; - PG needs the schema name to be omitted. Pull request courtesy - Diana Clarke. - - .. change:: - :tags: bug, postgresql - - - Fixed bug where some varieties of SQL expression passed to the - "where" clause of :class:`_postgresql.ExcludeConstraint` would fail - to be accepted correctly. Pull request courtesy aisch. - - .. change:: - :tags: bug, orm, declarative - - - Fixed bug where in Py2K a unicode literal would not be accepted as the - string name of a class or other argument within declarative using - :func:`.backref` on :func:`_orm.relationship`. Pull request courtesy - Nils Philippsen. - - .. change:: - :tags: bug, mssql - - A descriptive ValueError is now raised in the event that SQL server - returns an invalid date or time format from a DATE or TIME - column, rather than failing with a NoneType error. Pull request - courtesy Ed Avis. - - .. change:: - :tags: bug, py3k - - Updates to internal getargspec() calls, some py36-related - fixture updates, and alterations to two iterators to "return" instead - of raising StopIteration, to allow tests to pass without - errors or warnings on Py3.5, Py3.6, pull requests courtesy - Jacob MacDonald, Luri de Silvio, and Phil Jones. - - .. change:: - :tags: bug, ext - :tickets: 3597 - - Fixed an issue in baked queries where the .get() method, used either - directly or within lazy loads, didn't consider the mapper's "get clause" - as part of the cache key, causing bound parameter mismatches if the - clause got re-generated. This clause is cached by mappers - on the fly but in highly concurrent scenarios may be generated more - than once when first accessed. - - .. change:: - :tags: feature, sql - - Added support for parameter-ordered SET clauses in an UPDATE - statement. This feature is available by passing the - :paramref:`~.sqlalchemy.sql.expression.update.preserve_parameter_order` - flag either to the core :class:`_expression.Update` construct or alternatively - adding it to the :paramref:`.Query.update.update_args` dictionary at - the ORM-level, also passing the parameters themselves as a list of 2-tuples. - Thanks to Gorka Eguileor for implementation and tests. - - .. seealso:: - - :ref:`tutorial_parameter_ordered_updates` - - .. change:: - :tags: bug, orm - :tickets: 3593 - - Fixed bug which is actually a regression that occurred between - versions 0.8.0 and 0.8.1, due :ticket:`2714`. The case where - joined eager loading needs to join out over a subclass-bound - relationship when "with_polymorphic" were also used would fail - to join from the correct entity. - - .. change:: - :tags: bug, orm - :tickets: 3592 - - Fixed joinedload bug which would occur when a. the query includes - limit/offset criteria that forces a subquery b. the relationship - uses "secondary" c. the primaryjoin of the relationship refers to - a column that is either not part of the primary key, or is a PK - col in a joined-inheritance subclass table that is under a different - attribute name than the parent table's primary key column d. the - query defers the columns that are present in the primaryjoin, typically - via not being included in load_only(); the necessary column(s) would - not be present in the subquery and produce invalid SQL. - - .. change:: - :tags: bug, orm - :tickets: 2696 - - A rare case which occurs when a :meth:`.Session.rollback` fails in the - scope of a :meth:`.Session.flush` operation that's raising an - exception, as has been observed in some MySQL SAVEPOINT cases, prevents - the original database exception from being observed when it was - emitted during flush, but only on Py2K because Py2K does not support - exception chaining; on Py3K the originating exception is chained. As - a workaround, a warning is emitted in this specific case showing at - least the string message of the original database error before we - proceed to raise the rollback-originating exception. - - .. change:: - :tags: bug, postgresql - :tickets: 3571 - - Fixed the ``.python_type`` attribute of :class:`_postgresql.INTERVAL` - to return ``datetime.timedelta`` in the same way as that of - :obj:`.types.Interval.python_type`, rather than raising - ``NotImplementedError``. - - .. change:: - :tags: bug, mssql - - - Fixed issue where DDL generated for the MSSQL types DATETIME2, - TIME and DATETIMEOFFSET with a precision of "zero" would not generate - the precision field. Pull request courtesy Jacobo de Vera. - - -.. changelog:: - :version: 1.0.9 - :released: October 20, 2015 - - .. change:: - :tags: bug, orm, postgresql - :tickets: 3556 - - Fixed regression in 1.0 where new feature of using "executemany" - for UPDATE statements in the ORM (e.g. :ref:`feature_updatemany`) - would break on PostgreSQL and other RETURNING backends - when using server-side version generation - schemes, as the server side value is retrieved via RETURNING which - is not supported with executemany. - - .. change:: - :tags: feature, ext - :tickets: 3551 - - Added the :paramref:`.AssociationProxy.info` parameter to the - :class:`.AssociationProxy` constructor, to suit the - :attr:`.AssociationProxy.info` accessor that was added in - :ticket:`2971`. This is possible because :class:`.AssociationProxy` - is constructed explicitly, unlike a hybrid which is constructed - implicitly via the decorator syntax. - - .. change:: - :tags: bug, oracle - :tickets: 3548 - - Fixed bug in Oracle dialect where reflection of tables and other - symbols with names quoted to force all-lower-case would not be - identified properly in reflection queries. The :class:`.quoted_name` - construct is now applied to incoming symbol names that detect as - forced into all-lower-case within the "name normalize" process. - - .. change:: - :tags: feature, orm - - Added new method :meth:`_query.Query.one_or_none`; same as - :meth:`_query.Query.one` but returns None if no row found. Pull request - courtesy esiegerman. - - .. change:: - :tags: bug, orm - :tickets: 3539 - - Fixed rare TypeError which could occur when stringifying certain - kinds of internal column loader options within internal logging. - - .. change:: - :tags: bug, orm - :tickets: 3525 - - Fixed bug in :meth:`.Session.bulk_save_objects` where a mapped - column that had some kind of "fetch on update" value and was not - locally present in the given object would cause an AttributeError - within the operation. - - .. change:: - :tags: bug, sql - :tickets: 3520 - - Fixed regression in 1.0-released default-processor for multi-VALUES - insert statement, :ticket:`3288`, where the column type for the - default-holding column would not be propagated to the compiled - statement in the case where the default was being used, - leading to bind-level type handlers not being invoked. - - .. change:: - :tags: bug, examples - - - Fixed two issues in the "history_meta" example where history tracking - could encounter empty history, and where a column keyed to an alternate - attribute name would fail to track properly. Fixes courtesy - Alex Fraser. - - .. change:: - :tags: bug, orm - :tickets: 3510 - - - Fixed 1.0 regression where the "noload" loader strategy would fail - to function for a many-to-one relationship. The loader used an - API to place "None" into the dictionary which no longer actually - writes a value; this is a side effect of :ticket:`3061`. - - .. change:: - :tags: bug, sybase - :tickets: 3508, 3509 - - - Fixed two issues regarding Sybase reflection, allowing tables - without primary keys to be reflected as well as ensured that - a SQL statement involved in foreign key detection is pre-fetched up - front to avoid driver issues upon nested queries. Fixes here - courtesy Eugene Zapolsky; note that we cannot currently test - Sybase to locally verify these changes. - - .. change:: - :tags: bug, postgresql - - - An adjustment to the new PostgreSQL feature of reflecting storage - options and USING of :ticket:`3455` released in 1.0.6, - to disable the feature for PostgreSQL versions < 8.2 where the - ``reloptions`` column is not provided; this allows Amazon Redshift - to again work as it is based on an 8.0.x version of PostgreSQL. - Fix courtesy Pete Hollobon. - - -.. changelog:: - :version: 1.0.8 - :released: July 22, 2015 - - .. change:: - :tags: bug, misc - :tickets: 3494 - - Fixed an issue where a particular base class within utils - didn't implement ``__slots__``, and therefore meant all subclasses - of that class didn't either, negating the rationale for ``__slots__`` - to be in use. Didn't cause any issue except on IronPython - which apparently does not implement ``__slots__`` behavior compatibly - with cPython. - - -.. changelog:: - :version: 1.0.7 - :released: July 20, 2015 - - .. change:: - :tags: feature, sql - :tickets: 3459 - - Added a :meth:`_expression.ColumnElement.cast` method which performs the same - purpose as the standalone :func:`_expression.cast` function. Pull - request courtesy Sebastian Bank. - - .. change:: - :tags: bug, engine - :tickets: 3481 - - Fixed regression where new methods on :class:`_engine.ResultProxy` used - by the ORM :class:`_query.Query` object (part of the performance - enhancements of :ticket:`3175`) would not raise the "this result - does not return rows" exception in the case where the driver - (typically MySQL) fails to generate cursor.description correctly; - an AttributeError against NoneType would be raised instead. - - .. change:: - :tags: bug, engine - :tickets: 3483 - - Fixed regression where :meth:`_engine.ResultProxy.keys` would return - un-adjusted internal symbol names for "anonymous" labels, which - are the "foo_1" types of labels we see generated for SQL functions - without labels and similar. This was a side effect of the - performance enhancements implemented as part of #918. - - - .. change:: - :tags: bug, sql - :tickets: 3490 - - Fixed bug where coercion of literal ``True`` or ``False`` constant - in conjunction with :func:`.and_` or :func:`.or_` would fail - with an AttributeError. - - .. change:: - :tags: bug, sql - :tickets: 3485 - - Fixed potential issue where a custom subclass - of :class:`.FunctionElement` or other column element that incorrectly - states 'None' or any other invalid object as the ``.type`` - attribute will report this exception instead of recursion overflow. - - .. change:: - :tags: bug, sql - - Fixed bug where the modulus SQL operator wouldn't work in reverse - due to a missing ``__rmod__`` method. Pull request courtesy - dan-gittik. - - .. change:: - :tags: feature, schema - - Added support for the MINVALUE, MAXVALUE, NO MINVALUE, NO MAXVALUE, - and CYCLE arguments for CREATE SEQUENCE as supported by PostgreSQL - and Oracle. Pull request courtesy jakeogh. - - .. change:: - :tags: bug, orm, declarative - :tickets: 3480 - - Fixed bug in :class:`.AbstractConcreteBase` extension where - a column setup on the ABC base which had a different attribute - name vs. column name would not be correctly mapped on the final - base class. The failure on 0.9 would be silent whereas on - 1.0 it raised an ArgumentError, so may not have been noticed - prior to 1.0. - - .. change:: - :tags: bug, orm - :tickets: 3469 - - Fixed 1.0 regression where value objects that override - ``__eq__()`` to return a non-boolean-capable object, such as - some geoalchemy types as well as numpy types, were being tested - for ``bool()`` during a unit of work update operation, where in - 0.9 the return value of ``__eq__()`` was tested against "is True" - to guard against this. - - .. change:: - :tags: bug, orm - :tickets: 3468 - - Fixed 1.0 regression where a "deferred" attribute would not populate - correctly if it were loaded within the "optimized inheritance load", - which is a special SELECT emitted in the case of joined table - inheritance used to populate expired or unloaded attributes against - a joined table without loading the base table. This is related to - the fact that SQLA 1.0 no longer guesses about loading deferred - columns and must be directed explicitly. - - .. change:: - :tags: bug, orm - :tickets: 3466 - - Fixed 1.0 regression where the "parent entity" of a synonym- - mapped attribute on top of an :func:`.aliased` object would - resolve to the original mapper, not the :func:`.aliased` - version of it, thereby causing problems for a :class:`_query.Query` - that relies on this attribute (e.g. it's the only representative - attribute given in the constructor) to figure out the correct FROM - clause for the query. - -.. changelog:: - :version: 1.0.6 - :released: June 25, 2015 - - .. change:: - :tags: bug, orm - :tickets: 3465 - - Fixed a major regression in the 1.0 series where the version_id_counter - feature would cause an object's version counter to be incremented - when there was no net change to the object's row, but instead an object - related to it via relationship (e.g. typically many-to-one) - were associated or de-associated with it, resulting in an UPDATE - statement that updates the object's version counter and nothing else. - In the use case where the relatively recent "server side" and/or - "programmatic/conditional" version counter feature were used - (e.g. setting version_id_generator to False), the bug could cause an - UPDATE without a valid SET clause to be emitted. - - .. change:: - :tags: bug, mssql - :tickets: 3464 - - Fixed issue when using :class:`_types.VARBINARY` type in conjunction with - an INSERT of NULL + pyodbc; pyodbc requires a special - object be passed in order to persist NULL. As the :class:`_types.VARBINARY` - type is now usually the default for :class:`.LargeBinary` due to - :ticket:`3039`, this issue is partially a regression in 1.0. - The pymssql driver appears to be unaffected. - - .. change:: - :tags: bug, postgresql, pypy - :tickets: 3439 - - Re-fixed this issue first released in 1.0.5 to fix psycopg2cffi - JSONB support once again, as they suddenly - switched on unconditional decoding of JSONB types in version 2.7.1. - Version detection now specifies 2.7.1 as where we should expect - the DBAPI to do json encoding for us. - - .. change:: - :tags: feature, postgresql - :tickets: 3455 - - Added support for storage parameters under CREATE INDEX, using - a new keyword argument ``postgresql_with``. Also added support for - reflection to support both the ``postgresql_with`` flag as well - as the ``postgresql_using`` flag, which will now be set on - :class:`.Index` objects that are reflected, as well present - in a new "dialect_options" dictionary in the result of - :meth:`_reflection.Inspector.get_indexes`. Pull request courtesy Pete Hollobon. - - .. seealso:: - - :ref:`postgresql_index_storage` - - .. change:: - :tags: bug, orm - :tickets: 3462 - - Fixed 1.0 regression where the enhanced behavior of single-inheritance - joins of :ticket:`3222` takes place inappropriately - for a JOIN along explicit join criteria with a single-inheritance - subclass that does not make use of any discriminator, resulting - in an additional "AND NULL" clause. - - .. change:: - :tags: bug, postgresql - :tickets: 3454 - - Repaired the :class:`.ExcludeConstraint` construct to support common - features that other objects like :class:`.Index` now do, that - the column expression may be specified as an arbitrary SQL - expression such as :obj:`_expression.cast` or :obj:`_expression.text`. - - .. change:: - :tags: feature, postgresql - - Added new execution option ``max_row_buffer`` which is interpreted - by the psycopg2 dialect when the ``stream_results`` option is - used, which sets a limit on the size of the row buffer that may be - allocated. This value is also provided based on the integer - value sent to :meth:`_query.Query.yield_per`. Pull request courtesy - mcclurem. - - .. change:: - :tags: bug, orm - :tickets: 3451 - - Fixed bug in new :meth:`.Session.bulk_update_mappings` feature where - the primary key columns used in the WHERE clause to locate the row - would also be included in the SET clause, setting their value to - themselves unnecessarily. Pull request courtesy Patrick Hayes. - - .. change:: - :tags: bug, orm - :tickets: 3448 - - Fixed an unexpected-use regression whereby custom - :class:`.types.TypeEngine.Comparator` objects that made use of the - ``__clause_element__()`` method and returned an object that was an - ORM-mapped :class:`.InstrumentedAttribute` and not explicitly a - :class:`_expression.ColumnElement` would fail to be correctly handled when passed - as an expression to :meth:`.Session.query`. The logic in 0.9 happened - to succeed on this, so this use case is now supported. - - .. change:: - :tags: bug, sql - :tickets: 3445 - - Fixed a bug where clause adaption as applied to a :class:`.Label` - object would fail to accommodate the labeled SQL expression - in all cases, such that any SQL operation that made use of - :meth:`.Label.self_group` would use the original unadapted - expression. One effect of this would be that an ORM :func:`.aliased` - construct would not fully accommodate attributes mapped by - :obj:`.column_property`, such that the un-aliased table could - leak out when the property were used in some kinds of SQL - comparisons. - - .. change:: - :tags: bug, documentation - :tickets: 2077 - - Fixed an internal "memoization" routine for method types such - that a Python descriptor is no longer used; repairs inspectability - of these methods including support for Sphinx documentation. - -.. changelog:: - :version: 1.0.5 - :released: June 7, 2015 - - .. change:: - :tags: feature, engine - - Added new engine event :meth:`_events.ConnectionEvents.engine_disposed`. - Called after the :meth:`_engine.Engine.dispose` method is called. - - .. change:: - :tags: bug, postgresql, pypy - :tickets: 3439 - - Repaired some typing and test issues related to the pypy - psycopg2cffi dialect, in particular that the current 2.7.0 version - does not have native support for the JSONB type. The version detection - for psycopg2 features has been tuned into a specific sub-version - for psycopg2cffi. Additionally, test coverage has been enabled - for the full series of psycopg2 features under psycopg2cffi. - - .. change:: - :tags: feature, ext - - Added support for ``*args`` to be passed to the baked query - initial callable, in the same way that ``*args`` are supported - for the :meth:`.BakedQuery.add_criteria` and - :meth:`.BakedQuery.with_criteria` methods. Initial PR courtesy - Naoki INADA. - - .. change:: - :tags: bug, engine - :tickets: 3435 - - Fixed bug where known boolean values used by - :func:`.engine_from_config` were not being parsed correctly; - these included ``pool_threadlocal`` and the psycopg2 argument - ``use_native_unicode``. - - .. change:: - :tags: bug, mssql - :tickets: 3424, 3430 - - Added a new dialect flag to the MSSQL dialect - ``legacy_schema_aliasing`` which when set to False will disable a - very old and obsolete behavior, that of the compiler's - attempt to turn all schema-qualified table names into alias names, - to work around old and no longer locatable issues where SQL - server could not parse a multi-part identifier name in all - circumstances. The behavior prevented more - sophisticated statements from working correctly, including those which - use hints, as well as CRUD statements that embed correlated SELECT - statements. Rather than continue to repair the feature to work - with more complex statements, it's better to just disable it - as it should no longer be needed for any modern SQL server - version. The flag defaults to True for the 1.0.x series, leaving - current behavior unchanged for this version series. In the 1.1 - series, it will default to False. For the 1.0 series, - when not set to either value explicitly, a warning is emitted - when a schema-qualified table is first used in a statement, which - suggests that the flag be set to False for all modern SQL Server - versions. - - .. seealso:: - - :ref:`legacy_schema_rendering` - - .. change:: - :tags: feature, engine - :tickets: 3379 - - Adjustments to the engine plugin hook, such that the - :meth:`.URL.get_dialect` method will continue to return the - ultimate :class:`.Dialect` object when a dialect plugin is used, - without the need for the caller to be aware of the - :meth:`.Dialect.get_dialect_cls` method. - - - .. change:: - :tags: bug, ext - :tickets: 3427 - - Fixed regression in the :mod:`sqlalchemy.ext.mutable` extension - as a result of the bugfix for :ticket:`3167`, - where attribute and validation events are no longer - called within the flush process. The mutable - extension was relying upon this behavior in the case where a column - level Python-side default were responsible for generating the new value - on INSERT or UPDATE, or when a value were fetched from the RETURNING - clause for "eager defaults" mode. The new value would not be subject - to any event when populated and the mutable extension could not - establish proper coercion or history listening. A new event - :meth:`.InstanceEvents.refresh_flush` is added which the mutable - extension now makes use of for this use case. - - .. change:: - :tags: feature, orm - :tickets: 3427 - - Added new event :meth:`.InstanceEvents.refresh_flush`, invoked - when an INSERT or UPDATE level default value fetched via RETURNING - or Python-side default is invoked within the flush process. This - is to provide a hook that is no longer present as a result of - :ticket:`3167`, where attribute and validation events are no longer - called within the flush process. - - .. change:: - :tags: feature, ext - :tickets: 3427 - - Added a new semi-public method to :class:`.MutableBase` - :meth:`.MutableBase._get_listen_keys`. Overriding this method - is needed in the case where a :class:`.MutableBase` subclass needs - events to propagate for attribute keys other than the key to which - the mutable type is associated with, when intercepting the - :meth:`.InstanceEvents.refresh` or - :meth:`.InstanceEvents.refresh_flush` events. The current example of - this is composites using :class:`.MutableComposite`. - - .. change:: - :tags: bug, engine - :tickets: 3421 - - Added support for the case of the misbehaving DBAPI that has - pep-249 exception names linked to exception classes of an entirely - different name, preventing SQLAlchemy's own exception wrapping from - wrapping the error appropriately. - The SQLAlchemy dialect in use needs to implement a new - accessor :attr:`.DefaultDialect.dbapi_exception_translation_map` - to support this feature; this is implemented now for the py-postgresql - dialect. - - .. change:: - :tags: bug, orm - :tickets: 3420 - - The "lightweight named tuple" used when a :class:`_query.Query` returns - rows failed to implement ``__slots__`` correctly such that it still - had a ``__dict__``. This is resolved, but in the extremely - unlikely case someone was assigning values to the returned tuples, - that will no longer work. - - .. change:: - :tags: bug, engine - :tickets: 3419 - - Fixed bug involving the case when pool checkout event handlers are used - and connection attempts are made in the handler itself which fail, - the owning connection record would not be freed until the stack trace - of the connect error itself were freed. For the case where a test - pool of only a single connection were used, this means the pool would - be fully checked out until that stack trace were freed. This mostly - impacts very specific debugging scenarios and is unlikely to have been - noticeable in any production application. The fix applies an - explicit checkin of the record before re-raising the caught exception. - - -.. changelog:: - :version: 1.0.4 - :released: May 7, 2015 - - .. change:: - :tags: bug, orm - :tickets: 3416 - - Fixed unexpected-use regression where in the odd case that the - primaryjoin of a relationship involved comparison to an unhashable - type such as an HSTORE, lazy loads would fail due to a hash-oriented - check on the statement parameters, modified in 1.0 as a result of - :ticket:`3061` to use hashing and modified in :ticket:`3368` - to occur in cases more common than "load on pending". - The values are now checked for the ``__hash__`` attribute beforehand. - - .. change:: - :tags: bug, orm - :tickets: 3412, 3347 - - Liberalized an assertion that was added as part of :ticket:`3347` - to protect against unknown conditions when splicing inner joins - together within joined eager loads with ``innerjoin=True``; if - some of the joins use a "secondary" table, the assertion needs to - unwrap further joins in order to pass. - - .. change:: - :tags: bug, schema - :tickets: 3411 - - Fixed bug in enhanced constraint-attachment logic introduced in - :ticket:`3341` where in the unusual case of a constraint that refers - to a mixture of :class:`_schema.Column` objects and string column names - at the same time, the auto-attach-on-column-attach logic will be - skipped; for the constraint to be auto-attached in this case, - all columns must be assembled on the target table up front. - Added a new section to the migration document regarding the - original feature as well as this change. - - .. seealso:: - - :ref:`change_3341` - - .. change:: - :tags: bug, orm - :tickets: 3409, 3320 - - Repaired / added to tests yet more expressions that were reported - as failing with the new 'entity' key value added to - :attr:`_query.Query.column_descriptions`, the logic to discover the "from" - clause is again reworked to accommodate columns from aliased classes, - as well as to report the correct value for the "aliased" flag in these - cases. - - -.. changelog:: - :version: 1.0.3 - :released: April 30, 2015 - - .. change:: - :tags: bug, orm, pypy - :tickets: 3405 - - Fixed regression from 0.9.10 prior to release due to :ticket:`3349` - where the check for query state on :meth:`_query.Query.update` or - :meth:`_query.Query.delete` compared the empty tuple to itself using ``is``, - which fails on PyPy to produce ``True`` in this case; this would - erroneously emit a warning in 0.9 and raise an exception in 1.0. - - .. change:: - :tags: feature, engine - :tickets: 3379 - - New features added to support engine/pool plugins with advanced - functionality. Added a new "soft invalidate" feature to the - connection pool at the level of the checked out connection wrapper - as well as the :class:`._ConnectionRecord`. This works similarly - to a modern pool invalidation in that connections aren't actively - closed, but are recycled only on next checkout; this is essentially - a per-connection version of that feature. A new event - :meth:`_events.PoolEvents.soft_invalidate` is added to complement it. - - Also added new flag - :attr:`.ExceptionContext.invalidate_pool_on_disconnect`. - Allows an error handler within :meth:`_events.ConnectionEvents.handle_error` - to maintain a "disconnect" condition, but to handle calling invalidate - on individual connections in a specific manner within the event. - - .. change:: - :tags: feature, engine - :tickets: 3355 - - Added new event :class:`.DialectEvents.do_connect`, which allows - interception / replacement of when the :meth:`.Dialect.connect` - hook is called to create a DBAPI connection. Also added - dialect plugin hooks :meth:`.Dialect.get_dialect_cls` and - :meth:`.Dialect.engine_created` which allow external plugins to - add events to existing dialects using entry points. - - .. change:: - :tags: bug, orm - :tickets: 3403, 3320 - - Fixed regression from 0.9.10 prior to release where the new addition - of ``entity`` to the :attr:`_query.Query.column_descriptions` accessor - would fail if the target entity was produced from a core selectable - such as a :class:`_schema.Table` or :class:`_expression.CTE` object. - - .. change:: - :tags: feature, sql - - Added a placeholder method :meth:`.TypeEngine.compare_against_backend` - which is now consumed by Alembic migrations as of 0.7.6. User-defined - types can implement this method to assist in the comparison of - a type against one reflected from the database. - - .. change:: - :tags: bug, orm - :tickets: 3402 - - Fixed regression within the flush process when an attribute were - set to a SQL expression for an UPDATE, and the SQL expression when - compared to the previous value of the attribute would produce a SQL - comparison other than ``==`` or ``!=``, the exception "Boolean value - of this clause is not defined" would raise. The fix ensures that - the unit of work will not interpret the SQL expression in this way. - - .. change:: - :tags: bug, ext - :tickets: 3397 - - Fixed bug in association proxy where an any()/has() - on an relationship->scalar non-object attribute comparison would fail, - e.g. - ``filter(Parent.some_collection_to_attribute.any(Child.attr == 'foo'))`` - - .. change:: - :tags: bug, sql - :tickets: 3396 - - Fixed bug where the truncation of long labels in SQL could produce - a label that overlapped another label that is not truncated; this - because the length threshold for truncation was greater than - the portion of the label that remains after truncation. These - two values have now been made the same; label_length - 6. - The effect here is that shorter column labels will be "truncated" - where they would not have been truncated before. - - .. change:: - :tags: bug, orm - :tickets: 3392 - - Fixed unexpected use regression due to :ticket:`2992` where - textual elements placed - into the :meth:`_query.Query.order_by` clause in conjunction with joined - eager loading would be added to the columns clause of the inner query - in such a way that they were assumed to be table-bound column names, - in the case where the joined eager load needs to wrap the query - in a subquery to accommodate for a limit/offset. - - Originally, the behavior here was intentional, in that a query such - as ``query(User).order_by('name').limit(1)`` - would order by ``user.name`` even if the query was modified by - joined eager loading to be within a subquery, as ``'name'`` would - be interpreted as a symbol to be located within the FROM clauses, - in this case ``User.name``, which would then be copied into the - columns clause to ensure it were present for ORDER BY. However, the - feature fails to anticipate the case where ``order_by("name")`` refers - to a specific label name present in the local columns clause already - and not a name bound to a selectable in the FROM clause. - - Beyond that, the feature also fails for deprecated cases such as - ``order_by("name desc")``, which, while it emits a - warning that :func:`_expression.text` should be used here (note that the issue - does not impact cases where :func:`_expression.text` is used explicitly), - still produces a different query than previously where the "name desc" - expression is copied into the columns clause inappropriately. The - resolution is such that the "joined eager loading" aspect of the - feature will skip over these so-called "label reference" expressions - when augmenting the inner columns clause, as though they were - :func:`_expression.text` constructs already. - - .. change:: - :tags: bug, sql - :tickets: 3391 - - Fixed regression due to :ticket:`3282` where the ``tables`` collection - passed as a keyword argument to the :meth:`.DDLEvents.before_create`, - :meth:`.DDLEvents.after_create`, :meth:`.DDLEvents.before_drop`, and - :meth:`.DDLEvents.after_drop` events would no longer be a list - of tables, but instead a list of tuples which contained a second - entry with foreign keys to be added or dropped. As the ``tables`` - collection, while documented as not necessarily stable, has come - to be relied upon, this change is considered a regression. - Additionally, in some cases for "drop", this collection would - be an iterator that would cause the operation to fail if - prematurely iterated. The collection is now a list of table - objects in all cases and test coverage for the format of this - collection is now added. - - - .. change:: - :tags: bug, orm - :tickets: 3388 - - Fixed a regression regarding the :meth:`.MapperEvents.instrument_class` - event where its invocation was moved to be after the class manager's - instrumentation of the class, which is the opposite of what the - documentation for the event explicitly states. The rationale for the - switch was due to Declarative taking the step of setting up - the full "instrumentation manager" for a class before it was mapped - for the purpose of the new ``@declared_attr`` features - described in :ref:`feature_3150`, but the change was also made - against the classical use of :class:`_orm.Mapper` for consistency. - However, SQLSoup relies upon the instrumentation event happening - before any instrumentation under classical mapping. - The behavior is reverted in the case of classical and declarative - mapping, the latter implemented by using a simple memoization - without using class manager. - - .. change:: - :tags: bug, orm - :tickets: 3387 - - Fixed issue in new :meth:`.QueryEvents.before_compile` event where - changes made to the :class:`_query.Query` object's collection of entities - to load within the event would render in the SQL, but would not - be reflected during the loading process. - -.. changelog:: - :version: 1.0.2 - :released: April 24, 2015 - - .. change:: - :tags: bug, sql - :tickets: 3338, 3385 - - Fixed a regression that was incorrectly fixed in 1.0.0b4 - (hence becoming two regressions); reports that - SELECT statements would GROUP BY a label name and fail was misconstrued - that certain backends such as SQL Server should not be emitting - ORDER BY or GROUP BY on a simple label name at all; when in fact, - we had forgotten that 0.9 was already emitting ORDER BY on a simple - label name for all backends, as described in :ref:`migration_1068`, - even though 1.0 includes a rewrite of this logic as part of - :ticket:`2992`. As far - as emitting GROUP BY against a simple label, even PostgreSQL has - cases where it will raise an error even though the label to group - on should be apparent, so it is clear that GROUP BY should never - be rendered in this way automatically. - - In 1.0.2, SQL Server, Firebird and others will again emit ORDER BY on - a simple label name when passed a - :class:`.Label` construct that is also present in the columns clause. - Additionally, no backend will emit GROUP BY against the simple label - name only when passed a :class:`.Label` construct. - - .. change:: - :tags: bug, orm, declarative - :tickets: 3383 - - Fixed unexpected use regression regarding the declarative - ``__declare_first__`` and ``__declare_last__`` accessors where these - would no longer be called on the superclass of the declarative base. - -.. changelog:: - :version: 1.0.1 - :released: April 23, 2015 - - .. change:: - :tags: bug, firebird - :tickets: 3380 - - Fixed a regression due to :ticket:`3034` where limit/offset - clauses were not properly interpreted by the Firebird dialect. - Pull request courtesy effem-git. - - .. change:: - :tags: bug, firebird - :tickets: 3381 - - Fixed support for "literal_binds" mode when using limit/offset - with Firebird, so that the values are again rendered inline when - this is selected. Related to :ticket:`3034`. - - .. change:: - :tags: bug, sqlite - :tickets: 3378 - - Fixed a regression due to :ticket:`3282`, where due to the fact that - we attempt to assume the availability of ALTER when creating/dropping - schemas, in the case of SQLite we simply said to not worry about - foreign keys at all, since ALTER is not available, when creating - and dropping tables. This meant that the sorting of tables was - basically skipped in the case of SQLite, and for the vast majority - of SQLite use cases, this is not an issue. - - However, users who were doing DROPs on SQLite - with tables that contained data and with referential integrity - turned on would then experience errors, as the - dependency sorting *does* matter in the case of DROP with - enforced constraints, when those tables have data (SQLite will still - happily let you create foreign keys to nonexistent tables and drop - tables referring to existing ones with constraints enabled, as long as - there's no data being referenced). - - In order to maintain the new feature of :ticket:`3282` while still - allowing a SQLite DROP operation to maintain ordering, we now - do the sort with full FKs taken under consideration, and if we encounter - an unresolvable cycle, only *then* do we forego attempting to sort - the tables; we instead emit a warning and go with the unsorted list. - If an environment needs both ordered DROPs *and* has foreign key - cycles, then the warning notes they will need to restore the - ``use_alter`` flag to their :class:`_schema.ForeignKey` and - :class:`_schema.ForeignKeyConstraint` objects so that just those objects will - be omitted from the dependency sort. - - .. seealso:: - - :ref:`feature_3282` - contains an updated note about SQLite. - - .. change:: - :tags: bug, sql - :tickets: 3372 - - Fixed issue where a straight SELECT EXISTS query would fail to - assign the proper result type of Boolean to the result mapping, and - instead would leak column types from within the query into the - result map. This issue exists in 0.9 and earlier as well, however - has less of an impact in those versions. In 1.0, due to :ticket:`918` - this becomes a regression in that we now rely upon the result mapping - to be very accurate, else we can assign result-type processors to - the wrong column. In all versions, this issue also has the effect - that a simple EXISTS will not apply the Boolean type handler, leading - to simple 1/0 values for backends without native boolean instead of - True/False. The fix includes that an EXISTS columns argument - will be anon-labeled like other column expressions; a similar fix is - implemented for pure-boolean expressions like ``not_(True())``. - - .. change:: - :tags: bug, orm - :tickets: 3374 - - Fixed issue where a query of the form - ``query(B).filter(B.a != A(id=7))`` would render the ``NEVER_SET`` - symbol, when - given a transient object. For a persistent object, it would - always use the persisted database value and not the currently - set value. Assuming autoflush is turned on, this usually would - not be apparent for persistent values, as any pending changes - would be flushed first in any case. However, this is inconsistent - vs. the logic used for the non-negated comparison, - ``query(B).filter(B.a == A(id=7))``, which does use the - current value and additionally allows comparisons to transient - objects. The comparison now uses the current value and not - the database-persisted value. - - Unlike the other ``NEVER_SET`` issues that are repaired as regressions - caused by :ticket:`3061` in this release, this particular issue is - present at least as far back as 0.8 and possibly earlier, however it - was discovered as a result of repairing the related ``NEVER_SET`` - issues. - - .. seealso:: - - :ref:`bug_3374` - - .. change:: - :tags: bug, orm - :tickets: 3371 - - Fixed unexpected use regression cause by :ticket:`3061` where - the NEVER_SET - symbol could leak into relationship-oriented queries, including - ``filter()`` and ``with_parent()`` queries. The ``None`` symbol - is returned in all cases, however many of these queries have never - been correctly supported in any case, and produce comparisons - to NULL without using the IS operator. For this reason, a warning - is also added to that subset of relationship queries that don't - currently provide for ``IS NULL``. - - .. seealso:: - - :ref:`bug_3371` - - - .. change:: - :tags: bug, orm - :tickets: 3368 - - Fixed a regression caused by :ticket:`3061` where the - NEVER_SET symbol could leak into a lazyload query, subsequent - to the flush of a pending object. This would occur typically - for a many-to-one relationship that does not use a simple - "get" strategy. The good news is that the fix improves efficiency - vs. 0.9, because we can now skip the SELECT statement entirely - when we detect NEVER_SET symbols present in the parameters; prior to - :ticket:`3061`, we couldn't discern if the None here were set or not. - - -.. changelog:: - :version: 1.0.0 - :released: April 16, 2015 - - .. change:: - :tags: bug, orm - :tickets: 3367 - - Identified an inconsistency when handling :meth:`_query.Query.join` to the - same target more than once; it implicitly dedupes only in the case of - a relationship join, and due to :ticket:`3233`, in 1.0 a join - to the same table twice behaves differently than 0.9 in that it no - longer erroneously aliases. To help document this change, - the verbiage regarding :ticket:`3233` in the migration notes has - been generalized, and a warning has been added when :meth:`_query.Query.join` - is called against the same target relationship more than once. - - .. change:: - :tags: bug, orm - :tickets: 3364 - - Made a small improvement to the heuristics of relationship when - determining remote side with semi-self-referential (e.g. two joined - inh subclasses referring to each other), non-simple join conditions - such that the parententity is taken into account and can reduce the - need for using the ``remote()`` annotation; this can restore some - cases that might have worked without the annotation prior to 0.9.4 - via :ticket:`2948`. - - .. change:: - :tags: bug, mssql - :tickets: 3360 - - Fixed a regression where the "last inserted id" mechanics would - fail to store the correct value for MSSQL on an INSERT where the - primary key value was present in the insert params before execution, - as well as in the case where an INSERT from SELECT would state the - target columns as column objects, instead of string keys. - - - .. change:: - :tags: bug, mssql - - Using the ``Binary`` constructor now present in pymssql rather than - patching one in. Pull request courtesy Ramiro Morales. - - .. change:: - :tags: bug, tests - :tickets: 3356 - - Fixed the pathing used when tests run; for sqla_nose.py and py.test, - the "./lib" prefix is again inserted at the head of sys.path but - only if sys.flags.no_user_site isn't set; this makes it act just - like the way Python puts "." in the current path by default. - For tox, we are setting the PYTHONNOUSERSITE flag now. - - .. change:: - :tags: feature, sql - :tickets: 3084 - - The topological sorting used to sort :class:`_schema.Table` objects - and available via the :attr:`_schema.MetaData.sorted_tables` collection - will now produce a **deterministic** ordering; that is, the same - ordering each time given a set of tables with particular names - and dependencies. This is to help with comparison of DDL scripts - and other use cases. The tables are sent to the topological sort - sorted by name, and the topological sort itself will process - the incoming data in an ordered fashion. Pull request - courtesy Sebastian Bank. - - .. seealso:: - - :ref:`feature_3084` - - .. change:: - :tags: feature, orm - - Added new argument :paramref:`.Query.update.update_args` which allows - kw arguments such as ``mysql_limit`` to be passed to the underlying - :class:`_expression.Update` construct. Pull request courtesy Amir Sadoughi. - -.. changelog:: - :version: 1.0.0b5 - :released: April 3, 2015 - - .. change:: - :tags: bug, orm - :tickets: 3349 - - :class:`_query.Query` doesn't support joins, subselects, or special - FROM clauses when using the :meth:`_query.Query.update` or - :meth:`_query.Query.delete` methods; instead of silently ignoring these - fields if methods like :meth:`_query.Query.join` or - :meth:`_query.Query.select_from` has been called, an error is raised. - In 0.9.10 this only emits a warning. - - .. change:: - :tags: bug, orm - - Added a list() call around a weak dictionary used within the - commit phase of the session, which without it could cause - a "dictionary changed size during iter" error if garbage collection - interacted within the process. Change was introduced by - #3139. - - .. change:: - :tags: bug, postgresql - :tickets: 3343 - - Fixed bug where updated PG index reflection as a result of - :ticket:`3184` would cause index operations to fail on PostgreSQL - versions 8.4 and earlier. The enhancements are now - disabled when using an older version of PostgreSQL. - - .. change:: - :tags: bug, sql - :tickets: 3346 - - The warning emitted by the unicode type for a non-unicode type - has been liberalized to warn for values that aren't even string - values, such as integers; previously, the updated warning system - of 1.0 made use of string formatting operations which - would raise an internal TypeError. While these cases should ideally - raise totally, some backends like SQLite and MySQL do accept them - and are potentially in use by legacy code, not to mention that they - will always pass through if unicode conversion is turned off - for the target backend. - - .. change:: - :tags: bug, orm - :tickets: 3347 - - Fixed a bug related to "nested" inner join eager loading, which - exists in 0.9 as well but is more of a regression in 1.0 due to - :ticket:`3008` which turns on "nested" by default, such that - a joined eager load that travels across sibling paths from a common - ancestor using innerjoin=True will correctly splice each "innerjoin" - sibling into the appropriate part of the join, when a series of - inner/outer joins are mixed together. - -.. changelog:: - :version: 1.0.0b4 - :released: March 29, 2015 - - .. change:: - :tags: bug, mssql, oracle, firebird, sybase - :tickets: 3338 - - Turned off the "simple order by" flag on the MSSQL, Oracle dialects; - this is the flag that per :ticket:`2992` causes an order by or group by - an expression that's also in the columns clause to be copied by - label, even if referenced as the expression object. The behavior - for MSSQL is now the old behavior that copies the whole expression - in by default, as MSSQL can be picky on these particularly in - GROUP BY expressions. The flag is also turned off defensively - for the Firebird and Sybase dialects. - - .. note:: this resolution was incorrect, please see version 1.0.2 - for a rework of this resolution. - - .. change:: - :tags: feature, schema - :tickets: 3341 - - The "auto-attach" feature of constraints such as :class:`.UniqueConstraint` - and :class:`.CheckConstraint` has been further enhanced such that - when the constraint is associated with non-table-bound :class:`_schema.Column` - objects, the constraint will set up event listeners with the - columns themselves such that the constraint auto attaches at the - same time the columns are associated with the table. This in particular - helps in some edge cases in declarative but is also of general use. - - .. seealso:: - - :ref:`change_3341` - - .. change:: - :tags: bug, sql - :tickets: 3340 - - Fixed bug in new "label resolution" feature of :ticket:`2992` where - a label that was anonymous, then labeled again with a name, would - fail to be locatable via a textual label. This situation occurs - naturally when a mapped :func:`.column_property` is given an - explicit label in a query. - - .. change:: - :tags: bug, sql - :tickets: 3335 - - Fixed bug in new "label resolution" feature of :ticket:`2992` where - the string label placed in the order_by() or group_by() of a statement - would place higher priority on the name as found - inside the FROM clause instead of a more locally available name - inside the columns clause. - -.. changelog:: - :version: 1.0.0b3 - :released: March 20, 2015 - - .. change:: - :tags: bug, mysql - :tickets: 2771 - - Repaired the commit for issue #2771 which was inadvertently commented - out. - - -.. changelog:: - :version: 1.0.0b2 - :released: March 20, 2015 - - .. change:: - :tags: bug, mysql - :tickets: 2771 - - Fixes to fully support using the ``'utf8mb4'`` MySQL-specific charset - with MySQL dialects, in particular MySQL-Python and PyMySQL. In - addition, MySQL databases that report more unusual charsets such as - 'koi8u' or 'eucjpms' will also work correctly. Pull request - courtesy Thomas Grainger. - - .. change:: - :tags: change, orm, declarative - :tickets: 3331 - - Loosened some restrictions that were added to ``@declared_attr`` - objects, such that they were prevented from being called outside - of the declarative process; this is related to the enhancements - of #3150 which allow ``@declared_attr`` to return a value that is - cached based on the current class as it's being configured. - The exception raise has been removed, and the behavior changed - so that outside of the declarative process, the function decorated by - ``@declared_attr`` is called every time just like a regular - ``@property``, without using any caching, as none is available - at this stage. - - .. change:: - :tags: bug, engine - :tickets: 3330, 3329 - - The "auto close" for :class:`_engine.ResultProxy` is now a "soft" close. - That is, after exhausting all rows using the fetch methods, the - DBAPI cursor is released as before and the object may be safely - discarded, but the fetch methods may continue to be called for which - they will return an end-of-result object (None for fetchone, empty list - for fetchmany and fetchall). Only if :meth:`_engine.ResultProxy.close` - is called explicitly will these methods raise the "result is closed" - error. - - .. seealso:: - - :ref:`change_3330` - - .. change:: - :tags: bug, orm - :tickets: 3327 - - Fixed unexpected use regression from pullreq github:137 where - Py2K unicode literals (e.g. ``u""``) would not be accepted by the - :paramref:`_orm.relationship.cascade` option. - Pull request courtesy Julien Castets. - - -.. changelog:: - :version: 1.0.0b1 - :released: March 13, 2015 - - Version 1.0.0b1 is the first release of the 1.0 series. Many changes - described here are also present in the 0.9 and sometimes the 0.8 - series as well. For changes that are specific to 1.0 with an emphasis - on compatibility concerns, see :doc:`/changelog/migration_10`. - - .. change:: - :tags: feature, ext - :tickets: 3054 - - Added a new extension suite :mod:`sqlalchemy.ext.baked`. This - simple but unusual system allows for a dramatic savings in Python - overhead for the construction and processing of orm :class:`_query.Query` - objects, from query construction up through rendering of a string - SQL statement. - - .. seealso:: - - :ref:`baked_toplevel` - - .. change:: - :tags: bug, postgresql - :tickets: 3319 - - The PostgreSQL :class:`_postgresql.ENUM` type will emit a - DROP TYPE instruction when a plain ``table.drop()`` is called, - assuming the object is not associated directly with a - :class:`_schema.MetaData` object. In order to accommodate the use case of - an enumerated type shared between multiple tables, the type should - be associated directly with the :class:`_schema.MetaData` object; in this - case the type will only be created at the metadata level, or if - created directly. The rules for create/drop of - PostgreSQL enumerated types have been highly reworked in general. - - .. seealso:: - - :ref:`change_3319` - - .. change:: - :tags: feature, orm - :tickets: 3317 - - Added a new event suite :class:`.QueryEvents`. The - :meth:`.QueryEvents.before_compile` event allows the creation - of functions which may place additional modifications to - :class:`_query.Query` objects before the construction of the SELECT - statement. It is hoped that this event be made much more - useful via the advent of a new inspection system that will - allow for detailed modifications to be made against - :class:`_query.Query` objects in an automated fashion. - - .. seealso:: - - :class:`.QueryEvents` - - - .. change:: - :tags: feature, orm - :tickets: 3249 - - The subquery wrapping which occurs when joined eager loading - is used with a one-to-many query that also features LIMIT, - OFFSET, or DISTINCT has been disabled in the case of a one-to-one - relationship, that is a one-to-many with - :paramref:`_orm.relationship.uselist` set to False. This will produce - more efficient queries in these cases. - - .. seealso:: - - :ref:`change_3249` - - - .. change:: - :tags: bug, orm - :tickets: 3301 - - Fixed bug where the session attachment error "object is already - attached to session X" would fail to prevent the object from - also being attached to the new session, in the case that execution - continued after the error raise occurred. - - .. change:: - :tags: bug, orm, declarative - :tickets: 3219, 3240 - - Fixed bug where using an ``__abstract__`` mixin in the middle - of a declarative inheritance hierarchy would prevent attributes - and configuration being correctly propagated from the base class - to the inheriting class. - - .. change:: - :tags: feature, sql - :tickets: 918 - - The SQL compiler now generates the mapping of expected columns - such that they are matched to the received result set positionally, - rather than by name. Originally, this was seen as a way to handle - cases where we had columns returned with difficult-to-predict names, - though in modern use that issue has been overcome by anonymous - labeling. In this version, the approach basically reduces function - call count per-result by a few dozen calls, or more for larger - sets of result columns. The approach still degrades into a modern - version of the old approach if any discrepancy in size exists between - the compiled set of columns versus what was received, so there's no - issue for partially or fully textual compilation scenarios where these - lists might not line up. - - .. change:: - :tags: feature, postgresql - - The PG8000 dialect now supports the - :paramref:`_sa.create_engine.encoding` parameter, by setting up - the client encoding on the connection which is then intercepted - by pg8000. Pull request courtesy Tony Locke. - - .. change:: - :tags: feature, postgresql - - Added support for PG8000's native JSONB feature. Pull request - courtesy Tony Locke. - - .. change:: - :tags: change, orm - - Mapped attributes marked as deferred without explicit undeferral - will now remain "deferred" even if their column is otherwise - present in the result set in some way. This is a performance - enhancement in that an ORM load no longer spends time searching - for each deferred column when the result set is obtained. However, - for an application that has been relying upon this, an explicit - :func:`.undefer` or similar option should now be used. - - .. change:: - :tags: feature, orm - :tickets: 3307 - - Mapped state internals have been reworked to allow for a 50% reduction - in callcounts specific to the "expiration" of objects, as in - the "auto expire" feature of :meth:`.Session.commit` and - for :meth:`.Session.expire_all`, as well as in the "cleanup" step - which occurs when object states are garbage collected. - - .. change:: - :tags: bug, mysql - - The MySQL dialect now supports CAST on types that are constructed - as :class:`.TypeDecorator` objects. - - .. change:: - :tags: bug, mysql - :tickets: 3237 - - A warning is emitted when :func:`.cast` is used with the MySQL - dialect on a type where MySQL does not support CAST; MySQL only - supports CAST on a subset of datatypes. SQLAlchemy has for a long - time just omitted the CAST for unsupported types in the case of - MySQL. While we don't want to change this now, we emit a warning - to show that it's taken place. A warning is also emitted when - a CAST is used with an older MySQL version (< 4) that doesn't support - CAST at all, it's skipped in this case as well. - - .. change:: - :tags: feature, sql - :tickets: 3087 - - Literal values within a :class:`.DefaultClause`, which is invoked - when using the :paramref:`_schema.Column.server_default` parameter, will - now be rendered using the "inline" compiler, so that they are rendered - as-is, rather than as bound parameters. - - .. seealso:: - - :ref:`change_3087` - - .. change:: - :tags: feature, oracle - - Added support for cx_oracle connections to a specific service - name, as opposed to a tns name, by passing ``?service_name=`` - to the URL. Pull request courtesy Sławomir Ehlert. - - .. change:: - :tags: feature, mysql - :tickets: 3155 - - The MySQL dialect now renders TIMESTAMP with NULL / NOT NULL in - all cases, so that MySQL 5.6.6 with the - ``explicit_defaults_for_timestamp`` flag enabled will - will allow TIMESTAMP to continue to work as expected when - ``nullable=False``. Existing applications are unaffected as - SQLAlchemy has always emitted NULL for a TIMESTAMP column that - is ``nullable=True``. - - .. seealso:: - - :ref:`change_3155` - - :ref:`mysql_timestamp_null` - - .. change:: - :tags: bug, schema - :tickets: 3299, 3067 - - The :class:`.CheckConstraint` construct now supports naming - conventions that include the token ``%(column_0_name)s``; the - constraint expression is scanned for columns. Additionally, - naming conventions for check constraints that don't include the - ``%(constraint_name)s`` token will now work for :class:`.SchemaType`- - generated constraints, such as those of :class:`.Boolean` and - :class:`.Enum`; this stopped working in 0.9.7 due to :ticket:`3067`. - - .. seealso:: - - :ref:`naming_check_constraints` - - :ref:`naming_schematypes` - - - .. change:: - :tags: feature, postgresql, pypy - :tickets: 3052 - - Added support for the psycopg2cffi DBAPI on pypy. Pull request - courtesy shauns. - - .. seealso:: - - :mod:`sqlalchemy.dialects.postgresql.psycopg2cffi` - - .. change:: - :tags: feature, orm - :tickets: 3262 - - A warning is emitted when the same polymorphic identity is assigned - to two different mappers in the same hierarchy. This is typically a - user error and means that the two different mapping types cannot be - correctly distinguished at load time. Pull request courtesy - Sebastian Bank. - - .. change:: - :tags: feature, sql - - The type of expression is reported when an object passed to a - SQL expression unit can't be interpreted as a SQL fragment; - pull request courtesy Ryan P. Kelly. - - .. change:: - :tags: bug, orm - :tickets: 3227, 3242, 1326 - - The primary :class:`_orm.Mapper` of a :class:`_query.Query` is now passed to the - :meth:`.Session.get_bind` method when calling upon - :meth:`_query.Query.count`, :meth:`_query.Query.update`, :meth:`_query.Query.delete`, - as well as queries against mapped columns, - :obj:`.column_property` objects, and SQL functions and expressions - derived from mapped columns. This allows sessions that rely upon - either customized :meth:`.Session.get_bind` schemes or "bound" metadata - to work in all relevant cases. - - .. seealso:: - - :ref:`bug_3227` - - .. change:: - :tags: enhancement, sql - :tickets: 3074 - - Custom dialects that implement :class:`.GenericTypeCompiler` can - now be constructed such that the visit methods receive an indication - of the owning expression object, if any. Any visit method that - accepts keyword arguments (e.g. ``**kw``) will in most cases - receive a keyword argument ``type_expression``, referring to the - expression object that the type is contained within. For columns - in DDL, the dialect's compiler class may need to alter its - ``get_column_specification()`` method to support this as well. - The ``UserDefinedType.get_col_spec()`` method will also receive - ``type_expression`` if it provides ``**kw`` in its argument - signature. - - .. change:: - :tags: bug, sql - :tickets: 3288 - - The multi-values version of :meth:`_expression.Insert.values` has been - repaired to work more usefully with tables that have Python- - side default values and/or functions, as well as server-side - defaults. The feature will now work with a dialect that uses - "positional" parameters; a Python callable will also be - invoked individually for each row just as is the case with an - "executemany" style invocation; a server- side default column - will no longer implicitly receive the value explicitly - specified for the first row, instead refusing to invoke - without an explicit value. - - .. seealso:: - - :ref:`bug_3288` - - .. change:: - :tags: feature, general - - Structural memory use has been improved via much more significant use - of ``__slots__`` for many internal objects. This optimization is - particularly geared towards the base memory size of large applications - that have lots of tables and columns, and greatly reduces memory - size for a variety of high-volume objects including event listening - internals, comparator objects and parts of the ORM attribute and - loader strategy system. - - .. seealso:: - - :ref:`feature_slots` - - .. change:: - :tags: bug, mysql - :tickets: 3283 - - The :class:`.mysql.SET` type has been overhauled to no longer - assume that the empty string, or a set with a single empty string - value, is in fact a set with a single empty string; instead, this - is by default treated as the empty set. In order to handle persistence - of a :class:`.mysql.SET` that actually wants to include the blank - value ``''`` as a legitimate value, a new bitwise operational mode - is added which is enabled by the - :paramref:`.mysql.SET.retrieve_as_bitwise` flag, which will persist - and retrieve values unambiguously using their bitflag positioning. - Storage and retrieval of unicode values for driver configurations - that aren't converting unicode natively is also repaired. - - .. seealso:: - - :ref:`change_3283` - - - .. change:: - :tags: feature, schema - :tickets: 3282 - - The DDL generation system of :meth:`_schema.MetaData.create_all` - and :meth:`_schema.MetaData.drop_all` has been enhanced to in most - cases automatically handle the case of mutually dependent - foreign key constraints; the need for the - :paramref:`_schema.ForeignKeyConstraint.use_alter` flag is greatly - reduced. The system also works for constraints which aren't given - a name up front; only in the case of DROP is a name required for - at least one of the constraints involved in the cycle. - - .. seealso:: - - :ref:`feature_3282` - - .. change:: - :tags: feature, schema - - Added a new accessor :attr:`_schema.Table.foreign_key_constraints` - to complement the :attr:`_schema.Table.foreign_keys` collection, - as well as :attr:`_schema.ForeignKeyConstraint.referred_table`. - - .. change:: - :tags: bug, sqlite - :tickets: 3244, 3261 - - UNIQUE and FOREIGN KEY constraints are now fully reflected on - SQLite both with and without names. Previously, foreign key - names were ignored and unnamed unique constraints were skipped. - Thanks to Jon Nelson for assistance with this. - - .. change:: - :tags: feature, examples - - A new suite of examples dedicated to providing a detailed study - into performance of SQLAlchemy ORM and Core, as well as the DBAPI, - from multiple perspectives. The suite runs within a container - that provides built in profiling displays both through console - output as well as graphically via the RunSnake tool. - - .. seealso:: - - :ref:`examples_performance` - - .. change:: - :tags: feature, orm - :tickets: 3100 - - A new series of :class:`.Session` methods which provide hooks - directly into the unit of work's facility for emitting INSERT - and UPDATE statements has been created. When used correctly, - this expert-oriented system can allow ORM-mappings to be used - to generate bulk insert and update statements batched into - executemany groups, allowing the statements to proceed at - speeds that rival direct use of the Core. - - .. seealso:: - - :ref:`bulk_operations` - - .. change:: - :tags: feature, mssql - :tickets: 3039 - - SQL Server 2012 now recommends VARCHAR(max), NVARCHAR(max), - VARBINARY(max) for large text/binary types. The MSSQL dialect will - now respect this based on version detection, as well as the new - ``deprecate_large_types`` flag. - - .. seealso:: - - :ref:`mssql_large_type_deprecation` - - .. change:: - :tags: bug, sqlite - :tickets: 3257 - - The SQLite dialect, when using the :class:`_sqlite.DATE`, - :class:`_sqlite.TIME`, - or :class:`_sqlite.DATETIME` types, and given a ``storage_format`` that - only renders numbers, will render the types in DDL as - ``DATE_CHAR``, ``TIME_CHAR``, and ``DATETIME_CHAR``, so that despite the - lack of alpha characters in the values, the column will still - deliver the "text affinity". Normally this is not needed, as the - textual values within the default storage formats already - imply text. - - .. seealso:: - - :ref:`sqlite_datetime` - - .. change:: - :tags: bug, engine - :tickets: 3266 - - The engine-level error handling and wrapping routines will now - take effect in all engine connection use cases, including - when user-custom connect routines are used via the - :paramref:`_sa.create_engine.creator` parameter, as well as when - the :class:`_engine.Connection` encounters a connection error on - revalidation. - - .. seealso:: - - :ref:`change_3266` - - .. change:: - :tags: feature, oracle - - New Oracle DDL features for tables, indexes: COMPRESS, BITMAP. - Patch courtesy Gabor Gombas. - - .. change:: - :tags: bug, oracle - - An alias name will be properly quoted when referred to using the - ``%(name)s`` token inside the :meth:`_expression.Select.with_hint` method. - Previously, the Oracle backend hadn't implemented this quoting. - - .. change:: - :tags: feature, oracle - :tickets: 3220 - - Added support for CTEs under Oracle. This includes some tweaks - to the aliasing syntax, as well as a new CTE feature - :meth:`_expression.CTE.suffix_with`, which is useful for adding in special - Oracle-specific directives to the CTE. - - .. seealso:: - - :ref:`change_3220` - - .. change:: - :tags: feature, mysql - :tickets: 3121 - - Updated the "supports_unicode_statements" flag to True for MySQLdb - and Pymysql under Python 2. This refers to the SQL statements - themselves, not the parameters, and affects issues such as table - and column names using non-ASCII characters. These drivers both - appear to support Python 2 Unicode objects without issue in modern - versions. - - .. change:: - :tags: bug, mysql - :tickets: 3263 - - The :meth:`.ColumnOperators.match` operator is now handled such that the - return type is not strictly assumed to be boolean; it now - returns a :class:`.Boolean` subclass called :class:`.MatchType`. - The type will still produce boolean behavior when used in Python - expressions, however the dialect can override its behavior at - result time. In the case of MySQL, while the MATCH operator - is typically used in a boolean context within an expression, - if one actually queries for the value of a match expression, a - floating point value is returned; this value is not compatible - with SQLAlchemy's C-based boolean processor, so MySQL's result-set - behavior now follows that of the :class:`.Float` type. - A new operator object ``notmatch_op`` is also added to better allow - dialects to define the negation of a match operation. - - .. seealso:: - - :ref:`change_3263` - - .. change:: - :tags: bug, postgresql - :tickets: 3264 - - The :meth:`.PGDialect.has_table` method will now query against - ``pg_catalog.pg_table_is_visible(c.oid)``, rather than testing - for an exact schema match, when the schema name is None; this - so that the method will also illustrate that temporary tables - are present. Note that this is a behavioral change, as PostgreSQL - allows a non-temporary table to silently overwrite an existing - temporary table of the same name, so this changes the behavior - of ``checkfirst`` in that unusual scenario. - - .. seealso:: - - :ref:`change_3264` - - .. change:: - :tags: bug, sql - :tickets: 3260 - - Fixed bug in :meth:`_schema.Table.tometadata` method where the - :class:`.CheckConstraint` associated with a :class:`.Boolean` - or :class:`.Enum` type object would be doubled in the target table. - The copy process now tracks the production of this constraint object - as local to a type object. - - .. change:: - :tags: feature, orm - :tickets: 3217 - - Added a parameter :paramref:`.Query.join.isouter` which is synonymous - with calling :meth:`_query.Query.outerjoin`; this flag is to provide a more - consistent interface compared to Core :meth:`_expression.FromClause.join`. - Pull request courtesy Jonathan Vanasco. - - .. change:: - :tags: bug, sql - :tickets: 3243 - - The behavioral contract of the :attr:`_schema.ForeignKeyConstraint.columns` - collection has been made consistent; this attribute is now a - :class:`_expression.ColumnCollection` like that of all other constraints and - is initialized at the point when the constraint is associated with - a :class:`_schema.Table`. - - .. seealso:: - - :ref:`change_3243` - - .. change:: - :tags: bug, orm - :tickets: 3256 - - The :meth:`.PropComparator.of_type` modifier has been - improved in conjunction with loader directives such as - :func:`_orm.joinedload` and :func:`.contains_eager` such that if - two :meth:`.PropComparator.of_type` modifiers of the same - base type/path are encountered, they will be joined together - into a single "polymorphic" entity, rather than replacing - the entity of type A with the one of type B. E.g. - a joinedload of ``A.b.of_type(BSub1)->BSub1.c`` combined with - joinedload of ``A.b.of_type(BSub2)->BSub2.c`` will create a - single joinedload of ``A.b.of_type((BSub1, BSub2)) -> BSub1.c, BSub2.c``, - without the need for the ``with_polymorphic`` to be explicit - in the query. - - .. seealso:: - - :ref:`eagerloading_polymorphic_subtypes` - contains an updated - example illustrating the new format. - - .. change:: - :tags: bug, sql - :tickets: 3245 - - The :attr:`_schema.Column.key` attribute is now used as the source of - anonymous bound parameter names within expressions, to match the - existing use of this value as the key when rendered in an INSERT - or UPDATE statement. This allows :attr:`_schema.Column.key` to be used - as a "substitute" string to work around a difficult column name - that doesn't translate well into a bound parameter name. Note that - the paramstyle is configurable on :func:`_sa.create_engine` in any case, - and most DBAPIs today support a named and positional style. - - .. change:: - :tags: bug, sql - - Fixed the name of the :paramref:`.PoolEvents.reset.dbapi_connection` - parameter as passed to this event; in particular this affects - usage of the "named" argument style for this event. Pull request - courtesy Jason Goldberger. - - .. change:: - :tags: feature, sql - - Added a new parameter :paramref:`.Table.tometadata.name` to - the :meth:`_schema.Table.tometadata` method. Similar to - :paramref:`.Table.tometadata.schema`, this argument causes the newly - copied :class:`_schema.Table` to take on the new name instead of - the existing one. An interesting capability this adds is that of - copying a :class:`_schema.Table` object to the *same* :class:`_schema.MetaData` - target with a new name. Pull request courtesy n.d. parker. - - .. change:: - :tags: bug, orm - - Repaired support of the ``copy.deepcopy()`` call when used by the - :class:`.orm.util.CascadeOptions` argument, which occurs - if ``copy.deepcopy()`` is being used with :func:`_orm.relationship` - (not an officially supported use case). Pull request courtesy - duesenfranz. - - .. change:: - :tags: bug, sql - :tickets: 3170 - - Reversing a change that was made in 0.9, the "singleton" nature - of the "constants" :func:`.null`, :func:`.true`, and :func:`.false` - has been reverted. These functions returning a "singleton" object - had the effect that different instances would be treated as the - same regardless of lexical use, which in particular would impact - the rendering of the columns clause of a SELECT statement. - - .. seealso:: - - :ref:`bug_3170` - - .. change:: - :tags: bug, orm - :tickets: 3139 - - Fixed bug where :meth:`.Session.expunge` would not fully detach - the given object if the object had been subject to a delete - operation that was flushed, but not committed. This would also - affect related operations like :func:`.make_transient`. - - .. seealso:: - - :ref:`bug_3139` - - .. change:: - :tags: bug, orm - :tickets: 3230 - - A warning is emitted in the case of multiple relationships that - ultimately will populate a foreign key column in conflict with - another, where the relationships are attempting to copy values - from different source columns. This occurs in the case where - composite foreign keys with overlapping columns are mapped to - relationships that each refer to a different referenced column. - A new documentation section illustrates the example as well as how - to overcome the issue by specifying "foreign" columns specifically - on a per-relationship basis. - - .. seealso:: - - :ref:`relationship_overlapping_foreignkeys` - - .. change:: - :tags: feature, sql - :tickets: 3172 - - Exception messages have been spiffed up a bit. The SQL statement - and parameters are not displayed if None, reducing confusion for - error messages that weren't related to a statement. The full - module and classname for the DBAPI-level exception is displayed, - making it clear that this is a wrapped DBAPI exception. The - statement and parameters themselves are bounded within a bracketed - sections to better isolate them from the error message and from - each other. - - .. change:: - :tags: bug, orm - :tickets: 3228 - - The :meth:`_query.Query.update` method will now convert string key - names in the given dictionary of values into mapped attribute names - against the mapped class being updated. Previously, string names - were taken in directly and passed to the core update statement without - any means to resolve against the mapped entity. Support for synonyms - and hybrid attributes as the subject attributes of - :meth:`_query.Query.update` are also supported. - - .. seealso:: - - :ref:`bug_3228` - - .. change:: - :tags: bug, orm - :tickets: 3035 - - Improvements to the mechanism used by :class:`.Session` to locate - "binds" (e.g. engines to use), such engines can be associated with - mixin classes, concrete subclasses, as well as a wider variety - of table metadata such as joined inheritance tables. - - .. seealso:: - - :ref:`bug_3035` - - .. change:: - :tags: bug, general - :tickets: 3218 - - The ``__module__`` attribute is now set for all those SQL and - ORM functions that are derived as "public factory" symbols, which - should assist with documentation tools being able to report on the - target module. - - .. change:: - :tags: feature, sql - - :meth:`_expression.Insert.from_select` now includes Python and SQL-expression - defaults if otherwise unspecified; the limitation where non- - server column defaults aren't included in an INSERT FROM - SELECT is now lifted and these expressions are rendered as - constants into the SELECT statement. - - .. seealso:: - - :ref:`feature_insert_from_select_defaults` - - .. change:: - :tags: bug, orm - :tickets: 3233 - - Fixed bug in single table inheritance where a chain of joins - that included the same single inh entity more than once - (normally this should raise an error) could, in some cases - depending on what was being joined "from", implicitly alias the - second case of the single inh entity, producing - a query that "worked". But as this implicit aliasing is not - intended in the case of single table inheritance, it didn't - really "work" fully and was very misleading, since it wouldn't - always appear. - - .. seealso:: - - :ref:`bug_3233` - - - .. change:: - :tags: bug, orm - :tickets: 3222 - - The ON clause rendered when using :meth:`_query.Query.join`, - :meth:`_query.Query.outerjoin`, or the standalone :func:`_orm.join` / - :func:`_orm.outerjoin` functions to a single-inheritance subclass will - now include the "single table criteria" in the ON clause even - if the ON clause is otherwise hand-rolled; it is now added to the - criteria using AND, the same way as if joining to a single-table - target using relationship or similar. - - This is sort of in-between feature and bug. - - .. seealso:: - - :ref:`migration_3222` - - .. change:: - :tags: feature, sql - :tickets: 3184 - - The :class:`.UniqueConstraint` construct is now included when - reflecting a :class:`_schema.Table` object, for databases where this - is applicable. In order to achieve this - with sufficient accuracy, MySQL and PostgreSQL now contain features - that correct for the duplication of indexes and unique constraints - when reflecting tables, indexes, and constraints. - In the case of MySQL, there is not actually a "unique constraint" - concept independent of a "unique index", so for this backend - :class:`.UniqueConstraint` continues to remain non-present for a - reflected :class:`_schema.Table`. For PostgreSQL, the query used to - detect indexes against ``pg_index`` has been improved to check for - the same construct in ``pg_constraint``, and the implicitly - constructed unique index is not included with a - reflected :class:`_schema.Table`. - - In both cases, the :meth:`_reflection.Inspector.get_indexes` and the - :meth:`_reflection.Inspector.get_unique_constraints` methods return both - constructs individually, but include a new token - ``duplicates_constraint`` in the case of PostgreSQL or - ``duplicates_index`` in the case - of MySQL to indicate when this condition is detected. - Pull request courtesy Johannes Erdfelt. - - .. seealso:: - - :ref:`feature_3184` - - .. change:: - :tags: feature, postgresql - - Added support for the FILTER keyword as applied to aggregate - functions, supported by PostgreSQL 9.4. Pull request - courtesy Ilja Everilä. - - .. seealso:: - - :ref:`feature_gh134` - - .. change:: - :tags: bug, sql, engine - :tickets: 3215 - - Fixed bug where a "branched" connection, that is the kind you get - when you call :meth:`_engine.Connection.connect`, would not share invalidation - status with the parent. The architecture of branching has been tweaked - a bit so that the branched connection defers to the parent for - all invalidation status and operations. - - .. change:: - :tags: bug, sql, engine - :tickets: 3190 - - Fixed bug where a "branched" connection, that is the kind you get - when you call :meth:`_engine.Connection.connect`, would not share transaction - status with the parent. The architecture of branching has been tweaked - a bit so that the branched connection defers to the parent for - all transactional status and operations. - - .. change:: - :tags: bug, orm, declarative - :tickets: 2670 - - A relationship set up with :class:`.declared_attr` on - a :class:`.AbstractConcreteBase` base class will now be configured - on the abstract base mapping automatically, in addition to being - set up on descendant concrete classes as usual. - - .. seealso:: - - :ref:`feature_3150` - - .. change:: - :tags: feature, orm, declarative - :tickets: 3150 - - The :class:`.declared_attr` construct has newly improved - behaviors and features in conjunction with declarative. The - decorated function will now have access to the final column - copies present on the local mixin when invoked, and will also - be invoked exactly once for each mapped class, the returned result - being memoized. A new modifier :attr:`.declared_attr.cascading` - is added as well. - - .. seealso:: - - :ref:`feature_3150` - - .. change:: - :tags: feature, ext - :tickets: 3210 - - The :mod:`sqlalchemy.ext.automap` extension will now set - ``cascade="all, delete-orphan"`` automatically on a one-to-many - relationship/backref where the foreign key is detected as containing - one or more non-nullable columns. This argument is present in the - keywords passed to :func:`.automap.generate_relationship` in this - case and can still be overridden. Additionally, if the - :class:`_schema.ForeignKeyConstraint` specifies ``ondelete="CASCADE"`` - for a non-nullable or ``ondelete="SET NULL"`` for a nullable set - of columns, the argument ``passive_deletes=True`` is also added to the - relationship. Note that not all backends support reflection of - ondelete, but backends that do include PostgreSQL and MySQL. - - .. change:: - :tags: feature, sql - :tickets: 3206 - - Added new method :meth:`_expression.Select.with_statement_hint` and ORM - method :meth:`_query.Query.with_statement_hint` to support statement-level - hints that are not specific to a table. - - .. change:: - :tags: bug, sqlite - :tickets: 3203 - - SQLite now supports reflection of unique constraints from - temp tables; previously, this would fail with a TypeError. - Pull request courtesy Johannes Erdfelt. - - .. seealso:: - - :ref:`change_3204` - changes regarding SQLite temporary - table and view reflection. - - .. change:: - :tags: bug, sqlite - :tickets: 3204 - - Added :meth:`_reflection.Inspector.get_temp_table_names` and - :meth:`_reflection.Inspector.get_temp_view_names`; currently, only the - SQLite and Oracle dialects support these methods. The return of - temporary table and view names has been **removed** from SQLite and - Oracle's version of :meth:`_reflection.Inspector.get_table_names` and - :meth:`_reflection.Inspector.get_view_names`; other database backends cannot - support this information (such as MySQL), and the scope of operation - is different in that the tables can be local to a session and - typically aren't supported in remote schemas. - - .. seealso:: - - :ref:`change_3204` - - .. change:: - :tags: feature, postgresql - :tickets: 2891 - - Support has been added for reflection of materialized views - and foreign tables, as well as support for materialized views - within :meth:`_reflection.Inspector.get_view_names`, and a new method - :meth:`.PGInspector.get_foreign_table_names` available on the - PostgreSQL version of :class:`_reflection.Inspector`. Pull request courtesy - Rodrigo Menezes. - - .. seealso:: - - :ref:`feature_2891` - - - .. change:: - :tags: feature, orm - - Added new event handlers :meth:`.AttributeEvents.init_collection` - and :meth:`.AttributeEvents.dispose_collection`, which track when - a collection is first associated with an instance and when it is - replaced. These handlers supersede the :meth:`.collection.linker` - annotation. The old hook remains supported through an event adapter. - - .. change:: - :tags: bug, orm - :tickets: 3148, 3188 - - A major rework to the behavior of expression labels, most - specifically when used with ColumnProperty constructs with - custom SQL expressions and in conjunction with the "order by - labels" logic first introduced in 0.9. Fixes include that an - ``order_by(Entity.some_col_prop)`` will now make use of "order by - label" rules even if Entity has been subject to aliasing, - either via inheritance rendering or via the use of the - ``aliased()`` construct; rendering of the same column property - multiple times with aliasing (e.g. ``query(Entity.some_prop, - entity_alias.some_prop)``) will label each occurrence of the - entity with a distinct label, and additionally "order by - label" rules will work for both (e.g. - ``order_by(Entity.some_prop, entity_alias.some_prop)``). - Additional issues that could prevent the "order by label" - logic from working in 0.9, most notably that the state of a - Label could change such that "order by label" would stop - working depending on how things were called, has been fixed. - - .. seealso:: - - :ref:`bug_3188` - - - .. change:: - :tags: bug, mysql - :tickets: 3186 - - MySQL boolean symbols "true", "false" work again. 0.9's change - in :ticket:`2682` disallowed the MySQL dialect from making use of the - "true" and "false" symbols in the context of "IS" / "IS NOT", but - MySQL supports this syntax even though it has no boolean type. - MySQL remains "non native boolean", but the :func:`.true` - and :func:`.false` symbols again produce the - keywords "true" and "false", so that an expression like - ``column.is_(true())`` again works on MySQL. - - .. seealso:: - - :ref:`bug_3186` - - .. change:: - :tags: changed, mssql - :tickets: 3182 - - The hostname-based connection format for SQL Server when using - pyodbc will no longer specify a default "driver name", and a warning - is emitted if this is missing. The optimal driver name for SQL Server - changes frequently and is per-platform, so hostname based connections - need to specify this. DSN-based connections are preferred. - - .. seealso:: - - :ref:`change_3182` - - .. change:: - :tags: changed, sql - - The :func:`_expression.column` and :func:`_expression.table` - constructs are now importable from the "from sqlalchemy" namespace, - just like every other Core construct. - - .. change:: - :tags: changed, sql - :tickets: 2992 - - The implicit conversion of strings to :func:`_expression.text` constructs - when passed to most builder methods of :func:`_expression.select` as - well as :class:`_query.Query` now emits a warning with just the - plain string sent. The textual conversion still proceeds normally, - however. The only method that accepts a string without a warning - are the "label reference" methods like order_by(), group_by(); - these functions will now at compile time attempt to resolve a single - string argument to a column or label expression present in the - selectable; if none is located, the expression still renders, but - you get the warning again. The rationale here is that the implicit - conversion from string to text is more unexpected than not these days, - and it is better that the user send more direction to the Core / ORM - when passing a raw string as to what direction should be taken. - Core/ORM tutorials have been updated to go more in depth as to how text - is handled. - - .. seealso:: - - :ref:`migration_2992` - - - .. change:: - :tags: feature, engine - :tickets: 3178 - - A new style of warning can be emitted which will "filter" up to - N occurrences of a parameterized string. This allows parameterized - warnings that can refer to their arguments to be delivered a fixed - number of times until allowing Python warning filters to squelch them, - and prevents memory from growing unbounded within Python's - warning registries. - - .. seealso:: - - :ref:`feature_3178` - - .. change:: - :tags: feature, orm - - The :class:`_query.Query` will raise an exception when :meth:`_query.Query.yield_per` - is used with mappings or options where either - subquery eager loading, or joined eager loading with collections, - would take place. These loading strategies are - not currently compatible with yield_per, so by raising this error, - the method is safer to use. Eager loads can be disabled with - the ``lazyload('*')`` option or :meth:`_query.Query.enable_eagerloads`. - - .. seealso:: - - :ref:`migration_yield_per_eager_loading` - - .. change:: - :tags: bug, orm - :tickets: 3177 - - Changed the approach by which the "single inheritance criterion" - is applied, when using :meth:`_query.Query.from_self`, or its common - user :meth:`_query.Query.count`. The criteria to limit rows to those - with a certain type is now indicated on the inside subquery, - not the outside one, so that even if the "type" column is not - available in the columns clause, we can filter on it on the "inner" - query. - - .. seealso:: - - :ref:`migration_3177` - - .. change:: - :tags: changed, orm - - The ``proc()`` callable passed to the ``create_row_processor()`` - method of custom :class:`.Bundle` classes now accepts only a single - "row" argument. - - .. seealso:: - - :ref:`bundle_api_change` - - .. change:: - :tags: changed, orm - - Deprecated event hooks removed: ``populate_instance``, - ``create_instance``, ``translate_row``, ``append_result`` - - .. seealso:: - - :ref:`migration_deprecated_orm_events` - - .. change:: - :tags: bug, orm - :tickets: 3145 - - Made a small adjustment to the mechanics of lazy loading, - such that it has less chance of interfering with a joinload() in the - very rare circumstance that an object points to itself; in this - scenario, the object refers to itself while loading its attributes - which can cause a mixup between loaders. The use case of - "object points to itself" is not fully supported, but the fix also - removes some overhead so for now is part of testing. - - .. change:: - :tags: feature, orm - :tickets: 3176 - - A new implementation for :class:`.KeyedTuple` used by the - :class:`_query.Query` object offers dramatic speed improvements when - fetching large numbers of column-oriented rows. - - .. seealso:: - - :ref:`feature_3176` - - .. change:: - :tags: feature, orm - :tickets: 3008 - - The behavior of :paramref:`_orm.joinedload.innerjoin` as well as - :paramref:`_orm.relationship.innerjoin` is now to use "nested" - inner joins, that is, right-nested, as the default behavior when an - inner join joined eager load is chained to an outer join eager load. - - .. seealso:: - - :ref:`migration_3008` - - .. change:: - :tags: bug, orm - :tickets: 3171 - - The "resurrect" ORM event has been removed. This event hook had - no purpose since the old "mutable attribute" system was removed - in 0.8. - - .. change:: - :tags: bug, sql - :tickets: 3169 - - Using :meth:`_expression.Insert.from_select` now implies ``inline=True`` - on :func:`_expression.insert`. This helps to fix a bug where an - INSERT...FROM SELECT construct would inadvertently be compiled - as "implicit returning" on supporting backends, which would - cause breakage in the case of an INSERT that inserts zero rows - (as implicit returning expects a row), as well as arbitrary - return data in the case of an INSERT that inserts multiple - rows (e.g. only the first row of many). - A similar change is also applied to an INSERT..VALUES - with multiple parameter sets; implicit RETURNING will no longer emit - for this statement either. As both of these constructs deal - with variable numbers of rows, the - :attr:`_engine.ResultProxy.inserted_primary_key` accessor does not - apply. Previously, there was a documentation note that one - may prefer ``inline=True`` with INSERT..FROM SELECT as some databases - don't support returning and therefore can't do "implicit" returning, - but there's no reason an INSERT...FROM SELECT needs implicit returning - in any case. Regular explicit :meth:`_expression.Insert.returning` should - be used to return variable numbers of result rows if inserted - data is needed. - - .. change:: - :tags: bug, orm - :tickets: 3167 - - Fixed bug where attribute "set" events or columns with - ``@validates`` would have events triggered within the flush process, - when those columns were the targets of a "fetch and populate" - operation, such as an autoincremented primary key, a Python side - default, or a server-side default "eagerly" fetched via RETURNING. - - .. change:: - :tags: feature, oracle - - Added support for the Oracle table option ON COMMIT. - - .. change:: - :tags: feature, postgresql - :tickets: 2051 - - Added support for PG table options TABLESPACE, ON COMMIT, - WITH(OUT) OIDS, and INHERITS, when rendering DDL via - the :class:`_schema.Table` construct. Pull request courtesy - malikdiarra. - - .. seealso:: - - :ref:`postgresql_table_options` - - .. change:: - :tags: bug, orm, py3k - - The :class:`.IdentityMap` exposed from :attr:`.Session.identity_map` - now returns lists for ``items()`` and ``values()`` in Py3K. - Early porting to Py3K here had these returning iterators, when - they technically should be "iterable views"..for now, lists are OK. - - .. change:: - :tags: orm, feature - - UPDATE statements can now be batched within an ORM flush - into more performant executemany() call, similarly to how INSERT - statements can be batched; this will be invoked within flush - to the degree that subsequent UPDATE statements for the - same mapping and table involve the identical columns within the - VALUES clause, that no SET-level SQL expressions - are embedded, and that the versioning requirements for the mapping - are compatible with the backend dialect's ability to return - a correct rowcount for an executemany operation. - - .. change:: - :tags: engine, bug - :tickets: 3163 - - Removing (or adding) an event listener at the same time that the event - is being run itself, either from inside the listener or from a - concurrent thread, now raises a RuntimeError, as the collection used is - now an instance of ``collections.deque()`` and does not support changes - while being iterated. Previously, a plain Python list was used where - removal from inside the event itself would produce silent failures. - - .. change:: - :tags: orm, feature - :tickets: 2963 - - The ``info`` parameter has been added to the constructor for - :class:`.SynonymProperty` and :class:`.ComparableProperty`. - - .. change:: - :tags: sql, feature - :tickets: 2963 - - The ``info`` parameter has been added as a constructor argument - to all schema constructs including :class:`_schema.MetaData`, - :class:`.Index`, :class:`_schema.ForeignKey`, :class:`_schema.ForeignKeyConstraint`, - :class:`.UniqueConstraint`, :class:`.PrimaryKeyConstraint`, - :class:`.CheckConstraint`. - - .. change:: - :tags: orm, feature - :tickets: 2971 - - The :attr:`.InspectionAttr.info` collection is now moved down to - :class:`.InspectionAttr`, where in addition to being available - on all :class:`.MapperProperty` objects, it is also now available - on hybrid properties, association proxies, when accessed via - :attr:`_orm.Mapper.all_orm_descriptors`. - - .. change:: - :tags: sql, feature - :tickets: 3027 - - The :paramref:`_schema.Table.autoload_with` flag now implies that - :paramref:`_schema.Table.autoload` should be ``True``. Pull request - courtesy Malik Diarra. - - .. change:: - :tags: postgresql, feature - - Added new method :meth:`.PGInspector.get_enums`, when using the - inspector for PostgreSQL will provide a list of ENUM types. - Pull request courtesy Ilya Pekelny. - - .. change:: - :tags: mysql, bug - - The MySQL dialect will now disable :meth:`_events.ConnectionEvents.handle_error` - events from firing for those statements which it uses internally - to detect if a table exists or not. This is achieved using an - execution option ``skip_user_error_events`` that disables the handle - error event for the scope of that execution. In this way, user code - that rewrites exceptions doesn't need to worry about the MySQL - dialect or other dialects that occasionally need to catch - SQLAlchemy specific exceptions. - - .. change:: - :tags: mysql, bug - :tickets: 2515 - - Changed the default value of "raise_on_warnings" to False for - MySQLconnector. This was set at True for some reason. The "buffered" - flag unfortunately must stay at True as MySQLconnector does not allow - a cursor to be closed unless all results are fully fetched. - - .. change:: - :tags: bug, orm - :tickets: 3117 - - The "evaluator" for query.update()/delete() won't work with multi-table - updates, and needs to be set to `synchronize_session=False` or - `synchronize_session='fetch'`; this now raises an exception, with a - message to change the synchronize setting. - This is upgraded from a warning emitted as of 0.9.7. - - .. change:: - :tags: removed - - The Drizzle dialect has been removed from the Core; it is now - available as `sqlalchemy-drizzle `_, - an independent, third party dialect. The dialect is still based - almost entirely off of the MySQL dialect present in SQLAlchemy. - - .. seealso:: - - :ref:`change_2984` - - .. change:: - :tags: enhancement, orm - :tickets: 3061 - - Adjustment to attribute mechanics concerning when a value is - implicitly initialized to None via first access; this action, - which has always resulted in a population of the attribute, - no longer does so; the None value is returned but the underlying - attribute receives no set event. This is consistent with how collections - work and allows attribute mechanics to behave more consistently; - in particular, getting an attribute with no value does not squash - the event that should proceed if the value is actually set to None. - - .. seealso:: - - :ref:`migration_3061` - - .. change:: - :tags: feature, sql - :tickets: 3034 - - The :meth:`_expression.Select.limit` and :meth:`_expression.Select.offset` methods - now accept any SQL expression, in addition to integer values, as - arguments. Typically this is used to allow a bound parameter to be - passed, which can be substituted with a value later thus allowing - Python-side caching of the SQL query. The implementation - here is fully backwards compatible with existing third party dialects, - however those dialects which implement special LIMIT/OFFSET systems - will need modification in order to take advantage of the new - capabilities. Limit and offset also support "literal_binds" mode, - where bound parameters are rendered inline as strings based on - a compile-time option. - Work on this feature is courtesy of Dobes Vandermeer. - - - .. seealso:: - - :ref:`feature_3034`. diff --git a/doc/build/changelog/changelog_11.rst b/doc/build/changelog/changelog_11.rst deleted file mode 100644 index c84effc3905..00000000000 --- a/doc/build/changelog/changelog_11.rst +++ /dev/null @@ -1,2739 +0,0 @@ -============= -1.1 Changelog -============= - -.. changelog_imports:: - - .. include:: changelog_10.rst - :start-line: 5 - - - .. include:: changelog_09.rst - :start-line: 5 - - - .. include:: changelog_08.rst - :start-line: 5 - - - .. include:: changelog_07.rst - :start-line: 5 - - -.. changelog:: - :version: 1.1.18 - :released: March 6, 2018 - - .. change:: - :tags: bug, mysql - :tickets: 4205 - :versions: 1.2.5 - - MySQL dialects now query the server version using ``SELECT @@version`` - explicitly to the server to ensure we are getting the correct version - information back. Proxy servers like MaxScale interfere with the value - that is passed to the DBAPI's connection.server_version value so this - is no longer reliable. - - .. change:: - :tags: bug, postgresql, py3k - :tickets: 4208 - :versions: 1.2.5 - - Fixed bug in PostgreSQL COLLATE / ARRAY adjustment first introduced - in :ticket:`4006` where new behaviors in Python 3.7 regular expressions - caused the fix to fail. - -.. changelog:: - :version: 1.1.17 - :released: February 22, 2018 - - .. change:: - :tags: bug, ext - :tickets: 4185 - - Repaired regression caused in 1.2.3 and 1.1.16 regarding association proxy - objects, revising the approach to :ticket:`4185` when calculating the - "owning class" of an association proxy to default to choosing the current - class if the proxy object is not directly associated with a mapped class, - such as a mixin. - -.. changelog:: - :version: 1.1.16 - :released: February 16, 2018 - - .. change:: - :tags: bug, postgresql - :versions: 1.2.3 - - Added "SSL SYSCALL error: Operation timed out" to the list - of messages that trigger a "disconnect" scenario for the - psycopg2 driver. Pull request courtesy André Cruz. - - .. change:: - :tags: bug, orm - :tickets: 4187 - :versions: 1.2.3 - - Fixed issue in post_update feature where an UPDATE is emitted - when the parent object has been deleted but the dependent object - is not. This issue has existed for a long time however - since 1.2 now asserts rows matched for post_update, this - was raising an error. - - .. change:: - :tags: bug, mysql - :tickets: 4136 - :versions: 1.2.0b4 - - Fixed bug where the MySQL "concat" and "match" operators failed to - propagate kwargs to the left and right expressions, causing compiler - options such as "literal_binds" to fail. - - .. change:: - :tags: bug, sql - :versions: 1.2.0b4 - - Added :func:`.nullsfirst` and :func:`.nullslast` as top level imports - in the ``sqlalchemy.`` and ``sqlalchemy.sql.`` namespace. Pull request - courtesy Lele Gaifax. - - .. change:: - :tags: bug, orm - :tickets: 4185 - :versions: 1.2.3 - - Fixed regression caused by fix for issue :ticket:`4116` affecting versions - 1.2.2 as well as 1.1.15, which had the effect of mis-calculation of the - "owning class" of an :class:`.AssociationProxy` as the ``NoneType`` class - in some declarative mixin/inheritance situations as well as if the - association proxy were accessed off of an un-mapped class. The "figure out - the owner" logic has been replaced by an in-depth routine that searches - through the complete mapper hierarchy assigned to the class or subclass to - determine the correct (we hope) match; will not assign the owner if no - match is found. An exception is now raised if the proxy is used - against an un-mapped instance. - - - .. change:: - :tags: bug, sql - :tickets: 4162 - :versions: 1.2.1 - - Fixed bug in :meth:`_expression.Insert.values` where using the "multi-values" - format in combination with :class:`_schema.Column` objects as keys rather - than strings would fail. Pull request courtesy Aubrey Stark-Toller. - - .. change:: - :tags: bug, postgresql - :versions: 1.2.3 - - Added "TRUNCATE" to the list of keywords accepted by the - PostgreSQL dialect as an "autocommit"-triggering keyword. - Pull request courtesy Jacob Hayes. - - .. change:: - :tags: bug, pool - :tickets: 4184 - :versions: 1.2.3 - - Fixed a fairly serious connection pool bug where a connection that is - acquired after being refreshed as a result of a user-defined - :class:`_exc.DisconnectionError` or due to the 1.2-released "pre_ping" feature - would not be correctly reset if the connection were returned to the pool by - weakref cleanup (e.g. the front-facing object is garbage collected); the - weakref would still refer to the previously invalidated DBAPI connection - which would have the reset operation erroneously called upon it instead. - This would lead to stack traces in the logs and a connection being checked - into the pool without being reset, which can cause locking issues. - - - .. change:: - :tags: bug, orm - :tickets: 4151 - :versions: 1.2.1 - - Fixed bug where an object that is expunged during a rollback of - a nested or subtransaction which also had its primary key mutated - would not be correctly removed from the session, causing subsequent - issues in using the session. - -.. changelog:: - :version: 1.1.15 - :released: November 3, 2017 - - .. change:: - :tags: bug, sqlite - :tickets: 4099 - :versions: 1.2.0b3 - - Fixed bug where SQLite CHECK constraint reflection would fail - if the referenced table were in a remote schema, e.g. on SQLite a - remote database referred to by ATTACH. - - .. change:: - :tags: bug, mysql - :tickets: 4097 - :versions: 1.2.0b3 - - Warning emitted when MariaDB 10.2.8 or earlier in the 10.2 - series is detected as there are major issues with CHECK - constraints within these versions that were resolved as of - 10.2.9. - - Note that this changelog message was NOT released with - SQLAlchemy 1.2.0b3 and was added retroactively. - - .. change:: - :tags: bug, mssql - :tickets: 4095 - :versions: 1.2.0b3 - - Added a full range of "connection closed" exception codes to the - PyODBC dialect for SQL Server, including '08S01', '01002', '08003', - '08007', '08S02', '08001', 'HYT00', 'HY010'. Previously, only '08S01' - was covered. - - .. change:: - :tags: bug, sql - :tickets: 4126 - :versions: 1.2.0 - - Fixed bug where ``__repr__`` of :class:`.ColumnDefault` would fail - if the argument were a tuple. Pull request courtesy Nicolas Caniart. - - .. change:: - :tags: bug, orm, declarative - :tickets: 4124 - :versions: 1.2.0 - - Fixed bug where a descriptor that is elsewhere a mapped column - or relationship within a hierarchy based on :class:`.AbstractConcreteBase` - would be referred towards during a refresh operation, causing an error - as the attribute is not mapped as a mapper property. - A similar issue can arise for other attributes like the "type" column - added by :class:`.AbstractConcreteBase` if the class fails to include - "concrete=True" in its mapper, however the check here should also - prevent that scenario from causing a problem. - - .. change:: 4006 - :tags: bug, postgresql - :tickets: 4006 - :versions: 1.2.0b3 - - Made further fixes to the :class:`_types.ARRAY` class in conjunction with - COLLATE, as the fix made in :ticket:`4006` failed to accommodate - for a multidimensional array. - - .. change:: - :tags: bug, orm, ext - :tickets: 4116 - :versions: 1.2.0 - - Fixed bug where the association proxy would inadvertently link itself - to an :class:`.AliasedClass` object if it were called first with - the :class:`.AliasedClass` as a parent, causing errors upon subsequent - usage. - - .. change:: - :tags: bug, mysql - :tickets: 4120 - :versions: 1.2.0 - - MySQL 5.7.20 now warns for use of the @tx_isolation variable; a version - check is now performed and uses @transaction_isolation instead - to prevent this warning. - - .. change:: - :tags: bug, postgresql - :tickets: 4107 - :versions: 1.2.0b3 - - Fixed bug in :obj:`_functions.array_agg` function where passing an argument - that is already of type :class:`_types.ARRAY`, such as a PostgreSQL - :obj:`_postgresql.array` construct, would produce a ``ValueError``, due - to the function attempting to nest the arrays. - - .. change:: - :tags: bug, orm - :tickets: 4078 - :versions: 1.2.0b3 - - Fixed bug where ORM relationship would warn against conflicting sync - targets (e.g. two relationships would both write to the same column) for - sibling classes in an inheritance hierarchy, where the two relationships - would never actually conflict during writes. - - .. change:: - :tags: bug, postgresql - :tickets: 4074 - :versions: 1.2.0b3 - - Fixed bug in PostgreSQL :meth:`.postgresql.dml.Insert.on_conflict_do_update` - which would prevent the insert statement from being used as a CTE, - e.g. via :meth:`_expression.Insert.cte`, within another statement. - - .. change:: - :tags: bug, orm - :tickets: 4103 - :versions: 1.2.0b3 - - Fixed bug where correlated select used against single-table inheritance - entity would fail to render correctly in the outer query, due to adjustment - for single inheritance discriminator criteria inappropriately re-applying - the criteria to the outer query. - - .. change:: - :tags: bug, mysql - :tickets: 4096 - :versions: 1.2.0b3 - - Fixed issue where CURRENT_TIMESTAMP would not reflect correctly - in the MariaDB 10.2 series due to a syntax change, where the function - is now represented as ``current_timestamp()``. - - .. change:: - :tags: bug, mysql - :tickets: 4098 - :versions: 1.2.0b3 - - MariaDB 10.2 now supports CHECK constraints (warning: use version 10.2.9 - or greater due to upstream issues noted in :ticket:`4097`). Reflection - now takes these CHECK constraints into account when they are present in - the ``SHOW CREATE TABLE`` output. - - .. change:: - :tags: bug, sql - :tickets: 4093 - :versions: 1.2.0b3 - - Fixed bug where the recently added :meth:`.ColumnOperators.any_` - and :meth:`.ColumnOperators.all_` methods didn't work when called - as methods, as opposed to using the standalone functions - :func:`_expression.any_` and :func:`_expression.all_`. Also - added documentation examples for these relatively unintuitive - SQL operators. - -.. changelog:: - :version: 1.1.14 - :released: September 5, 2017 - - .. change:: - :tags: bug, orm - :tickets: 4069 - :versions: 1.2.0b3 - - Fixed bug in :meth:`.Session.merge` following along similar lines as that - of :ticket:`4030`, where an internal check for a target object in - the identity map could lead to an error if it were to be garbage collected - immediately before the merge routine actually retrieves the object. - - .. change:: - :tags: bug, orm - :tickets: 4048 - :versions: 1.2.0b3 - - Fixed bug where an :func:`.undefer_group` option would not be recognized - if it extended from a relationship that was loading using joined eager - loading. Additionally, as the bug led to excess work being performed, - Python function call counts are also improved by 20% within the initial - calculation of result set columns, complementing the joined eager load - improvements of :ticket:`3915`. - - .. change:: - :tags: bug, orm - :tickets: 4068 - - Fixed race condition in ORM identity map which would cause objects - to be inappropriately removed during a load operation, causing - duplicate object identities to occur, particularly under joined eager - loading which involves deduplication of objects. The issue is specific - to garbage collection of weak references and is observed only under the - PyPy interpreter. - - .. change:: - :tags: bug, orm - :tickets: 4056 - :versions: 1.2.0b3 - - Fixed bug in :meth:`.Session.merge` where objects in a collection that had - the primary key attribute set to ``None`` for a key that is typically - autoincrementing would be considered to be a database-persisted key for - part of the internal deduplication process, causing only one object to - actually be inserted in the database. - - .. change:: - :tags: bug, sql - :tickets: 4053 - - Altered the range specification for window functions to allow - for two of the same PRECEDING or FOLLOWING keywords in a range - by allowing for the left side of the range to be positive - and for the right to be negative, e.g. (1, 3) is - "1 FOLLOWING AND 3 FOLLOWING". - - .. change:: - :tags: bug, orm - :tickets: 4067 - :versions: 1.2.0b3 - - An :class:`.InvalidRequestError` is raised when a :func:`.synonym` - is used against an attribute that is not against a :class:`.MapperProperty`, - such as an association proxy. Previously, a recursion overflow would - occur trying to locate non-existent attributes. - -.. changelog:: - :version: 1.1.13 - :released: August 3, 2017 - -.. changelog:: - :version: 1.1.12 - :released: July 24, 2017 - - .. change:: cache_order_sequence - :tags: feature, oracle, postgresql - :versions: 1.2.0b1 - - Added new keywords :paramref:`.Sequence.cache` and - :paramref:`.Sequence.order` to :class:`.Sequence`, to allow rendering - of the CACHE parameter understood by Oracle and PostgreSQL, and the - ORDER parameter understood by Oracle. Pull request - courtesy David Moore. - - - .. change:: 4033 - :tags: bug, orm - :tickets: 4033 - :versions: 1.2.0b2 - - Fixed regression from 1.1.11 where adding additional non-entity - columns to a query that includes an entity with subqueryload - relationships would fail, due to an inspection added in 1.1.11 as a - result of :ticket:`4011`. - - - .. change:: 4031 - :tags: bug, orm - :versions: 1.2.0b2 - :tickets: 4031 - - Fixed bug involving JSON NULL evaluation logic added in 1.1 as part - of :ticket:`3514` where the logic would not accommodate ORM - mapped attributes named differently from the :class:`_schema.Column` - that was mapped. - - .. change:: 4030 - :tags: bug, orm - :versions: 1.2.0b2 - :tickets: 4030 - - Added ``KeyError`` checks to all methods within - :class:`.WeakInstanceDict` where a check for ``key in dict`` is - followed by indexed access to that key, to guard against a race against - garbage collection that under load can remove the key from the dict - after the code assumes its present, leading to very infrequent - ``KeyError`` raises. - -.. changelog:: - :version: 1.1.11 - :released: Monday, June 19, 2017 - - .. change:: 4012 - :tags: bug, sql - :tickets: 4012 - :versions: 1.2.0b1 - - Fixed AttributeError which would occur in :class:`.WithinGroup` - construct during an iteration of the structure. - - .. change:: 4011 - :tags: bug, orm - :tickets: 4011 - :versions: 1.2.0b1 - - Fixed issue with subquery eagerloading which continues on from - the series of issues fixed in :ticket:`2699`, :ticket:`3106`, - :ticket:`3893` involving that the "subquery" contains the correct - FROM clause when beginning from a joined inheritance subclass - and then subquery eager loading onto a relationship from - the base class, while the query also includes criteria against - the subclass. The fix in the previous tickets did not accommodate - for additional subqueryload operations loading more deeply from - the first level, so the fix has been further generalized. - - .. change:: 4005 - :tags: bug, postgresql - :tickets: 4005 - :versions: 1.2.0b1 - - Continuing with the fix that correctly handles PostgreSQL - version string "10devel" released in 1.1.8, an additional regexp - bump to handle version strings of the form "10beta1". While - PostgreSQL now offers better ways to get this information, we - are sticking w/ the regexp at least through 1.1.x for the least - amount of risk to compatibility w/ older or alternate PostgreSQL - databases. - - .. change:: 4006 - :tags: bug, postgresql - :tickets: 4006 - :versions: 1.2.0b1 - - Fixed bug where using :class:`_types.ARRAY` with a string type that - features a collation would fail to produce the correct syntax - within CREATE TABLE. - - .. change:: 4007 - :tags: bug, mysql - :tickets: 4007 - :versions: 1.2.0b1 - - MySQL 5.7 has introduced permission limiting for the "SHOW VARIABLES" - command; the MySQL dialect will now handle when SHOW returns no - row, in particular for the initial fetch of SQL_MODE, and will - emit a warning that user permissions should be modified to allow the - row to be present. - - .. change:: 3994 - :tags: bug, mssql - :tickets: 3994 - :versions: 1.2.0b1 - - Fixed bug where SQL Server transaction isolation must be fetched - from a different view when using Azure data warehouse, the query - is now attempted against both views and then a NotImplemented - is raised unconditionally if failure continues to provide the - best resiliency against future arbitrary API changes in new - SQL Server versions. - - .. change:: 3997 - :tags: bug, oracle - :tickets: 3997 - :versions: 1.2.0b1 - - Support for two-phase transactions has been removed entirely for - cx_Oracle when version 6.0b1 or later of the DBAPI is in use. The two- - phase feature historically has never been usable under cx_Oracle 5.x in - any case, and cx_Oracle 6.x has removed the connection-level "twophase" - flag upon which this feature relied. - - .. change:: 3973 - :tags: bug, mssql - :tickets: 3973 - :versions: 1.2.0b1 - - Added a placeholder type :class:`_mssql.XML` to the SQL Server - dialect, so that a reflected table which includes this type can - be re-rendered as a CREATE TABLE. The type has no special round-trip - behavior nor does it currently support additional qualifying - arguments. - -.. changelog:: - :version: 1.1.10 - :released: Friday, May 19, 2017 - - .. change:: 3986 - :tags: bug, orm - :versions: 1.2.0b1 - :tickets: 3986 - - Fixed bug where a cascade such as "delete-orphan" (but others as well) - would fail to locate an object linked to a relationship that itself - is local to a subclass in an inheritance relationship, thus causing - the operation to not take place. - - .. change:: 3975 - :tags: bug, oracle - :versions: 1.2.0b1 - :tickets: 3975 - - Fixed bug in cx_Oracle dialect where version string parsing would - fail for cx_Oracle version 6.0b1 due to the "b" character. Version - string parsing is now via a regexp rather than a simple split. - - .. change:: 3949 - :tags: bug, schema - :versions: 1.2.0b1 - :tickets: 3949 - - An :class:`.ArgumentError` is now raised if a - :class:`_schema.ForeignKeyConstraint` object is created with a mismatched - number of "local" and "remote" columns, which otherwise causes the - internal state of the constraint to be incorrect. Note that this - also impacts the condition where a dialect's reflection process - produces a mismatched set of columns for a foreign key constraint. - - .. change:: 3980 - :tags: bug, ext - :versions: 1.2.0b1 - :tickets: 3980 - - Protected against testing "None" as a class in the case where - declarative classes are being garbage collected and new - automap prepare() operations are taking place concurrently, very - infrequently hitting a weakref that has not been fully acted upon - after gc. - - .. change:: - :tags: bug, postgresql - :versions: 1.2.0b1 - - Added "autocommit" support for GRANT, REVOKE keywords. Pull request - courtesy Jacob Hayes. - - .. change:: 3966 - :tags: bug, mysql - :versions: 1.2.0b1 - :tickets: 3966 - - Removed an ancient and unnecessary intercept of the UTC_TIMESTAMP - MySQL function, which was getting in the way of using it with a - parameter. - - .. change:: 3961 - :tags: bug, mysql - :versions: 1.2.0b1 - :tickets: 3961 - - Fixed bug in MySQL dialect regarding rendering of table options in - conjunction with PARTITION options when rendering CREATE TABLE. - The PARTITION related options need to follow the table options, - whereas previously this ordering was not enforced. - - -.. changelog:: - :version: 1.1.9 - :released: April 4, 2017 - - .. change:: 3956 - :tags: bug, ext - :tickets: 3956 - - Fixed regression released in 1.1.8 due to :ticket:`3950` where the - deeper search for information about column types in the case of a - "schema type" or a :class:`.TypeDecorator` would produce an attribute - error if the mapping also contained a :obj:`.column_property`. - - .. change:: 3952 - :tags: bug, sql - :versions: 1.2.0b1 - :tickets: 3952 - - Fixed regression released in 1.1.5 due to :ticket:`3859` where - adjustments to the "right-hand-side" evaluation of an expression - based on :class:`.Variant` to honor the underlying type's - "right-hand-side" rules caused the :class:`.Variant` type - to be inappropriately lost, in those cases when we *do* want the - left-hand side type to be transferred directly to the right hand side - so that bind-level rules can be applied to the expression's argument. - - .. change:: 3955 - :tags: bug, sql, postgresql - :versions: 1.2.0b1 - :tickets: 3955 - - Changed the mechanics of :class:`_engine.ResultProxy` to unconditionally - delay the "autoclose" step until the :class:`_engine.Connection` is done - with the object; in the case where PostgreSQL ON CONFLICT with - RETURNING returns no rows, autoclose was occurring in this previously - non-existent use case, causing the usual autocommit behavior that - occurs unconditionally upon INSERT/UPDATE/DELETE to fail. - -.. changelog:: - :version: 1.1.8 - :released: March 31, 2017 - - .. change:: 3950 - :tags: bug, ext - :versions: 1.2.0b1 - :tickets: 3950 - - Fixed bug in :mod:`sqlalchemy.ext.mutable` where the - :meth:`.Mutable.as_mutable` method would not track a type that had - been copied using :meth:`.TypeEngine.copy`. This became more of - a regression in 1.1 compared to 1.0 because the :class:`.TypeDecorator` - class is now a subclass of :class:`.SchemaEventTarget`, which among - other things indicates to the parent :class:`_schema.Column` that the type - should be copied when the :class:`_schema.Column` is. These copies are - common when using declarative with mixins or abstract classes. - - .. change:: - :tags: bug, ext - :versions: 1.2.0b1 - - Added support for bound parameters, e.g. those normally set up - via :meth:`_query.Query.params`, to the :meth:`.baked.Result.count` - method. Previously, support for parameters were omitted. Pull request - courtesy Pat Deegan. - - .. change:: - :tags: bug, postgresql - :versions: 1.2.0b1 - - Added support for parsing the PostgreSQL version string for - a development version like "PostgreSQL 10devel". Pull request - courtesy Sean McCully. - -.. changelog:: - :version: 1.1.7 - :released: March 27, 2017 - - .. change:: - :tags: feature, orm - :tickets: 3933 - :versions: 1.2.0b1 - - An :func:`.aliased()` construct can now be passed to the - :meth:`_query.Query.select_entity_from` method. Entities will be pulled - from the selectable represented by the :func:`.aliased` construct. - This allows special options for :func:`.aliased` such as - :paramref:`.aliased.adapt_on_names` to be used in conjunction with - :meth:`_query.Query.select_entity_from`. - - .. change:: - :tags: bug, engine - :tickets: 3946 - :versions: 1.2.0b1 - - Added an exception handler that will warn for the "cause" exception on - Py2K when the "autorollback" feature of :class:`_engine.Connection` itself - raises an exception. In Py3K, the two exceptions are naturally reported - by the interpreter as one occurring during the handling of the other. - This is continuing with the series of changes for rollback failure - handling that were last visited as part of :ticket:`2696` in 1.0.12. - - .. change:: - :tags: bug, orm - :tickets: 3947 - :versions: 1.2.0b1 - - Fixed a race condition which could occur under threaded environments - as a result of the caching added via :ticket:`3915`. An internal - collection of ``Column`` objects could be regenerated on an alias - object inappropriately, confusing a joined eager loader when it - attempts to render SQL and collect results and resulting in an - attribute error. The collection is now generated up front before - the alias object is cached and shared among threads. - - .. change:: - :tags: bug, sql, postgresql - :tickets: 2892 - - Added support for the :class:`.Variant` and the :class:`.SchemaType` - objects to be compatible with each other. That is, a variant - can be created against a type like :class:`.Enum`, and the instructions - to create constraints and/or database-specific type objects will - propagate correctly as per the variant's dialect mapping. - - .. change:: - :tags: bug, sql - :tickets: 3931 - - Fixed bug in compiler where the string identifier of a savepoint would - be cached in the identifier quoting dictionary; as these identifiers - are arbitrary, a small memory leak could occur if a single - :class:`_engine.Connection` had an unbounded number of savepoints used, - as well as if the savepoint clause constructs were used directly - with an unbounded umber of savepoint names. The memory leak does - **not** impact the vast majority of cases as normally the - :class:`_engine.Connection`, which renders savepoint names with a simple - counter starting at "1", is used on a per-transaction or - per-fixed-number-of-transactions basis before being discarded. - - .. change:: - :tags: bug, sql - :tickets: 3924 - - Fixed bug in new "schema translate" feature where the translated schema - name would be invoked in terms of an alias name when rendered along - with a column expression; occurred only when the source translate - name was "None". The "schema translate" feature now only takes - effect for :class:`.SchemaItem` and :class:`.SchemaType` subclasses, - that is, objects that correspond to a DDL-creatable structure in - a database. - -.. changelog:: - :version: 1.1.6 - :released: February 28, 2017 - - .. change:: - :tags: bug, mysql - - Added new MySQL 8.0 reserved words to the MySQL dialect for proper - quoting. Pull request courtesy Hanno Schlichting. - - .. change:: 3915 - :tags: bug, orm - :tickets: 3915 - - Addressed some long unattended performance concerns within the joined - eager loader query construction system that have accumulated since - earlier versions as a result of increased abstraction. The use of ad- - hoc :class:`.AliasedClass` objects per query, which produces lots of - column lookup overhead each time, has been replaced with a cached - approach that makes use of a small pool of :class:`.AliasedClass` - objects that are reused between invocations of joined eager loading. - Some mechanics involving eager join path construction have also been - optimized. Callcounts for an end-to-end query construction + single - row fetch test with a worst-case joined loader scenario have been - reduced by about 60% vs. 1.1.5 and 42% vs. that of 0.8.6. - - .. change:: 3804 - :tags: bug, postgresql - :tickets: 3804 - - Added regular expressions for the "IMPORT FOREIGN SCHEMA", - "REFRESH MATERIALIZED VIEW" PostgreSQL statements so that they - autocommit when invoked via a connection or engine without - an explicit transaction. Pull requests courtesy Frazer McLean - and Paweł Stiasny. - - .. change:: 3909 - :tags: bug, orm - :tickets: 3909 - - Fixed a major inefficiency in the "eager_defaults" feature whereby - an unnecessary SELECT would be emitted for column values where the - ORM had explicitly inserted NULL, corresponding to attributes that - were unset on the object but did not have any server default - specified, as well as expired attributes on update that nevertheless - had no server onupdate set up. As these columns are not part of the - RETURNING that eager_defaults tries to use, they should not be - post-SELECTed either. - - .. change:: 3908 - :tags: bug, orm - :tickets: 3908 - - Fixed two closely related bugs involving the mapper eager_defaults - flag in conjunction with single-table inheritance; one where the - eager defaults logic would inadvertently try to access a column - that's part of the mapper's "exclude_properties" list (used by - Declarative with single table inheritance) during the eager defaults - fetch, and the other where the full load of the row in order to - fetch the defaults would fail to use the correct inheriting mapper. - - - .. change:: 3905 - :tags: bug, sql - :tickets: 3905 - - Fixed bug whereby the :meth:`.DDLEvents.column_reflect` event would not - allow a non-textual expression to be passed as the value of the - "default" for the new column, such as a :class:`.FetchedValue` - object to indicate a generic triggered default or a - :func:`_expression.text` construct. Clarified the documentation - in this regard as well. - - .. change:: 3901 - :tags: bug, ext - :tickets: 3901 - - Fixed bug in new :mod:`sqlalchemy.ext.indexable` extension - where setting of a property that itself refers to another property - would fail. - - .. change:: 3900 - :tags: bug, postgresql - :tickets: 3900 - - Fixed bug in PostgreSQL :class:`.ExcludeConstraint` where the - "whereclause" and "using" parameters would not be copied during an - operation like :meth:`_schema.Table.tometadata`. - - .. change:: 3898 - :tags: bug, mssql - :tickets: 3898 - - Added a version check to the "get_isolation_level" feature, which is - invoked upon first connect, so that it skips for SQL Server version - 2000, as the necessary system view is not available prior to SQL Server - 2005. - - .. change:: 3897 - :tags: feature, ext - :tickets: 3896 - - Added :meth:`.baked.Result.scalar` and :meth:`.baked.Result.count` - to the "baked" query system. - - .. change:: 3895 - :tags: bug, orm, declarative - :tickets: 3895 - - Fixed bug where the "automatic exclude" feature of declarative that - ensures a column local to a single table inheritance subclass does - not appear as an attribute on other derivations of the base would - not take effect for multiple levels of subclassing from the base. - - .. change:: 3893 - :tags: bug, orm - :tickets: 3893 - - Fixed bug first introduced in 0.9.7 as a result of :ticket:`3106` - which would cause an incorrect query in some forms of multi-level - subqueryload against aliased entities, with an unnecessary extra - FROM entity in the innermost subquery. - -.. changelog:: - :version: 1.1.5 - :released: January 17, 2017 - - .. change:: mysql_index_prefix - :tags: feature, mysql - - Added a new parameter ``mysql_prefix`` supported by the :class:`.Index` - construct, allows specification of MySQL-specific prefixes such as - "FULLTEXT". Pull request courtesy Joseph Schorr. - - .. change:: 3854 - :tags: bug, orm - :tickets: 3854 - - Fixed bug in subquery loading where an object encountered as an - "existing" row, e.g. already loaded from a different path in the - same query, would not invoke subquery loaders for unloaded attributes - that specified this loading. This issue is in the same area - as that of :ticket:`3431`, :ticket:`3811` which involved - similar issues with joined loading. - - .. change:: 3888 - :tags: bug, postgresql - :tickets: 3888 - - Fixed bug in new "ON CONFLICT DO UPDATE" feature where the "set" - values for the UPDATE clause would not be subject to type-level - processing, as normally takes effect to handle both user-defined - type level conversions as well as dialect-required conversions, such - as those required for JSON datatypes. Additionally, clarified that - the keys in the ``set_`` dictionary should match the "key" of the - column, if distinct from the column name. A warning is emitted - for remaining column names that don't match column keys; for - compatibility reasons, these are emitted as they were previously. - - .. change:: 3872 - :tags: bug, examples - :tickets: 3872 - - Fixed two issues with the versioned_history example, one is that - the history table now gets autoincrement=False to avoid 1.1's new - errors regarding composite primary keys with autoincrement; the other - is that the sqlite_autoincrement flag is now used to ensure on SQLite, - unique identifiers are used for the lifespan of a table even if - some rows are deleted. Pull request courtesy Carlos García Montoro. - - .. change:: 3882 - :tags: bug, sql - :tickets: 3882 - - Fixed bug originally introduced in 0.9 via :ticket:`1068` where - order_by() would order by the label name based on name - alone, that is, even if the labeled expression were not at all the same - expression otherwise present, implicitly or explicitly, in the - selectable. The logic that orders by label now ensures that the - labeled expression is related to the one that resolves to that name - before ordering by the label name; additionally, the name has to - resolve to an actual label explicit in the expression elsewhere, not - just a column name. This logic is carefully kept separate from the - order by(textual name) feature that has a slightly different purpose. - - .. change:: try_finally_for_noautoflush - :tags: bug, orm - - The :attr:`.Session.no_autoflush` context manager now ensures that - the autoflush flag is reset within a "finally" block, so that if - an exception is raised within the block, the state still resets - appropriately. Pull request courtesy Emin Arakelian. - - .. change:: 3878 - :tags: bug, sql - :tickets: 3878 - - Fixed 1.1 regression where "import *" would not work for - sqlalchemy.sql.expression, due to mis-spelled ``any_`` and ``all_`` - functions. - - .. change:: 3880 - :tags: bg, sql - :tickets: 3880 - - Fixed bug where literal_binds compiler flag was not honored by the - :class:`_expression.Insert` construct for the "multiple values" feature; the - subsequent values are now rendered as literals. - - .. change:: 3877 - :tags: bug, oracle, postgresql - :tickets: 3877 - - Fixed bug where an INSERT from SELECT where the source table contains - an autoincrementing Sequence would fail to compile correctly. - - .. change:: 3876 - :tags: bug, mssql - :tickets: 3876 - - Fixed bug where SQL Server dialects would attempt to select the - last row identity for an INSERT from SELECT, failing in the case when - the SELECT has no rows. For such a statement, - the inline flag is set to True indicating no last primary key - should be fetched. - - .. change:: 3875 - :tags: bug, oracle - :tickets: 3875 - - Fixed bug where the "COMPRESSION" keyword was used in the ALL_TABLES - query on Oracle 9.2; even though Oracle docs state table compression - was introduced in 9i, the actual column is not present until - 10.1. - - .. change:: 3874 - :tags: bug, orm - :tickets: 3874 - - Fixed bug where the single-table inheritance query criteria would not - be inserted into the query in the case that the :class:`.Bundle` - construct were used as the selection criteria. - - .. change:: repr_for_url_reflect - :tags: bug, sql - - The engine URL embedded in the exception for "could not reflect" - in :meth:`_schema.MetaData.reflect` now conceals the password; also - the ``__repr__`` for :class:`.TLEngine` now acts like that of - :class:`_engine.Engine`, concealing the URL password. Pull request courtesy - Valery Yundin. - - .. change:: 3867 - :tags: bug, mysql - :tickets: 3867 - - The MySQL dialect now will not warn when a reflected column has a - "COMMENT" keyword on it, but note however the comment is not yet - reflected; this is on the roadmap for a future release. Pull request - courtesy Lele Long. - - .. change:: pg_timestamp_zero_prec - :tags: bug, postgresql - - The :class:`_postgresql.TIME` and :class:`_postgresql.TIMESTAMP` - datatypes now support a setting of zero for "precision"; previously - a zero would be ignored. Pull request courtesy Ionuț Ciocîrlan. - - .. change:: 3861 - :tags: bug, engine - :tickets: 3861 - - The "extend_existing" option of :class:`_schema.Table` reflection would - cause indexes and constraints to be doubled up in the case that the parameter - were used with :meth:`_schema.MetaData.reflect` (as the automap extension does) - due to tables being reflected both within the foreign key path as well - as directly. A new de-duplicating set is passed through within the - :meth:`_schema.MetaData.reflect` sequence to prevent double reflection in this - way. - - .. change:: 3859 - :tags: bug, sql - :tickets: 3859 - - Fixed issue in :class:`.Variant` where the "right hand coercion" logic, - inherited from :class:`.TypeDecorator`, would - coerce the right-hand side into the :class:`.Variant` itself, rather than - what the default type for the :class:`.Variant` would do. In the - case of :class:`.Variant`, we want the type to act mostly like the base - type so the default logic of :class:`.TypeDecorator` is now overridden - to fall back to the underlying wrapped type's logic. Is mostly relevant - for JSON at the moment. - - .. change:: 3856 - :tags: bug, orm - :tickets: 3856 - - Fixed bug related to :ticket:`3177`, where a UNION or other set operation - emitted by a :class:`_query.Query` would apply "single-inheritance" criteria - to the outside of the union (also referencing the wrong selectable), - even though this criteria is now expected to - be already present on the inside subqueries. The single-inheritance - criteria is now omitted once union() or another set operation is - called against :class:`_query.Query` in the same way as :meth:`_query.Query.from_self`. - - .. change:: 3548 - :tags: bug, firebird - :tickets: 3548 - - Ported the fix for Oracle quoted-lowercase names to Firebird, so that - a table name that is quoted as lower case can be reflected properly - including when the table name comes from the get_table_names() - inspection function. - -.. changelog:: - :version: 1.1.4 - :released: November 15, 2016 - - .. change:: 3842 - :tags: bug, sql - :tickets: 3842 - - Fixed bug where newly added warning for primary key on insert w/o - autoincrement setting (see :ticket:`3216`) would fail to emit - correctly when invoked upon a lower-case :func:`.table` construct. - - .. change:: 3852 - :tags: bug, orm - :tickets: 3852 - - Fixed regression in collections due to :ticket:`3457` whereby - deserialize during pickle or deepcopy would fail to establish all - attributes of an ORM collection, causing further mutation operations to - fail. - - .. change:: default_schema - :tags: bug, engine - - Removed long-broken "default_schema_name()" method from - :class:`_engine.Connection`. This method was left over from a very old - version and was non-working (e.g. would raise). Pull request - courtesy Benjamin Dopplinger. - - .. change:: pragma - :tags: bug, sqlite - - Added quotes to the PRAGMA directives in the pysqlcipher dialect - to support additional cipher arguments appropriately. Pull request - courtesy Kevin Jurczyk. - - .. change:: 3846 - :tags: bug, postgresql - :tickets: 3846, 3807 - - Fixed regression caused by the fix in :ticket:`3807` (version 1.1.0) - where we ensured that the tablename was qualified in the WHERE clause - of the DO UPDATE portion of PostgreSQL's ON CONFLICT, however you - *cannot* put the table name in the WHERE clause in the actual ON - CONFLICT itself. This was an incorrect assumption, so that portion - of the change in :ticket:`3807` is rolled back. - - .. change:: 3845 - :tags: bug, orm - :tickets: 3845 - - Fixed long-standing bug where the "noload" relationship loading - strategy would cause backrefs and/or back_populates options to be - ignored. - - .. change:: sscursor_mysql - :tags: feature, mysql - - Added support for server side cursors to the mysqlclient and - pymysql dialects. This feature is available via the - :paramref:`.Connection.execution_options.stream_results` flag as well - as the ``server_side_cursors=True`` dialect argument in the - same way that it has been for psycopg2 on PostgreSQL. Pull request - courtesy Roman Podoliaka. - - .. change:: - :tags: bug, mysql - :tickets: 3841 - - MySQL's native ENUM type supports any non-valid value being sent, and - in response will return a blank string. A hardcoded rule to check for - "is returning the blank string" has been added to the MySQL - implementation for ENUM so that this blank string is returned to the - application rather than being rejected as a non-valid value. Note that - if your MySQL enum is linking values to objects, you still get the - blank string back. - - .. change:: - :tags: bug, sqlite, py3k - - Added an optional import for the pysqlcipher3 DBAPI when using the - pysqlcipher dialect. This package will attempt to be imported - if the Python-2 only pysqlcipher DBAPI is non-present. - Pull request courtesy Kevin Jurczyk. - -.. changelog:: - :version: 1.1.3 - :released: October 27, 2016 - - .. change:: - :tags: bug, orm - :tickets: 3839 - - Fixed regression caused by :ticket:`2677` whereby calling - :meth:`.Session.delete` on an object that was already flushed as - deleted in that session would fail to set up the object in the - identity map (or reject the object), causing flush errors as the - object were in a state not accommodated by the unit of work. - The pre-1.1 behavior in this case has been restored, which is that - the object is put back into the identity map so that the DELETE - statement will be attempted again, which emits a warning that the number - of expected rows was not matched (unless the row were restored outside - of the session). - - .. change:: - :tags: bug, postgresql - :tickets: 3835 - - PostgreSQL table reflection will ensure that the - :paramref:`_schema.Column.autoincrement` flag is set to False when reflecting - a primary key column that is not of an :class:`.Integer` datatype, - even if the default is related to an integer-generating sequence. - This can happen if a column is created as SERIAL and the datatype - is changed. The autoincrement flag can only be True if the datatype - is of integer affinity in the 1.1 series. - - .. change:: - :tags: bug, orm - :tickets: 3836 - - Fixed regression where some :class:`_query.Query` methods like - :meth:`_query.Query.update` and others would fail if the :class:`_query.Query` - were against a series of mapped columns, rather than the mapped - entity as a whole. - - .. change:: - :tags: bug, sql - :tickets: 3833 - - Fixed bug involving new value translation and validation feature - in :class:`.Enum` whereby using the enum object in a string - concatenation would maintain the :class:`.Enum` type as the type - of the expression overall, producing missing lookups. A string - concatenation against an :class:`.Enum`-typed column now uses - :class:`.String` as the datatype of the expression itself. - - .. change:: - :tags: bug, sql - :tickets: 3832 - - Fixed regression which occurred as a side effect of :ticket:`2919`, - which in the less typical case of a user-defined - :class:`.TypeDecorator` that was also itself an instance of - :class:`.SchemaType` (rather than the implementation being such) - would cause the column attachment events to be skipped for the - type itself. - - -.. changelog:: - :version: 1.1.2 - :released: October 17, 2016 - - .. change:: - :tags: bug, sql - :tickets: 3823 - - Fixed a regression caused by a newly added function that performs the - "wrap callable" function of sql :class:`.DefaultGenerator` objects, - an attribute error raised for ``__module__`` when the default callable - was a ``functools.partial`` or other object that doesn't have a - ``__module__`` attribute. - - .. change:: - :tags: bug, orm - :tickets: 3824 - - Fixed bug involving the rule to disable a joined collection eager - loader on the other side of a many-to-one lazy loader, first added - in :ticket:`1495`, where the rule would fail if the parent object - had some other lazyloader-bound query options associated with it. - - .. change:: - :tags: bug, orm - :tickets: 3822 - - Fixed self-referential entity, deferred column loading issue in a - similar style as that of :ticket:`3431`, :ticket:`3811` where an entity - is present in multiple positions within the row due to self-referential - eager loading; when the deferred loader only applies to one of the - paths, the "present" column loader will now override the deferred non- - load for that entity regardless of row ordering. - - .. change:: - :tags: bug, sql, postgresql - :tickets: 3827 - - Fixed regression in :class:`.Enum` type where event handlers were not - transferred in the case of the type object being copied, due to a - conflicting copy() method added as part of :ticket:`3250`. This copy - occurs normally in situations when the column is copied, such as - in tometadata() or when using declarative mixins with columns. The - event handler not being present would impact the constraint being - created for a non-native enumerated type, but more critically the - ENUM object on the PostgreSQL backend. - - - .. change:: - :tags: bug, postgresql, sql - :tickets: 3828 - - Changed the naming convention used when generating bound parameters - for a multi-VALUES insert statement, so that the numbered parameter - names don't conflict with the anonymized parameters of a WHERE clause, - as is now common in a PostgreSQL ON CONFLICT construct. - -.. changelog:: - :version: 1.1.1 - :released: October 7, 2016 - - .. change:: - :tags: bug, mssql - :tickets: 3820 - - The "SELECT SERVERPROPERTY" - query added in :ticket:`3810` and :ticket:`3814` is failing on unknown - combinations of Pyodbc and SQL Server. While failure of this function - was anticipated, the exception catch was not broad enough so it now - catches all forms of pyodbc.Error. - - .. change:: - :tags: bug, core - :tickets: 3216 - - Changed the CompileError raised when various primary key missing - situations are detected to a warning. The statement is again - passed to the database where it will fail and the DBAPI error (usually - IntegrityError) raises as usual. - - .. seealso:: - - :ref:`change_3216` - -.. changelog:: - :version: 1.1.0 - :released: October 5, 2016 - - .. change:: - :tags: bug, sql - :tickets: 3805 - - Execution options can now be propagated from within a - statement at compile time to the outermost statement, so that - if an embedded element wants to set "autocommit" to be True for example, - it can propagate this to the enclosing statement. Currently, this - feature is enabled for a DML-oriented CTE embedded inside of a SELECT - statement, e.g. INSERT/UPDATE/DELETE inside of SELECT. - - .. change:: - :tags: bug, orm - :tickets: 3802 - - ORM attributes can now be assigned any object that is has a - ``__clause_element__()`` attribute, which will result in inline - SQL the way any :class:`_expression.ClauseElement` class does. This covers other - mapped attributes not otherwise transformed by further expression - constructs. - - .. change:: - :tags: feature, orm - :tickets: 3812 - - Enhanced the new "raise" lazy loader strategy to also include a - "raise_on_sql" variant, available both via :paramref:`.orm.relationship.lazy` - as well as :func:`_orm.raiseload`. This variant only raises if the - lazy load would actually emit SQL, vs. raising if the lazy loader - mechanism is invoked at all. - - .. change:: - :tags: bug, postgresql - :tickets: 3813 - - An adjustment to ON CONFLICT such that the "inserted_primary_key" - logic is able to accommodate the case where there's no INSERT or - UPDATE and there's no net change. The value comes out as None - in this case, rather than failing on an exception. - - .. change:: - :tags: bug, orm - :tickets: 3811 - - Made an adjustment to the bug fix first introduced in [ticket:3431] - that involves an object appearing in multiple contexts in a single - result set, such that an eager loader that would set the related - object value to be None will still fire off, thus satisfying the - load of that attribute. Previously, the adjustment only honored - a non-None value arriving for an eagerly loaded attribute in a - secondary row. - - .. change:: - :tags: bug, orm - :tickets: 3808 - - Fixed bug in new :meth:`.SessionEvents.persistent_to_deleted` event - where the target object could be garbage collected before the event - is fired off. - - .. change:: - :tags: bug, sql - :tickets: 3809 - - A string sent as a column default via the - :paramref:`_schema.Column.server_default` parameter is now escaped for quotes. - - .. seealso:: - - :ref:`change_3809` - - .. change:: - :tags: bug, postgresql - :tickets: 3807 - - Fixed issue in new PG "on conflict" construct where columns including - those of the "excluded" namespace would not be table-qualified - in the WHERE clauses in the statement. - - .. change:: - :tags: bug, sql, postgresql - :tickets: 3806 - - Added compiler-level flags used by PostgreSQL to place additional - parenthesis than would normally be generated by precedence rules - around operations involving JSON, HSTORE indexing operators as well as - within their operands since it has been observed that PostgreSQL's - precedence rules for at least the HSTORE indexing operator is not - consistent between 9.4 and 9.5. - - .. change:: - :tags: bug, sql, mysql - :tickets: 3803 - - The ``BaseException`` exception class is now intercepted by the - exception-handling routines of :class:`_engine.Connection`, and includes - handling by the :meth:`_events.ConnectionEvents.handle_error` - event. The :class:`_engine.Connection` is now **invalidated** by default in - the case of a system level exception that is not a subclass of - ``Exception``, including ``KeyboardInterrupt`` and the greenlet - ``GreenletExit`` class, to prevent further operations from occurring - upon a database connection that is in an unknown and possibly - corrupted state. The MySQL drivers are most targeted by this change - however the change is across all DBAPIs. - - .. seealso:: - - :ref:`change_3803` - - .. change:: - :tags: bug, sql - :tickets: 3799 - - The "eq" and "ne" operators are no longer part of the list of - "associative" operators, while they remain considered to be - "commutative". This allows an expression like ``(x == y) == z`` - to be maintained at the SQL level with parenthesis. Pull request - courtesy John Passaro. - - .. change:: - :tags: bug, orm - :tickets: 3767 - - The primaryjoin of a :func:`_orm.relationship` construct can now include - a :func:`.bindparam` object that includes a callable function to - generate values. Previously, the lazy loader strategy would - be incompatible with this use, and additionally would fail to correctly - detect if the "use_get" criteria should be used if the primary key - were involved with the bound parameter. - - .. change:: - :tags: bug, orm - :tickets: 3801 - - An UPDATE emitted from the ORM flush process can now accommodate a - SQL expression element for a column within the primary key of an - object, if the target database supports RETURNING in order to provide - the new value, or if the PK value is set "to itself" for the purposes - of bumping some other trigger / onupdate on the column. - - .. change:: - :tags: bug, orm - :tickets: 3788 - - Fixed bug where the "simple many-to-one" condition that allows lazy - loading to use get() from identity map would fail to be invoked if the - primaryjoin of the relationship had multiple clauses separated by AND - which were not in the same order as that of the primary key columns - being compared in each clause. This ordering - difference occurs for a composite foreign key where the table-bound - columns on the referencing side were not in the same order in the .c - collection as the primary key columns on the referenced side....which - in turn occurs a lot if one is using declarative mixins and/or - declared_attr to set up columns. - - .. change:: - :tags: bug, sql - :tickets: 3789 - - Stringify of expression with unnamed :class:`_schema.Column` objects, as - occurs in lots of situations including ORM error reporting, - will now render the name in string context as "" - rather than raising a compile error. - - .. change:: - :tags: bug, sql - :tickets: 3786 - - Raise a more descriptive exception / message when ClauseElement - or non-SQLAlchemy objects that are not "executable" are erroneously - passed to ``.execute()``; a new exception ObjectNotExecutableError - is raised consistently in all cases. - - .. change:: - :tags: bug, orm - :tickets: 3776 - - An exception is raised when two ``@validates`` decorators on a mapping - make use of the same name. Only one validator of a certain name - at a time is supported, there's no mechanism to chain these together, - as the order of the validators at the level of function decorator - can't be made deterministic. - - .. seealso:: - - :ref:`change_3776` - - .. change:: - :tags: bug, orm - - Mapper errors raised during :func:`.configure_mappers` now explicitly - include the name of the originating mapper in the exception message - to help in those situations where the wrapped exception does not - itself include the source mapper. Pull request courtesy - John Perkins. - - .. change:: - :tags: bug, mysql - :tickets: 3766 - - Fixed bug where the "literal_binds" flag would not be propagated - to a CAST expression under MySQL. - - .. change:: - :tags: bug, sql, postgresql, mysql - :tickets: 3765 - - Fixed regression in JSON datatypes where the "literal processor" for - a JSON index value would not be invoked. The native String and Integer - datatypes are now called upon from within the JSONIndexType - and JSONPathType. This is applied to the generic, PostgreSQL, and - MySQL JSON types and also has a dependency on :ticket:`3766`. - - .. change:: - :tags: change, orm - - Passing False to :meth:`_query.Query.order_by` in order to cancel - all order by's is deprecated; there is no longer any difference - between calling this method with False or with None. - - .. change:: - :tags: feature, orm - - The :meth:`_query.Query.group_by` method now resets the group by collection - if an argument of ``None`` is passed, in the same way that - :meth:`_query.Query.order_by` has worked for a long time. Pull request - courtesy Iuri Diniz. - - .. change:: - :tags: bug, sql - :tickets: 3763 - - Fixed bug where :class:`.Index` would fail to extract columns from - compound SQL expressions if those SQL expressions were wrapped inside - of an ORM-style ``__clause_element__()`` construct. This bug - exists in 1.0.x as well, however in 1.1 is more noticeable as - hybrid_property @expression now returns a wrapped element. - - .. change:: - :tags: change, orm, declarative - - Constructing a declarative base class that inherits from another class - will also inherit its docstring. This means - :func:`~.ext.declarative.as_declarative` acts more like a normal class - decorator. - -.. changelog:: - :version: 1.1.0b3 - :released: July 26, 2016 - - .. change:: - :tags: change, orm - :tickets: 3749 - - Removed a warning that dates back to 0.4 which emits when a same-named - relationship is placed on two mappers that inherits via joined or - single table inheritance. The warning does not apply to the - current unit of work implementation. - - .. seealso:: - - :ref:`change_3749` - - - .. change:: - :tags: bug, sql - :tickets: 3745 - - Fixed bug in new CTE feature for update/insert/delete stated - as a CTE inside of an enclosing statement (typically SELECT) whereby - oninsert and onupdate values weren't called upon for the embedded - statement. - - .. change:: - :tags: bug, sql - :tickets: 3744 - - Fixed bug in new CTE feature for update/insert/delete whereby - an anonymous (e.g. no name passed) :class:`_expression.CTE` construct around - the statement would fail. - - .. change:: - :tags: bug, ext - - sqlalchemy.ext.indexable will intercept IndexError as well - as KeyError when raising as AttributeError. - - .. change:: - :tags: feature, ext - - Added a "default" parameter to the new sqlalchemy.ext.indexable - extension. - -.. changelog:: - :version: 1.1.0b2 - :released: July 1, 2016 - - .. change:: - :tags: bug, ext, postgresql - :tickets: 3732 - - Made a slight behavioral change in the ``sqlalchemy.ext.compiler`` - extension, whereby the existing compilation schemes for an established - construct would be removed if that construct itself didn't already - have its own dedicated ``__visit_name__``. This was a - rare occurrence in 1.0, however in 1.1 :class:`_postgresql.ARRAY` - subclasses :class:`_types.ARRAY` and has this behavior. - As a result, setting up a compilation handler for another dialect - such as SQLite would render the main :class:`_postgresql.ARRAY` - object no longer compilable. - - .. change:: - :tags: bug, sql - :tickets: 3730 - - The processing performed by the :class:`.Boolean` datatype for backends - that only feature integer types has been made consistent between the - pure Python and C-extension versions, in that the C-extension version - will accept any integer value from the database as a boolean, not just - zero and one; additionally, non-boolean integer values being sent to - the database are coerced to exactly zero or one, instead of being - passed as the original integer value. - - .. seealso:: - - :ref:`change_3730` - - .. change:: - :tags: bug, sql - :tickets: 3725 - - Rolled back the validation rules a bit in :class:`.Enum` to allow - unknown string values to pass through, unless the flag - ``validate_string=True`` is passed to the Enum; any other kind of object is - still of course rejected. While the immediate use - is to allow comparisons to enums with LIKE, the fact that this - use exists indicates there may be more unknown-string-comparison use - cases than we expected, which hints that perhaps there are some - unknown string-INSERT cases too. - - .. change:: - :tags: bug, mysql - :tickets: 3726 - - Dialed back the "order the primary key columns per auto-increment" - described in :ref:`change_mysql_3216` a bit, so that if the - :class:`.PrimaryKeyConstraint` is explicitly defined, the order - of columns is maintained exactly, allowing control of this behavior - when necessary. - -.. changelog:: - :version: 1.1.0b1 - :released: June 16, 2016 - - .. change:: - :tags: feature, sql - :tickets: 3718 - - Added TABLESAMPLE support via the new :meth:`_expression.FromClause.tablesample` - method and standalone function. Pull request courtesy Ilja Everilä. - - .. seealso:: - - :ref:`change_3718` - - .. change:: - :tags: feature, orm, ext - - A new ORM extension :ref:`indexable_toplevel` is added, which allows - construction of Python attributes which refer to specific elements - of "indexed" structures such as arrays and JSON fields. Pull request - courtesy Jeong YunWon. - - .. seealso:: - - :ref:`feature_indexable` - - .. change:: - :tags: bug, sql - :tickets: 3724 - - :meth:`_expression.FromClause.count` is deprecated. This function makes use of - an arbitrary column in the table and is not reliable; for Core use, - ``func.count()`` should be preferred. - - .. change:: - :tags: feature, postgresql - :tickets: 3529 - - Added support for PostgreSQL's INSERT..ON CONFLICT using a new - PostgreSQL-specific :class:`.postgresql.dml.Insert` object. - Pull request and extensive efforts here by Robin Thomas. - - .. seealso:: - - :ref:`change_3529` - - .. change:: - :tags: feature, postgresql - - The DDL for DROP INDEX will emit "CONCURRENTLY" if the - ``postgresql_concurrently`` flag is set upon the - :class:`.Index` and if the database in use is detected as - PostgreSQL version 9.2 or greater. For CREATE INDEX, database - version detection is also added which will omit the clause if - PG version is less than 8.2. Pull request courtesy Iuri de Silvio. - - .. change:: - :tags: bug, orm - :tickets: 3708 - - Fixed an issue where a many-to-one change of an object from one - parent to another could work inconsistently when combined with - an un-flushed modification of the foreign key attribute. The attribute - move now considers the database-committed value of the foreign key - in order to locate the "previous" parent of the object being - moved. This allows events to fire off correctly including - backref events. Previously, these events would not always fire. - Applications which may have relied on the previously broken - behavior may be affected. - - .. seealso:: - - :ref:`change_3708` - - .. change:: - :tags: feature, sql - :tickets: 3049 - - Added support for ranges in window functions, using the - :paramref:`.expression.over.range_` and - :paramref:`.expression.over.rows` parameters. - - .. seealso:: - - :ref:`change_3049` - - .. change:: - :tags: feature, orm - - Added new flag :paramref:`.Session.bulk_insert_mappings.render_nulls` - which allows an ORM bulk INSERT to occur with NULL values rendered; - this bypasses server side defaults, however allows all statements - to be formed with the same set of columns, allowing them to be - batched. Pull request courtesy Tobias Sauerwein. - - .. change:: - :tags: feature, postgresql - :tickets: 3588 - - Added new parameter :paramref:`.PGInspector.get_view_names.include`, - allowing specification for what kinds of views should be returned. - Currently "plain" and "materialized" views are included. Pull - request courtesy Sebastian Bank. - - .. change:: - :tags: feature, mssql - - The ``mssql_clustered`` flag available on :class:`.UniqueConstraint`, - :class:`.PrimaryKeyConstraint`, :class:`.Index` now defaults to - ``None``, and can be set to False which will render the NONCLUSTERED - keyword in particular for a primary key, allowing a different index to - be used as "clustered". Pull request courtesy Saulius Žemaitaitis. - - .. change:: - :tags: feature, orm - :tickets: 1311 - - Added new event :meth:`.AttributeEvents.init_scalar`, as well - as a new example suite illustrating its use. This event can be used - to provide a Core-generated default value to a Python-side attribute - before the object is persisted. - - .. seealso:: - - :ref:`change_1311` - - .. change:: - :tags: feature, postgresql - :tickets: 3720 - - Added ``postgresql_tablespace`` as an argument to :class:`.Index` - to allow specification of TABLESPACE for an index in PostgreSQL. - Complements the same-named parameter on :class:`_schema.Table`. Pull - request courtesy Benjamin Bertrand. - - .. change:: - :tags: orm, feature - - Added :paramref:`.AutomapBase.prepare.schema` to the - :meth:`.AutomapBase.prepare` method, to indicate which schema - tables should be reflected from if not the default schema. - Pull request courtesy Josh Marlow. - - .. change:: - :tags: feature, sqlite - - The SQLite dialect now reflects ON UPDATE and ON DELETE phrases - within foreign key constraints. Pull request courtesy - Michal Petrucha. - - .. change:: - :tags: bug, mssql - - Adjustments to the mxODBC dialect to make use of the ``BinaryNull`` - symbol when appropriate in conjunction with the ``VARBINARY`` - data type. Pull request courtesy Sheila Allen. - - .. change:: - :tags: feature, sql - - Implemented reflection of CHECK constraints for SQLite and PostgreSQL. - This is available via the new inspector method - :meth:`_reflection.Inspector.get_check_constraints` as well as when reflecting - :class:`_schema.Table` objects in the form of :class:`.CheckConstraint` - objects present in the constraints collection. Pull request courtesy - Alex Grönholm. - - .. change:: - :tags: feature, postgresql - - Added new parameter - :paramref:`.GenerativeSelect.with_for_update.key_share`, which - will render the ``FOR NO KEY UPDATE`` version of ``FOR UPDATE`` - and ``FOR KEY SHARE`` instead of ``FOR SHARE`` - on the PostgreSQL backend. Pull request courtesy Sergey Skopin. - - .. change:: - :tags: feature, postgresql, oracle - - Added new parameter - :paramref:`.GenerativeSelect.with_for_update.skip_locked`, which - will render the ``SKIP LOCKED`` phrase for a ``FOR UPDATE`` or - ``FOR SHARE`` lock on the PostgreSQL and Oracle backends. Pull - request courtesy Jack Zhou. - - .. change:: - :tags: change, orm - :tickets: 3394 - - The :paramref:`_orm.Mapper.order_by` parameter is deprecated. - This is an old parameter no longer relevant to how SQLAlchemy - works, once the Query object was introduced. By deprecating it - we establish that we aren't supporting non-working use cases - and that we encourage applications to move off of the use of this - parameter. - - .. seealso:: - - :ref:`change_3394` - - .. change:: - :tags: feature, postgresql - - Added a new dialect for the PyGreSQL PostgreSQL dialect. Thanks - to Christoph Zwerschke and Kaolin Imago Fire for their efforts. - - .. change:: - :tags: bug, ext - :tickets: 3653 - - The docstring specified on a hybrid property or method is now honored - at the class level, allowing it to work with tools like Sphinx - autodoc. The mechanics here necessarily involve some wrapping of - expressions to occur for hybrid properties, which may cause them - to appear differently using introspection. - - .. seealso:: - - :ref:`change_3653` - - .. change:: - :tags: feature, sql - - New :meth:`.ColumnOperators.is_distinct_from` and - :meth:`.ColumnOperators.isnot_distinct_from` operators; pull request - courtesy Sebastian Bank. - - .. seealso:: - - :ref:`change_is_distinct_from` - - .. change:: - :tags: bug, orm - :tickets: 3488 - - Fixed bug where deferred columns would inadvertently be set up - for database load on the next object-wide unexpire, when the object - were merged into the session with ``session.merge(obj, load=False)``. - - .. change:: - :tags: feature, sql - - Added a hook in :meth:`.DDLCompiler.visit_create_table` called - :meth:`.DDLCompiler.create_table_suffix`, allowing custom dialects - to add keywords after the "CREATE TABLE" clause. Pull request - courtesy Mark Sandan. - - .. change:: - :tags: feature, sql - - Negative integer indexes are now accommodated by rows - returned from a :class:`_engine.ResultProxy`. Pull request courtesy - Emanuele Gaifas. - - .. seealso:: - - :ref:`change_gh_231` - - .. change:: - :tags: feature, sqlite - :tickets: 3629 - - The SQLite dialect now reflects the names of primary key constraints. - Pull request courtesy Diana Clarke. - - .. seealso:: - - :ref:`change_3629` - - .. change:: - :tags: feature, sql - :tickets: 2857 - - Added :meth:`_expression.Select.lateral` and related constructs to allow - for the SQL standard LATERAL keyword, currently only supported - by PostgreSQL. - - .. seealso:: - - :ref:`change_2857` - - .. change:: - :tags: feature, sql - :tickets: 1957 - - Added support for rendering "FULL OUTER JOIN" to both Core and ORM. - Pull request courtesy Stefan Urbanek. - - .. seealso:: - - :ref:`change_1957` - - .. change:: - :tags: feature, engine - - Added connection pool events :meth:`ConnectionEvents.close`, - :meth:`_events.ConnectionEvents.detach`, - :meth:`_events.ConnectionEvents.close_detached`. - - .. change:: - :tags: bug, orm, mysql - :tickets: 3680 - - Further continuing on the common MySQL exception case of - a savepoint being cancelled first covered in :ticket:`2696`, - the failure mode in which the :class:`.Session` is placed when a - SAVEPOINT vanishes before rollback has been improved to allow the - :class:`.Session` to still function outside of that savepoint. - It is assumed that the savepoint operation failed and was cancelled. - - .. seealso:: - - :ref:`change_3680` - - .. change:: - :tags: feature, mssql - :tickets: 3534 - - Added basic isolation level support to the SQL Server dialects - via :paramref:`_sa.create_engine.isolation_level` and - :paramref:`.Connection.execution_options.isolation_level` - parameters. - - .. seealso:: - - :ref:`change_3534` - - .. change:: - :tags: feature, mysql - :tickets: 3332 - - Added support for "autocommit" on MySQL drivers, via the - AUTOCOMMIT isolation level setting. Pull request courtesy - Roman Podoliaka. - - .. seealso:: - - :ref:`change_3332` - - .. change:: - :tags: bug, orm - :tickets: 3677 - - Fixed bug where a newly inserted instance that is rolled back - would still potentially cause persistence conflicts on the next - transaction, because the instance would not be checked that it - was expired. This fix will resolve a large class of cases that - erroneously cause the "New instance with identity X conflicts with - persistent instance Y" error. - - .. seealso:: - - :ref:`change_3677` - - .. change:: - :tags: bug, orm - :tickets: 3662 - - An improvement to the workings of :meth:`_query.Query.correlate` such - that when a "polymorphic" entity is used which represents a straight - join of several tables, the statement will ensure that all the - tables within the join are part of what's correlating. - - .. seealso:: - - :ref:`change_3662` - - .. change:: - :tags: bug, orm - :tickets: 3431 - - Fixed bug which would cause an eagerly loaded many-to-one attribute - to not be loaded, if the joined eager load were from a row where the - same entity were present multiple times, some calling for the attribute - to be eagerly loaded and others not. The logic here is revised to - take in the attribute even though a different loader path has - handled the parent entity already. - - .. seealso:: - - :ref:`change_3431` - - .. change:: - :tags: feature, engine - :tickets: 2837 - - All string formatting of bound parameter sets and result rows for - logging, exception, and ``repr()`` purposes now truncate very large - scalar values within each collection, including an - "N characters truncated" - notation, similar to how the display for large multiple-parameter sets - are themselves truncated. - - - .. seealso:: - - :ref:`change_2837` - - .. change:: - :tags: feature, ext - :tickets: 3297 - - Added :class:`.MutableSet` and :class:`.MutableList` helper classes - to the :ref:`mutable_toplevel` extension. Pull request courtesy - Jeong YunWon. - - .. change:: - :tags: feature, sql - :tickets: 2551 - - CTE functionality has been expanded to support all DML, allowing - INSERT, UPDATE, and DELETE statements to both specify their own - WITH clause, as well as for these statements themselves to be - CTE expressions when they include a RETURNING clause. - - .. seealso:: - - :ref:`change_2551` - - .. change:: - :tags: bug, orm - :tickets: 3641 - - A refinement to the logic which adds columns to the resulting SQL when - :meth:`_query.Query.distinct` is combined with :meth:`_query.Query.order_by` such - that columns which are already present will not be added - a second time, even if they are labeled with a different name. - Regardless of this change, the extra columns added to the SQL have - never been returned in the final result, so this change only impacts - the string form of the statement as well as its behavior when used in - a Core execution context. Additionally, columns are no longer added - when the DISTINCT ON format is used, provided the query is not - wrapped inside a subquery due to joined eager loading. - - .. seealso:: - - :ref:`change_3641` - - .. change:: - :tags: feature, sql - :tickets: 3292, 3095 - - Added support for PEP-435-style enumerated classes, namely - Python 3's ``enum.Enum`` class but also including compatible - enumeration libraries, to the :class:`_types.Enum` datatype. - The :class:`_types.Enum` datatype now also performs in-Python validation - of incoming values, and adds an option to forego creating the - CHECK constraint :paramref:`.Enum.create_constraint`. - Pull request courtesy Alex Grönholm. - - .. seealso:: - - :ref:`change_3292` - - :ref:`change_3095` - - .. change:: - :tags: change, postgresql - - The ``sqlalchemy.dialects.postgres`` module, long deprecated, is - removed; this has emitted a warning for many years and projects - should be calling upon ``sqlalchemy.dialects.postgresql``. - Engine URLs of the form ``postgres://`` will still continue to function, - however. - - .. change:: - :tags: bug, sqlite - :tickets: 3634 - - The workaround for right-nested joins on SQLite, where they are rewritten - as subqueries in order to work around SQLite's lack of support for this - syntax, is lifted when SQLite version 3.7.16 or greater is detected. - - .. seealso:: - - :ref:`change_3634` - - .. change:: - :tags: bug, sqlite - :tickets: 3633 - - The workaround for SQLite's unexpected delivery of column names as - ``tablename.columnname`` for some kinds of queries is now disabled - when SQLite version 3.10.0 or greater is detected. - - .. seealso:: - - :ref:`change_3633` - - .. change:: - :tags: feature, orm - :tickets: 2349 - - Added new parameter :paramref:`.orm.mapper.passive_deletes` to - available mapper options. This allows a DELETE to proceed - for a joined-table inheritance mapping against the base table only, - while allowing for ON DELETE CASCADE to handle deleting the row - from the subclass tables. - - .. seealso:: - - :ref:`change_2349` - - - .. change:: - :tags: bug, sybase - :tickets: 2278 - - The unsupported Sybase dialect now raises ``NotImplementedError`` - when attempting to compile a query that includes "offset"; Sybase - has no straightforward "offset" feature. - - .. change:: - :tags: feature, orm - :tickets: 3631 - - Calling str() on a core SQL construct has been made more "friendly", - when the construct contains non-standard SQL elements such as - RETURNING, array index operations, or dialect-specific or custom - datatypes. A string is now returned in these cases rendering an - approximation of the construct (typically the PostgreSQL-style - version of it) rather than raising an error. - - .. seealso:: - - :ref:`change_3631` - - .. change:: - :tags: bug, orm - :tickets: 3630 - - Fixed issue where two same-named relationships that refer to - a base class and a concrete-inherited subclass would raise an error - if those relationships were set up using "backref", while setting up the - identical configuration using relationship() instead with the conflicting - names would succeed, as is allowed in the case of a concrete mapping. - - .. seealso:: - - :ref:`change_3630` - - .. change:: - :tags: feature, orm - :tickets: 3081 - - The ``str()`` call for :class:`_query.Query` will now take into account - the :class:`_engine.Engine` to which the :class:`.Session` is bound, when - generating the string form of the SQL, so that the actual SQL - that would be emitted to the database is shown, if possible. Previously, - only the engine associated with the :class:`_schema.MetaData` to which the - mappings are associated would be used, if present. If - no bind can be located either on the :class:`.Session` or on - the :class:`_schema.MetaData` to which the mappings are associated, then - the "default" dialect is used to render the SQL, as was the case - previously. - - .. seealso:: - - :ref:`change_3081` - - .. change:: - :tags: feature, sql - :tickets: 3501 - - A deep improvement to the recently added :meth:`_expression.TextClause.columns` - method, and its interaction with result-row processing, now allows - the columns passed to the method to be positionally matched with the - result columns in the statement, rather than matching on name alone. - The advantage to this includes that when linking a textual SQL statement - to an ORM or Core table model, no system of labeling or de-duping of - common column names needs to occur, which also means there's no need - to worry about how label names match to ORM columns and so-forth. In - addition, the :class:`_engine.ResultProxy` has been further enhanced to - map column and string keys to a row with greater precision in some - cases. - - .. seealso:: - - :ref:`change_3501` - feature overview - - :ref:`behavior_change_3501` - backwards compatibility remarks - - .. change:: - :tags: feature, engine - :tickets: 2685 - - Multi-tenancy schema translation for :class:`_schema.Table` objects is added. - This supports the use case of an application that uses the same set of - :class:`_schema.Table` objects in many schemas, such as schema-per-user. - A new execution option - :paramref:`.Connection.execution_options.schema_translate_map` is - added. - - .. seealso:: - - :ref:`change_2685` - - .. change:: - :tags: feature, engine - :tickets: 3536 - - Added a new entrypoint system to the engine to allow "plugins" to - be stated in the query string for a URL. Custom plugins can - be written which will be given the chance up front to alter and/or - consume the engine's URL and keyword arguments, and then at engine - create time will be given the engine itself to allow additional - modifications or event registration. Plugins are written as a - subclass of :class:`.CreateEnginePlugin`; see that class for - details. - - .. change:: - :tags: feature, mysql - :tickets: 3547 - - Added :class:`.mysql.JSON` for MySQL 5.7. The JSON type provides - persistence of JSON values in MySQL as well as basic operator support - of "getitem" and "getpath", making use of the ``JSON_EXTRACT`` - function in order to refer to individual paths in a JSON structure. - - .. seealso:: - - :ref:`change_3547` - - .. change:: - :tags: feature, sql - :tickets: 3619 - - Added a new type to core :class:`_types.JSON`. This is the - base of the PostgreSQL :class:`_postgresql.JSON` type as well as that - of the new :class:`.mysql.JSON` type, so that a PG/MySQL-agnostic - JSON column may be used. The type features basic index and path - searching support. - - .. seealso:: - - :ref:`change_3619` - - .. change:: - :tags: bug, sql - :tickets: 3616 - - Fixed an assertion that would raise somewhat inappropriately - if a :class:`.Index` were associated with a :class:`_schema.Column` that - is associated with a lower-case-t :class:`_expression.TableClause`; the - association should be ignored for the purposes of associating - the index with a :class:`_schema.Table`. - - .. change:: - :tags: bug, orm - :tickets: 3601 - - The :meth:`.Session.merge` method now tracks pending objects by - primary key before emitting an INSERT, and merges distinct objects with - duplicate primary keys together as they are encountered, which is - essentially semi-deterministic at best. This behavior - matches what happens already with persistent objects. - - .. seealso:: - - :ref:`change_3601` - - .. change:: - :tags: bug, postgresql - :tickets: 3587 - - Added support for reflecting the source of materialized views - to the PostgreSQL version of the :meth:`_reflection.Inspector.get_view_definition` - method. - - .. change:: - :tags: bug, orm - :tickets: 3582 - - Fixed bug where the "single table inheritance" criteria would be - added onto the end of a query in some inappropriate situations, such - as when querying from an exists() of a single-inheritance subclass. - - .. seealso:: - - :ref:`change_3582` - - .. change:: - :tags: enhancement, schema - - The default generation functions passed to :class:`_schema.Column` objects - are now run through "update_wrapper", or an equivalent function - if a callable non-function is passed, so that introspection tools - preserve the name and docstring of the wrapped function. Pull - request courtesy hsum. - - .. change:: - :tags: change, sql, mysql - :tickets: 3216 - - The system by which a :class:`_schema.Column` considers itself to be an - "auto increment" column has been changed, such that autoincrement - is no longer implicitly enabled for a :class:`_schema.Table` that has a - composite primary key. In order to accommodate being able to enable - autoincrement for a composite PK member column while at the same time - maintaining SQLAlchemy's long standing behavior of enabling - implicit autoincrement for a single integer primary key, a third - state has been added to the :paramref:`_schema.Column.autoincrement` parameter - ``"auto"``, which is now the default. - - .. seealso:: - - :ref:`change_3216` - - :ref:`change_mysql_3216` - - .. change:: - :tags: change, mysql - :tickets: 3216 - - The MySQL dialect no longer generates an extra "KEY" directive when - generating CREATE TABLE DDL for a table using InnoDB with a - composite primary key with AUTO_INCREMENT on a column that isn't the - first column; to overcome InnoDB's limitation here, the PRIMARY KEY - constraint is now generated with the AUTO_INCREMENT column placed - first in the list of columns. - - .. seealso:: - - :ref:`change_mysql_3216` - - :ref:`change_3216` - - .. change:: - :tags: change, sqlite - - Added support to the SQLite dialect for the - :meth:`_reflection.Inspector.get_schema_names` method to work with SQLite; - pull request courtesy Brian Van Klaveren. Also repaired support - for creation of indexes with schemas as well as reflection of - foreign key constraints in schema-bound tables. - - .. seealso:: - - :ref:`change_sqlite_schemas` - - .. change:: - :tags: change, mssql - :tickets: 3434 - - The ``legacy_schema_aliasing`` flag, introduced in version 1.0.5 - as part of :ticket:`3424` to allow disabling of the MSSQL dialect's - attempts to create aliases for schema-qualified tables, now defaults - to False; the old behavior is now disabled unless explicitly turned on. - - .. seealso:: - - :ref:`change_3434` - - .. change:: - :tags: bug, orm - :tickets: 3250 - - Added a new type-level modifier :meth:`.TypeEngine.evaluates_none` - which indicates to the ORM that a positive set of None should be - persisted as the value NULL, instead of omitting the column from - the INSERT statement. This feature is used both as part of the - implementation for :ticket:`3514` as well as a standalone feature - available on any type. - - .. seealso:: - - :ref:`change_3250` - - .. change:: - :tags: bug, postgresql - :tickets: 2729 - - The use of a :class:`_postgresql.ARRAY` object that refers - to a :class:`_types.Enum` or :class:`_postgresql.ENUM` subtype - will now emit the expected "CREATE TYPE" and "DROP TYPE" DDL when - the type is used within a "CREATE TABLE" or "DROP TABLE". - - .. seealso:: - - :ref:`change_2729` - - .. change:: - :tags: bug, sql - :tickets: 3531 - - The :func:`.type_coerce` construct is now a fully fledged Core - expression element which is late-evaluated at compile time. Previously, - the function was only a conversion function which would handle different - expression inputs by returning either a :class:`.Label` of a column-oriented - expression or a copy of a given :class:`.BindParameter` object, - which in particular prevented the operation from being logically - maintained when an ORM-level expression transformation would convert - a column to a bound parameter (e.g. for lazy loading). - - .. seealso:: - - :ref:`change_3531` - - .. change:: - :tags: bug, orm - :tickets: 3526 - - Internal calls to "bookkeeping" functions within - :meth:`.Session.bulk_save_objects` and related bulk methods have - been scaled back to the extent that this functionality is not - currently used, e.g. checks for column default values to be - fetched after an INSERT or UPDATE statement. - - .. change:: - :tags: feature, orm - :tickets: 2677 - - The :class:`.SessionEvents` suite now includes events to allow - unambiguous tracking of all object lifecycle state transitions - in terms of the :class:`.Session` itself, e.g. pending, - transient, persistent, detached. The state of the object - within each event is also defined. - - .. seealso:: - - :ref:`change_2677` - - .. change:: - :tags: feature, orm - :tickets: 2677 - - Added a new session lifecycle state :term:`deleted`. This new state - represents an object that has been deleted from the :term:`persistent` - state and will move to the :term:`detached` state once the transaction - is committed. This resolves the long-standing issue that objects - which were deleted existed in a gray area between persistent and - detached. The :attr:`.InstanceState.persistent` accessor will - **no longer** report on a deleted object as persistent; the - :attr:`.InstanceState.deleted` accessor will instead be True for - these objects, until they become detached. - - .. seealso:: - - :ref:`change_2677` - - .. change:: - :tags: change, orm - :tickets: 2677 - - The :paramref:`.Session.weak_identity_map` parameter is deprecated. - See the new recipe at :ref:`session_referencing_behavior` for - an event-based approach to maintaining strong identity map behavior. - - .. seealso:: - - :ref:`change_2677` - - .. change:: - :tags: bug, sql - :tickets: 2919 - - The :class:`.TypeDecorator` type extender will now work in conjunction - with a :class:`.SchemaType` implementation, typically :class:`.Enum` - or :class:`.Boolean` with regards to ensuring that the per-table - events are propagated from the implementation type to the outer type. - These events are used - to ensure that the constraints or PostgreSQL types (e.g. ENUM) - are correctly created (and possibly dropped) along with the parent - table. - - .. seealso:: - - :ref:`change_2919` - - .. change:: - :tags: feature, sql - :tickets: 1370 - - Added support for "set-aggregate" functions of the form - `` WITHIN GROUP (ORDER BY )``, using the - method :meth:`.FunctionElement.within_group`. A series of common - set-aggregate functions with return types derived from the set have - been added. This includes functions like :class:`.percentile_cont`, - :class:`.dense_rank` and others. - - .. seealso:: - - :ref:`change_3132` - - .. change:: - :tags: feature, sql, postgresql - :tickets: 3132 - - Added support for the SQL-standard function :class:`_functions.array_agg`, - which automatically returns an :class:`_postgresql.ARRAY` of the correct type - and supports index / slice operations, as well as - :func:`_postgresql.array_agg`, which returns a :class:`_postgresql.ARRAY` - with additional comparison features. As arrays are only - supported on PostgreSQL at the moment, only actually works on - PostgreSQL. Also added a new construct - :class:`_postgresql.aggregate_order_by` in support of PG's - "ORDER BY" extension. - - .. seealso:: - - :ref:`change_3132` - - .. change:: - :tags: feature, sql - :tickets: 3516 - - Added a new type to core :class:`_types.ARRAY`. This is the - base of the PostgreSQL :class:`_postgresql.ARRAY` type, and is now part of Core - to begin supporting various SQL-standard array-supporting features - including some functions and eventual support for native arrays - on other databases that have an "array" concept, such as DB2 or Oracle. - Additionally, new operators :func:`_expression.any_` and - :func:`_expression.all_` have been added. These support not just - array constructs on PostgreSQL, but also subqueries that are usable - on MySQL (but sadly not on PostgreSQL). - - .. seealso:: - - :ref:`change_3516` - - .. change:: - :tags: feature, orm - :tickets: 3321 - - Added new checks for the common error case of passing mapped classes - or mapped instances into contexts where they are interpreted as - SQL bound parameters; a new exception is raised for this. - - .. seealso:: - - :ref:`change_3321` - - .. change:: - :tags: bug, postgresql - :tickets: 3499 - - The "hashable" flag on special datatypes such as :class:`_postgresql.ARRAY`, - :class:`_postgresql.JSON` and :class:`_postgresql.HSTORE` is now - set to False, which allows these types to be fetchable in ORM - queries that include entities within the row. - - .. seealso:: - - :ref:`change_3499` - - :ref:`change_3499_postgresql` - - .. change:: - :tags: bug, postgresql - :tickets: 3487 - - The PostgreSQL :class:`_postgresql.ARRAY` type now supports multidimensional - indexed access, e.g. expressions such as ``somecol[5][6]`` without - any need for explicit casts or type coercions, provided - that the :paramref:`.postgresql.ARRAY.dimensions` parameter is set to the - desired number of dimensions. - - .. seealso:: - - :ref:`change_3503` - - .. change:: - :tags: bug, postgresql - :tickets: 3503 - - The return type for the :class:`_postgresql.JSON` and :class:`_postgresql.JSONB` - when using indexed access has been fixed to work like PostgreSQL itself, - and returns an expression that itself is of type :class:`_postgresql.JSON` - or :class:`_postgresql.JSONB`. Previously, the accessor would return - :class:`.NullType` which disallowed subsequent JSON-like operators to be - used. - - .. seealso:: - - :ref:`change_3503` - - .. change:: - :tags: bug, postgresql - :tickets: 3503 - - The :class:`_postgresql.JSON`, :class:`_postgresql.JSONB` and - :class:`_postgresql.HSTORE` datatypes now allow full control over the - return type from an indexed textual access operation, either ``column[someindex].astext`` - for a JSON type or ``column[someindex]`` for an HSTORE type, - via the :paramref:`.postgresql.JSON.astext_type` and - :paramref:`.postgresql.HSTORE.text_type` parameters. - - .. seealso:: - - :ref:`change_3503` - - - .. change:: - :tags: bug, postgresql - :tickets: 3503 - - The :attr:`.postgresql.JSON.Comparator.astext` modifier no longer - calls upon :meth:`_expression.ColumnElement.cast` implicitly, as PG's JSON/JSONB - types allow cross-casting between each other as well. Code that - makes use of :meth:`_expression.ColumnElement.cast` on JSON indexed access, - e.g. ``col[someindex].cast(Integer)``, will need to be changed - to call :attr:`.postgresql.JSON.Comparator.astext` explicitly. - - .. seealso:: - - :ref:`change_3503_cast` - - - .. change:: - :tags: bug, orm, postgresql - :tickets: 3514 - - Additional fixes have been made regarding the value of ``None`` - in conjunction with the PostgreSQL :class:`_postgresql.JSON` type. When - the :paramref:`_types.JSON.none_as_null` flag is left at its default - value of ``False``, the ORM will now correctly insert the JSON - "'null'" string into the column whenever the value on the ORM - object is set to the value ``None`` or when the value ``None`` - is used with :meth:`.Session.bulk_insert_mappings`, - **including** if the column has a default or server default on it. - - .. seealso:: - - :ref:`change_3514` - - :ref:`change_3250` - - .. change:: - :tags: feature, postgresql - :tickets: 3514 - - Added a new constant :attr:`.postgresql.JSON.NULL`, indicating - that the JSON NULL value should be used for a value - regardless of other settings. - - .. seealso:: - - :ref:`change_3514_jsonnull` - - .. change:: - :tags: bug, sql - :tickets: 2528 - - The behavior of the :func:`_expression.union` construct and related constructs - such as :meth:`_query.Query.union` now handle the case where the embedded - SELECT statements need to be parenthesized due to the fact that they - include LIMIT, OFFSET and/or ORDER BY. These queries **do not work - on SQLite**, and will fail on that backend as they did before, but - should now work on all other backends. - - .. seealso:: - - :ref:`change_2528` - - .. change:: - :tags: feature, orm - :tickets: 3512 - - Added new relationship loading strategy :func:`_orm.raiseload` (also - accessible via ``lazy='raise'``). This strategy behaves almost like - :func:`_orm.noload` but instead of returning ``None`` it raises an - InvalidRequestError. Pull request courtesy Adrian Moennich. - - .. seealso:: - - :ref:`change_3512` - - .. change:: - :tags: bug, mssql - :tickets: 3504 - - Fixed issue where the SQL Server dialect would reflect a string- - or other variable-length column type with unbounded length - by assigning the token ``"max"`` to the - length attribute of the string. While using the ``"max"`` token - explicitly is supported by the SQL Server dialect, it isn't part - of the normal contract of the base string types, and instead the - length should just be left as None. The dialect now assigns the - length to None on reflection of the type so that the type behaves - normally in other contexts. - - .. seealso:: - - :ref:`change_3504` diff --git a/doc/build/changelog/changelog_12.rst b/doc/build/changelog/changelog_12.rst deleted file mode 100644 index a0187bc8571..00000000000 --- a/doc/build/changelog/changelog_12.rst +++ /dev/null @@ -1,2945 +0,0 @@ -============= -1.2 Changelog -============= - -.. changelog_imports:: - - .. include:: changelog_11.rst - :start-line: 5 - - - .. include:: changelog_10.rst - :start-line: 5 - - -.. changelog:: - :version: 1.2.19 - :released: April 15, 2019 - - .. change:: - :tags: bug, orm - :tickets: 4507 - - Fixed a regression in 1.2 due to the introduction of baked queries for - relationship lazy loaders, where a race condition is created during the - generation of the "lazy clause" which occurs within a memoized attribute. If - two threads initialize the memoized attribute concurrently, the baked query - could be generated with bind parameter keys that are then replaced with new - keys by the next run, leading to a lazy load query that specifies the - related criteria as ``None``. The fix establishes that the parameter names - are fixed before the new clause and parameter objects are generated, so that - the names are the same every time. - - .. change:: - :tags: bug, oracle - :tickets: 4506 - - Added support for reflection of the :class:`_types.NCHAR` datatype to the Oracle - dialect, and added :class:`_types.NCHAR` to the list of types exported by the - Oracle dialect. - - - .. change:: - :tags: bug, examples - :tickets: 4528 - - Fixed bug in large_resultsets example case where a re-named "id" variable - due to code reformatting caused the test to fail. Pull request courtesy - Matt Schuchhardt. - - .. change:: - :tags: bug, mssql - :tickets: 4536 - :versions: 1.3.1 - - A commit() is emitted after an isolation level change to SNAPSHOT, as both - pyodbc and pymssql open an implicit transaction which blocks subsequent SQL - from being emitted in the current transaction. - - .. change:: - :tags: bug, engine - :tickets: 4406 - - Comparing two objects of :class:`.URL` using ``__eq__()`` did not take port - number into consideration, two objects differing only by port number were - considered equal. Port comparison is now added in ``__eq__()`` method of - :class:`.URL`, objects differing by port number are now not equal. - Additionally, ``__ne__()`` was not implemented for :class:`.URL` which - caused unexpected result when ``!=`` was used in Python2, since there are no - implied relationships among the comparison operators in Python2. - -.. changelog:: - :version: 1.2.18 - :released: February 15, 2019 - - .. change:: - :tags: bug, orm - :tickets: 4468 - - Fixed a regression in 1.2 where a wildcard/load_only loader option would - not work correctly against a loader path where of_type() were used to limit - to a particular subclass. The fix only works for of_type() of a simple - subclass so far, not a with_polymorphic entity which will be addressed in a - separate issue; it is unlikely this latter case was working previously. - - - .. change:: - :tags: bug, orm - :tickets: 4489 - - Fixed fairly simple but critical issue where the - :meth:`.SessionEvents.pending_to_persistent` event would be invoked for - objects not just when they move from pending to persistent, but when they - were also already persistent and just being updated, thus causing the event - to be invoked for all objects on every update. - - .. change:: - :tags: bug, sql - :tickets: 4485 - - Fixed issue where the :class:`_types.JSON` type had a read-only - :attr:`_types.JSON.should_evaluate_none` attribute, which would cause failures - when making use of the :meth:`.TypeEngine.evaluates_none` method in - conjunction with this type. Pull request courtesy Sanjana S. - - .. change:: - :tags: bug, mssql - :tickets: 4499 - - Fixed bug where the SQL Server "IDENTITY_INSERT" logic that allows an INSERT - to proceed with an explicit value on an IDENTITY column was not detecting - the case where :meth:`_expression.Insert.values` were used with a dictionary that - contained a :class:`_schema.Column` as key and a SQL expression as a value. - - .. change:: - :tags: bug, sqlite - :tickets: 4474 - - Fixed bug in SQLite DDL where using an expression as a server side default - required that it be contained within parenthesis to be accepted by the - sqlite parser. Pull request courtesy Bartlomiej Biernacki. - - .. change:: - :tags: bug, mysql - :tickets: 4492 - - Fixed a second regression caused by :ticket:`4344` (the first was - :ticket:`4361`), which works around MySQL issue 88718, where the lower - casing function used was not correct for Python 2 with OSX/Windows casing - conventions, which would then raise ``TypeError``. Full coverage has been - added to this logic so that every codepath is exercised in a mock style for - all three casing conventions on all versions of Python. MySQL 8.0 has - meanwhile fixed issue 88718 so the workaround is only applies to a - particular span of MySQL 8.0 versions. - -.. changelog:: - :version: 1.2.17 - :released: January 25, 2019 - - .. change:: - :tags: feature, orm - :tickets: 4461 - - Added new event hooks :meth:`.QueryEvents.before_compile_update` and - :meth:`.QueryEvents.before_compile_delete` which complement - :meth:`.QueryEvents.before_compile` in the case of the :meth:`_query.Query.update` - and :meth:`_query.Query.delete` methods. - - - .. change:: - :tags: bug, postgresql - :tickets: 4463 - - Revised the query used when reflecting CHECK constraints to make use of the - ``pg_get_constraintdef`` function, as the ``consrc`` column is being - deprecated in PG 12. Thanks to John A Stevenson for the tip. - - - .. change:: - :tags: bug, orm - :tickets: 4454 - - Fixed issue where when using single-table inheritance in conjunction with a - joined inheritance hierarchy that uses "with polymorphic" loading, the - "single table criteria" for that single-table entity could get confused for - that of other entities from the same hierarchy used in the same query.The - adaption of the "single table criteria" is made more specific to the target - entity to avoid it accidentally getting adapted to other tables in the - query. - - - .. change:: - :tags: bug, oracle - :tickets: 4457 - - Fixed regression in integer precision logic due to the refactor of the - cx_Oracle dialect in 1.2. We now no longer apply the cx_Oracle.NATIVE_INT - type to result columns sending integer values (detected as positive - precision with scale ==0) which encounters integer overflow issues with - values that go beyond the 32 bit boundary. Instead, the output variable - is left untyped so that cx_Oracle can choose the best option. - -.. changelog:: - :version: 1.2.16 - :released: January 11, 2019 - - .. change:: - :tag: bug, sql - :tickets: 4394 - - Fixed issue in "expanding IN" feature where using the same bound parameter - name more than once in a query would lead to a KeyError within the process - of rewriting the parameters in the query. - - .. change:: - :tags: bug, postgresql - :tickets: 4416 - - Fixed issue where a :class:`_postgresql.ENUM` or a custom domain present - in a remote schema would not be recognized within column reflection if - the name of the enum/domain or the name of the schema required quoting. - A new parsing scheme now fully parses out quoted or non-quoted tokens - including support for SQL-escaped quotes. - - .. change:: - :tags: bug, postgresql - - Fixed issue where multiple :class:`_postgresql.ENUM` objects referred to - by the same :class:`_schema.MetaData` object would fail to be created if - multiple objects had the same name under different schema names. The - internal memoization the PostgreSQL dialect uses to track if it has - created a particular :class:`_postgresql.ENUM` in the database during - a DDL creation sequence now takes schema name into account. - - .. change:: - :tags: bug, engine - :tickets: 4429 - - Fixed a regression introduced in version 1.2 where a refactor - of the :class:`.SQLAlchemyError` base exception class introduced an - inappropriate coercion of a plain string message into Unicode under - python 2k, which is not handled by the Python interpreter for characters - outside of the platform's encoding (typically ascii). The - :class:`.SQLAlchemyError` class now passes a bytestring through under - Py2K for ``__str__()`` as is the behavior of exception objects in general - under Py2K, does a safe coercion to unicode utf-8 with - backslash fallback for ``__unicode__()``. For Py3K the message is - typically unicode already, but if not is again safe-coerced with utf-8 - with backslash fallback for the ``__str__()`` method. - - .. change:: - :tags: bug, sql, oracle, mysql - :tickets: 4436 - - Fixed issue where the DDL emitted for :class:`.DropTableComment`, which - will be used by an upcoming version of Alembic, was incorrect for the MySQL - and Oracle databases. - - .. change:: - :tags: bug, sqlite - :tickets: 4431 - - Reflection of an index based on SQL expressions are now skipped with a - warning, in the same way as that of the Postgresql dialect, where we currently - do not support reflecting indexes that have SQL expressions within them. - Previously, an index with columns of None were produced which would break - tools like Alembic. - -.. changelog:: - :version: 1.2.15 - :released: December 11, 2018 - - .. change:: - :tags: bug, orm - :tickets: 4367 - - Fixed bug where the ORM annotations could be incorrect for the - primaryjoin/secondaryjoin a relationship if one used the pattern - ``ForeignKey(SomeClass.id)`` in the declarative mappings. This pattern - would leak undesired annotations into the join conditions which can break - aliasing operations done within :class:`_query.Query` that are not supposed to - impact elements in that join condition. These annotations are now removed - up front if present. - - .. change:: - :tags: bug, orm, declarative - :tickets: 4374 - - A warning is emitted in the case that a :func:`_expression.column` object is applied to - a declarative class, as it seems likely this intended to be a - :class:`_schema.Column` object. - - .. change:: - :tags: bug, orm - :tickets: 4366 - - In continuing with a similar theme as that of very recent :ticket:`4349`, - repaired issue with :meth:`.RelationshipProperty.Comparator.any` and - :meth:`.RelationshipProperty.Comparator.has` where the "secondary" - selectable needs to be explicitly part of the FROM clause in the - EXISTS subquery to suit the case where this "secondary" is a :class:`_expression.Join` - object. - - .. change:: - :tags: bug, orm - :tickets: 4363 - - Fixed regression caused by :ticket:`4349` where adding the "secondary" - table to the FROM clause for a dynamic loader would affect the ability of - the :class:`_query.Query` to make a subsequent join to another entity. The fix - adds the primary entity as the first element of the FROM list since - :meth:`_query.Query.join` wants to jump from that. Version 1.3 will have - a more comprehensive solution to this problem as well (:ticket:`4365`). - - - - - .. change:: - :tags: bug, orm - :tickets: 4400 - - Fixed bug where chaining of mapper options using - :meth:`.RelationshipProperty.of_type` in conjunction with a chained option - that refers to an attribute name by string only would fail to locate the - attribute. - - .. change:: - :tag: feature, mysql - :tickets: 4381 - - Added support for the ``write_timeout`` flag accepted by mysqlclient and - pymysql to be passed in the URL string. - - .. change:: - :tag: bug, postgresql - :tickets: 4377, 4380 - - Fixed issue where reflection of a PostgreSQL domain that is expressed as an - array would fail to be recognized. Pull request courtesy Jakub Synowiec. - - -.. changelog:: - :version: 1.2.14 - :released: November 10, 2018 - - .. change:: - :tags: bug, orm - :tickets: 4357 - - Fixed bug in :meth:`.Session.bulk_update_mappings` where alternate mapped - attribute names would result in the primary key column of the UPDATE - statement being included in the SET clause, as well as the WHERE clause; - while usually harmless, for SQL Server this can raise an error due to the - IDENTITY column. This is a continuation of the same bug that was fixed in - :ticket:`3849`, where testing was insufficient to catch this additional - flaw. - - .. change:: - :tags: bug, mysql - :tickets: 4361 - - Fixed regression caused by :ticket:`4344` released in 1.2.13, where the fix - for MySQL 8.0's case sensitivity problem with referenced column names when - reflecting foreign key referents is worked around using the - ``information_schema.columns`` view. The workaround was failing on OSX / - ``lower_case_table_names=2`` which produces non-matching casing for the - ``information_schema.columns`` vs. that of ``SHOW CREATE TABLE``, so in - case-insensitive SQL modes case-insensitive matching is now used. - - .. change:: - :tags: bug, orm - :tickets: 4347 - - Fixed a minor performance issue which could in some cases add unnecessary - overhead to result fetching, involving the use of ORM columns and entities - that include those same columns at the same time within a query. The issue - has to do with hash / eq overhead when referring to the column in different - ways. - -.. changelog:: - :version: 1.2.13 - :released: October 31, 2018 - - .. change:: - :tags: bug, postgresql - :tickets: 4337 - - Added support for the :class:`.aggregate_order_by` function to receive - multiple ORDER BY elements, previously only a single element was accepted. - - - .. change:: - :tags: bug, mysql - :tickets: 4348 - - Added word ``function`` to the list of reserved words for MySQL, which is - now a keyword in MySQL 8.0 - - .. change:: - :tags: feature, sql - :versions: 1.3.0b1 - - Refactored :class:`.SQLCompiler` to expose a - :meth:`.SQLCompiler.group_by_clause` method similar to the - :meth:`.SQLCompiler.order_by_clause` and :meth:`.SQLCompiler.limit_clause` - methods, which can be overridden by dialects to customize how GROUP BY - renders. Pull request courtesy Samuel Chou. - - .. change:: - :tags: bug, misc - - Fixed issue where part of the utility language helper internals was passing - the wrong kind of argument to the Python ``__import__`` builtin as the list - of modules to be imported. The issue produced no symptoms within the core - library but could cause issues with external applications that redefine the - ``__import__`` builtin or otherwise instrument it. Pull request courtesy Joe - Urciuoli. - - .. change:: - :tags: bug, orm - :tickets: 4349 - - Fixed bug where "dynamic" loader needs to explicitly set the "secondary" - table in the FROM clause of the query, to suit the case where the secondary - is a join object that is otherwise not pulled into the query from its - columns alone. - - - .. change:: - :tags: bug, orm, declarative - :tickets: 4350 - - Fixed regression caused by :ticket:`4326` in version 1.2.12 where using - :class:`.declared_attr` with a mixin in conjunction with - :func:`_orm.synonym` would fail to map the synonym properly to an inherited - subclass. - - .. change:: - :tags: bug, misc, py3k - :tickets: 4339 - - Fixed additional warnings generated by Python 3.7 due to changes in the - organization of the Python ``collections`` and ``collections.abc`` packages. - Previous ``collections`` warnings were fixed in version 1.2.11. Pull request - courtesy xtreak. - - .. change:: - :tags: bug, ext - - Added missing ``.index()`` method to list-based association collections - in the association proxy extension. - - .. change:: - :tags: bug, mysql - :tickets: 4344 - - Added a workaround for a MySQL bug #88718 introduced in the 8.0 series, - where the reflection of a foreign key constraint is not reporting the - correct case sensitivity for the referred column, leading to errors during - use of the reflected constraint such as when using the automap extension. - The workaround emits an additional query to the information_schema tables in - order to retrieve the correct case sensitive name. - - .. change:: - :tags: bug, sql - :tickets: 4341 - - Fixed bug where the :paramref:`.Enum.create_constraint` flag on the - :class:`.Enum` datatype would not be propagated to copies of the type, which - affects use cases such as declarative mixins and abstract bases. - - .. change:: - :tags: bug, orm, declarative - :tickets: 4352 - - The column conflict resolution technique discussed at - :ref:`orm_inheritance_column_conflicts` is now functional for a :class:`_schema.Column` - that is also a primary key column. Previously, a check for primary key - columns declared on a single-inheritance subclass would occur before the - column copy were allowed to pass. - - -.. changelog:: - :version: 1.2.12 - :released: September 19, 2018 - - .. change:: - :tags: bug, postgresql - :tickets: 4325 - - Fixed bug in PostgreSQL dialect where compiler keyword arguments such as - ``literal_binds=True`` were not being propagated to a DISTINCT ON - expression. - - .. change:: - :tags: bug, ext - :tickets: 4328 - - Fixed issue where :class:`.BakedQuery` did not include the specific query - class used by the :class:`.Session` as part of the cache key, leading to - incompatibilities when using custom query classes, in particular the - :class:`.ShardedQuery` which has some different argument signatures. - - .. change:: - :tags: bug, postgresql - :tickets: 4324 - - Fixed the :func:`_postgresql.array_agg` function, which is a slightly - altered version of the usual :func:`_functions.array_agg` function, to also - accept an incoming "type" argument without forcing an ARRAY around it, - essentially the same thing that was fixed for the generic function in 1.1 - in :ticket:`4107`. - - .. change:: - :tags: bug, postgresql - :tickets: 4323 - - Fixed bug in PostgreSQL ENUM reflection where a case-sensitive, quoted name - would be reported by the query including quotes, which would not match a - target column during table reflection as the quotes needed to be stripped - off. - - - .. change:: - :tags: bug, orm - - Added a check within the weakref cleanup for the :class:`.InstanceState` - object to check for the presence of the ``dict`` builtin, in an effort to - reduce error messages generated when these cleanups occur during interpreter - shutdown. Pull request courtesy Romuald Brunet. - - .. change:: - :tags: bug, orm, declarative - :tickets: 4326 - - Fixed bug where the declarative scan for attributes would receive the - expression proxy delivered by a hybrid attribute at the class level, and - not the hybrid attribute itself, when receiving the descriptor via the - ``@declared_attr`` callable on a subclass of an already-mapped class. This - would lead to an attribute that did not report itself as a hybrid when - viewed within :attr:`_orm.Mapper.all_orm_descriptors`. - - - .. change:: - :tags: bug, orm - :tickets: 4334 - :versions: 1.3.0b1 - - Fixed bug where use of :class:`_expression.Lateral` construct in conjunction with - :meth:`_query.Query.join` as well as :meth:`_query.Query.select_entity_from` would not - apply clause adaption to the right side of the join. "lateral" introduces - the use case of the right side of a join being correlatable. Previously, - adaptation of this clause wasn't considered. Note that in 1.2 only, - a selectable introduced by :meth:`_query.Query.subquery` is still not adapted - due to :ticket:`4304`; the selectable needs to be produced by the - :func:`_expression.select` function to be the right side of the "lateral" join. - - .. change:: - :tags: bug, oracle - :tickets: 4335 - - Fixed issue for cx_Oracle 7.0 where the behavior of Oracle param.getvalue() - now returns a list, rather than a single scalar value, breaking - autoincrement logic throughout the Core and ORM. The dml_ret_array_val - compatibility flag is used for cx_Oracle 6.3 and 6.4 to establish compatible - behavior with 7.0 and forward, for cx_Oracle 6.2.1 and prior a version - number check falls back to the old logic. - - - .. change:: - :tags: bug, orm - :tickets: 4327 - - Fixed 1.2 regression caused by :ticket:`3472` where the handling of an - "updated_at" style column within the context of a post-update operation - would also occur for a row that is to be deleted following the update, - meaning both that a column with a Python-side value generator would show - the now-deleted value that was emitted for the UPDATE before the DELETE - (which was not the previous behavior), as well as that a SQL- emitted value - generator would have the attribute expired, meaning the previous value - would be unreachable due to the row having been deleted and the object - detached from the session.The "postfetch" logic that was added as part of - :ticket:`3472` is now skipped entirely for an object that ultimately is to - be deleted. - -.. changelog:: - :version: 1.2.11 - :released: August 20, 2018 - - .. change:: - :tags: bug, py3k - - Started importing "collections" from "collections.abc" under Python 3.3 and - greater for Python 3.8 compatibility. Pull request courtesy Nathaniel - Knight. - - .. change:: - :tag: bug, sqlite - - Fixed issue where the "schema" name used for a SQLite database within table - reflection would not quote the schema name correctly. Pull request - courtesy Phillip Cloud. - - .. change:: - :tags: bug, sql - :tickets: 4320 - - Fixed issue that is closely related to :ticket:`3639` where an expression - rendered in a boolean context on a non-native boolean backend would - be compared to 1/0 even though it is already an implicitly boolean - expression, when :meth:`_expression.ColumnElement.self_group` were used. While this - does not affect the user-friendly backends (MySQL, SQLite) it was not - handled by Oracle (and possibly SQL Server). Whether or not the - expression is implicitly boolean on any database is now determined - up front as an additional check to not generate the integer comparison - within the compilation of the statement. - - .. change:: - :tags: bug, oracle - :tickets: 4309 - - For cx_Oracle, Integer datatypes will now be bound to "int", per advice - from the cx_Oracle developers. Previously, using cx_Oracle.NUMBER caused a - loss in precision within the cx_Oracle 6.x series. - - - .. change:: - :tags: bug, orm, declarative - :tickets: 4321 - - Fixed issue in previously untested use case, allowing a declarative mapped - class to inherit from a classically-mapped class outside of the declarative - base, including that it accommodates for unmapped intermediate classes. An - unmapped intermediate class may specify ``__abstract__``, which is now - interpreted correctly, or the intermediate class can remain unmarked, and - the classically mapped base class will be detected within the hierarchy - regardless. In order to anticipate existing scenarios which may be mixing - in classical mappings into existing declarative hierarchies, an error is - now raised if multiple mapped bases are detected for a given class. - - .. change:: - :tags: bug, sql - :tickets: 4322 - - Added missing window function parameters - :paramref:`.WithinGroup.over.range_` and :paramref:`.WithinGroup.over.rows` - parameters to the :meth:`.WithinGroup.over` and - :meth:`.FunctionFilter.over` methods, to correspond to the range/rows - feature added to the "over" method of SQL functions as part of - :ticket:`3049` in version 1.1. - - .. change:: - :tags: bug, sql - :tickets: 4313 - - Fixed bug where the multi-table support for UPDATE and DELETE statements - did not consider the additional FROM elements as targets for correlation, - when a correlated SELECT were also combined with the statement. This - change now includes that a SELECT statement in the WHERE clause for such a - statement will try to auto-correlate back to these additional tables in the - parent UPDATE/DELETE or unconditionally correlate if - :meth:`_expression.Select.correlate` is used. Note that auto-correlation raises an - error if the SELECT statement would have no FROM clauses as a result, which - can now occur if the parent UPDATE/DELETE specifies the same tables in its - additional set of tables; specify :meth:`_expression.Select.correlate` explicitly to - resolve. - -.. changelog:: - :version: 1.2.10 - :released: July 13, 2018 - - .. change:: - :tags: bug, sql - :tickets: 4300 - - Fixed bug where a :class:`.Sequence` would be dropped explicitly before any - :class:`_schema.Table` that refers to it, which breaks in the case when the - sequence is also involved in a server-side default for that table, when - using :meth:`_schema.MetaData.drop_all`. The step which processes sequences - to be dropped via non server-side column default functions is now invoked - after the table itself is dropped. - - .. change:: - :tags: bug, orm - :tickets: 4295 - - Fixed bug in :class:`.Bundle` construct where placing two columns of the - same name would be de-duplicated, when the :class:`.Bundle` were used as - part of the rendered SQL, such as in the ORDER BY or GROUP BY of the statement. - - - .. change:: - :tags: bug, orm - :tickets: 4298 - - Fixed regression in 1.2.9 due to :ticket:`4287` where using a - :class:`_orm.Load` option in conjunction with a string wildcard would result - in a TypeError. - -.. changelog:: - :version: 1.2.9 - :released: June 29, 2018 - - .. change:: - :tags: bug, mysql - - Fixed percent-sign doubling in mysql-connector-python dialect, which does - not require de-doubling of percent signs. Additionally, the mysql- - connector-python driver is inconsistent in how it passes the column names - in cursor.description, so a workaround decoder has been added to - conditionally decode these randomly-sometimes-bytes values to unicode only - if needed. Also improved test support for mysql-connector-python, however - it should be noted that this driver still has issues with unicode that - continue to be unresolved as of yet. - - - .. change:: - :tags: bug, mssql - :tickets: 4288 - - Fixed bug in MSSQL reflection where when two same-named tables in different - schemas had same-named primary key constraints, foreign key constraints - referring to one of the tables would have their columns doubled, causing - errors. Pull request courtesy Sean Dunn. - - .. change:: - :tags: bug, sql - :tickets: 4279 - - Fixed regression in 1.2 due to :ticket:`4147` where a :class:`_schema.Table` that - has had some of its indexed columns redefined with new ones, as would occur - when overriding columns during reflection or when using - :paramref:`_schema.Table.extend_existing`, such that the :meth:`_schema.Table.tometadata` - method would fail when attempting to copy those indexes as they still - referred to the replaced column. The copy logic now accommodates for this - condition. - - - .. change:: - :tags: bug, mysql - :tickets: 4293 - - Fixed bug in index reflection where on MySQL 8.0 an index that includes - ASC or DESC in an indexed column specification would not be correctly - reflected, as MySQL 8.0 introduces support for returning this information - in a table definition string. - - .. change:: - :tags: bug, orm - :tickets: 3505 - - Fixed issue where chaining multiple join elements inside of - :meth:`_query.Query.join` might not correctly adapt to the previous left-hand - side, when chaining joined inheritance classes that share the same base - class. - - .. change:: - :tags: bug, orm - :tickets: 4287 - - Fixed bug in cache key generation for baked queries which could cause a - too-short cache key to be generated for the case of eager loads across - subclasses. This could in turn cause the eagerload query to be cached in - place of a non-eagerload query, or vice versa, for a polymorphic "selectin" - load, or possibly for lazy loads or selectin loads as well. - - .. change:: - :tags: bug, sqlite - - Fixed issue in test suite where SQLite 3.24 added a new reserved word that - conflicted with a usage in TypeReflectionTest. Pull request courtesy Nils - Philippsen. - - .. change:: - :tags: feature, oracle - :tickets: 4290 - :versions: 1.3.0b1 - - Added a new event currently used only by the cx_Oracle dialect, - :meth:`.DialectEvents.setiputsizes`. The event passes a dictionary of - :class:`.BindParameter` objects to DBAPI-specific type objects that will be - passed, after conversion to parameter names, to the cx_Oracle - ``cursor.setinputsizes()`` method. This allows both visibility into the - setinputsizes process as well as the ability to alter the behavior of what - datatypes are passed to this method. - - .. seealso:: - - :ref:`cx_oracle_setinputsizes` - - .. change:: - :tags: bug, orm - :tickets: 4286 - - Fixed bug in new polymorphic selectin loading where the BakedQuery used - internally would be mutated by the given loader options, which would both - inappropriately mutate the subclass query as well as carry over the effect - to subsequent queries. - - .. change:: - :tags: bug, py3k - :tickets: 4291 - - Replaced the usage of inspect.formatargspec() with a vendored version - copied from the Python standard library, as inspect.formatargspec() - is deprecated and as of Python 3.7.0 is emitting a warning. - - .. change:: - :tags: feature, ext - :tickets: 4243 - :versions: 1.3.0b1 - - Added new attribute :attr:`_query.Query.lazy_loaded_from` which is populated - with an :class:`.InstanceState` that is using this :class:`_query.Query` in - order to lazy load a relationship. The rationale for this is that - it serves as a hint for the horizontal sharding feature to use, such that - the identity token of the state can be used as the default identity token - to use for the query within id_chooser(). - - .. change:: - :tags: bug, mysql - :tickets: 4283 - - Fixed bug in MySQLdb dialect and variants such as PyMySQL where an - additional "unicode returns" check upon connection makes explicit use of - the "utf8" character set, which in MySQL 8.0 emits a warning that utf8mb4 - should be used. This is now replaced with a utf8mb4 equivalent. - Documentation is also updated for the MySQL dialect to specify utf8mb4 in - all examples. Additional changes have been made to the test suite to use - utf8mb3 charsets and databases (there seem to be collation issues in some - edge cases with utf8mb4), and to support configuration default changes made - in MySQL 8.0 such as explicit_defaults_for_timestamp as well as new errors - raised for invalid MyISAM indexes. - - - - .. change:: - :tags: bug, mysql - :tickets: 3645 - - The :class:`_expression.Update` construct now accommodates a :class:`_expression.Join` object - as supported by MySQL for UPDATE..FROM. As the construct already - accepted an alias object for a similar purpose, the feature of UPDATE - against a non-table was already implied so this has been added. - - .. change:: - :tags: bug, mssql, py3k - :tickets: 4273 - - Fixed issue within the SQL Server dialect under Python 3 where when running - against a non-standard SQL server database that does not contain either the - "sys.dm_exec_sessions" or "sys.dm_pdw_nodes_exec_sessions" views, leading - to a failure to fetch the isolation level, the error raise would fail due - to an UnboundLocalError. - - - - .. change:: - :tags: bug, orm - :tickets: 4269 - - Fixed regression caused by :ticket:`4256` (itself a regression fix for - :ticket:`4228`) which breaks an undocumented behavior which converted for a - non-sequence of entities passed directly to the :class:`_query.Query` constructor - into a single-element sequence. While this behavior was never supported or - documented, it's already in use so has been added as a behavioral contract - to :class:`_query.Query`. - - .. change:: - :tags: bug, orm - :tickets: 4270 - - Fixed an issue that was both a performance regression in 1.2 as well as an - incorrect result regarding the "baked" lazy loader, involving the - generation of cache keys from the original :class:`_query.Query` object's loader - options. If the loader options were built up in a "branched" style using - common base elements for multiple options, the same options would be - rendered into the cache key repeatedly, causing both a performance issue as - well as generating the wrong cache key. This is fixed, along with a - performance improvement when such "branched" options are applied via - :meth:`_query.Query.options` to prevent the same option objects from being - applied repeatedly. - - .. change:: - :tags: bug, oracle, mysql - :tickets: 4275 - - Fixed INSERT FROM SELECT with CTEs for the Oracle and MySQL dialects, where - the CTE was being placed above the entire statement as is typical with - other databases, however Oracle and MariaDB 10.2 wants the CTE underneath - the "INSERT" segment. Note that the Oracle and MySQL dialects don't yet - work when a CTE is applied to a subquery inside of an UPDATE or DELETE - statement, as the CTE is still applied to the top rather than inside the - subquery. - - -.. changelog:: - :version: 1.2.8 - :released: May 28, 2018 - - .. change:: - :tags: bug, orm - :tickets: 4256 - - Fixed regression in 1.2.7 caused by :ticket:`4228`, which itself was fixing - a 1.2-level regression, where the ``query_cls`` callable passed to a - :class:`.Session` was assumed to be a subclass of :class:`_query.Query` with - class method availability, as opposed to an arbitrary callable. In - particular, the dogpile caching example illustrates ``query_cls`` as a - function and not a :class:`_query.Query` subclass. - - .. change:: - :tags: bug, engine - :tickets: 4252 - - Fixed connection pool issue whereby if a disconnection error were raised - during the connection pool's "reset on return" sequence in conjunction with - an explicit transaction opened against the enclosing :class:`_engine.Connection` - object (such as from calling :meth:`.Session.close` without a rollback or - commit, or calling :meth:`_engine.Connection.close` without first closing a - transaction declared with :meth:`_engine.Connection.begin`), a double-checkin would - result, which could then lead towards concurrent checkouts of the same - connection. The double-checkin condition is now prevented overall by an - assertion, as well as the specific double-checkin scenario has been - fixed. - - .. change:: - :tags: bug, oracle - :tickets: 4264 - - The Oracle BINARY_FLOAT and BINARY_DOUBLE datatypes now participate within - cx_Oracle.setinputsizes(), passing along NATIVE_FLOAT, so as to support the - NaN value. Additionally, :class:`_oracle.BINARY_FLOAT`, - :class:`_oracle.BINARY_DOUBLE` and :class:`_oracle.DOUBLE_PRECISION` now - subclass :class:`.Float`, since these are floating point datatypes, not - decimal. These datatypes were already defaulting the - :paramref:`.Float.asdecimal` flag to False in line with what - :class:`.Float` already does. - - .. change:: - :tags: bug, oracle - - Added reflection capabilities for the :class:`_oracle.BINARY_FLOAT`, - :class:`_oracle.BINARY_DOUBLE` datatypes. - - - .. change:: - :tags: bug, ext - :tickets: 4247 - - The horizontal sharding extension now makes use of the identity token - added to ORM identity keys as part of :ticket:`4137`, when an object - refresh or column-based deferred load or unexpiration operation occurs. - Since we know the "shard" that the object originated from, we make - use of this value when refreshing, thereby avoiding queries against - other shards that don't match this object's identity in any case. - - .. change:: - :tags: bug, sql - - Fixed issue where the "ambiguous literal" error message used when - interpreting literal values as SQL expression values would encounter a - tuple value, and fail to format the message properly. Pull request courtesy - Miguel Ventura. - - .. change:: - :tags: bug, mssql - :tickets: 4250 - - Fixed a 1.2 regression caused by :ticket:`4061` where the SQL Server - "BIT" type would be considered to be "native boolean". The goal here - was to avoid creating a CHECK constraint on the column, however the bigger - issue is that the BIT value does not behave like a true/false constant - and cannot be interpreted as a standalone expression, e.g. - "WHERE ". The SQL Server dialect now goes back to being - non-native boolean, but with an extra flag that still avoids creating - the CHECK constraint. - - .. change:: - :tags: bug, oracle - :tickets: 4259 - - Altered the Oracle dialect such that when an :class:`.Integer` type is in - use, the cx_Oracle.NUMERIC type is set up for setinputsizes(). In - SQLAlchemy 1.1 and earlier, cx_Oracle.NUMERIC was passed for all numeric - types unconditionally, and in 1.2 this was removed to allow for better - numeric precision. However, for integers, some database/client setups - will fail to coerce boolean values True/False into integers which introduces - regressive behavior when using SQLAlchemy 1.2. Overall, the setinputsizes - logic seems like it will need a lot more flexibility going forward so this - is a start for that. - - .. change:: - :tags: bug, engine - - Fixed a reference leak issue where the values of the parameter dictionary - used in a statement execution would remain referenced by the "compiled - cache", as a result of storing the key view used by Python 3 dictionary - keys(). Pull request courtesy Olivier Grisel. - - .. change:: - :tags: bug, orm - :tickets: 4128 - - Fixed a long-standing regression that occurred in version - 1.0, which prevented the use of a custom :class:`.MapperOption` - that alters the _params of a :class:`_query.Query` object for a - lazy load, since the lazy loader itself would overwrite those - parameters. This applies to the "temporal range" example - on the wiki. Note however that the - :meth:`_query.Query.populate_existing` method is now required in - order to rewrite the mapper options associated with an object - already loaded in the identity map. - - As part of this change, a custom defined - :class:`.MapperOption` will now cause lazy loaders related to - the target object to use a non-baked query by default unless - the :meth:`.MapperOption._generate_cache_key` method is implemented. - In particular, this repairs one regression which occurred when - using the dogpile.cache "advanced" example, which was not - returning cached results and instead emitting SQL due to an - incompatibility with the baked query loader; with the change, - the ``RelationshipCache`` option included for many releases - in the dogpile example will disable the "baked" query altogether. - Note that the dogpile example is also modernized to avoid both - of these issues as part of issue :ticket:`4258`. - - .. change:: - :tags: bug, ext - :tickets: 4266 - - Fixed a race condition which could occur if automap - :meth:`.AutomapBase.prepare` were used within a multi-threaded context - against other threads which may call :func:`.configure_mappers` as a - result of use of other mappers. The unfinished mapping work of automap - is particularly sensitive to being pulled in by a - :func:`.configure_mappers` step leading to errors. - - .. change:: - :tags: bug, orm - - Fixed bug where the new :meth:`.baked.Result.with_post_criteria` - method would not interact with a subquery-eager loader correctly, - in that the "post criteria" would not be applied to embedded - subquery eager loaders. This is related to :ticket:`4128` in that - the post criteria feature is now used by the lazy loader. - - .. change:: - :tags: bug, tests - :tickets: 4249 - - Fixed a bug in the test suite where if an external dialect returned - ``None`` for ``server_version_info``, the exclusion logic would raise an - ``AttributeError``. - - .. change:: - :tags: bug, orm - :tickets: 4258 - - Updated the dogpile.caching example to include new structures that - accommodate for the "baked" query system, which is used by default within - lazy loaders and some eager relationship loaders. The dogpile.caching - "relationship_caching" and "advanced" examples were also broken due to - :ticket:`4256`. The issue here is also worked-around by the fix in - :ticket:`4128`. - -.. changelog:: - :version: 1.2.7 - :released: April 20, 2018 - - .. change:: - :tags: bug, orm - :tickets: 4228 - - Fixed regression in 1.2 within sharded query feature where the - new "identity_token" element was not being correctly considered within - the scope of a lazy load operation, when searching the identity map - for a related many-to-one element. The new behavior will allow for - making use of the "id_chooser" in order to determine the best identity - key to retrieve from the identity map. In order to achieve this, some - refactoring of 1.2's "identity_token" approach has made some slight changes - to the implementation of ``ShardedQuery`` which should be noted for other - derivations of this class. - - .. change:: - :tags: bug, postgresql - :tickets: 4229 - - Fixed bug where the special "not equals" operator for the PostgreSQL - "range" datatypes such as DATERANGE would fail to render "IS NOT NULL" when - compared to the Python ``None`` value. - - - - .. change:: - :tags: bug, mssql - :tickets: 4234 - - Fixed 1.2 regression caused by :ticket:`4060` where the query used to - reflect SQL Server cross-schema foreign keys was limiting the criteria - incorrectly. - - - - .. change:: - :tags: bug, oracle - - The Oracle NUMBER datatype is reflected as INTEGER if the precision is NULL - and the scale is zero, as this is how INTEGER values come back when - reflected from Oracle's tables. Pull request courtesy Kent Bower. - - .. change:: - :tags: feature, postgresql - :tickets: 4160 - :versions: 1.3.0b1 - - Added new PG type :class:`_postgresql.REGCLASS` which assists in casting - table names to OID values. Pull request courtesy Sebastian Bank. - - .. change:: - :tags: bug, sql - :tickets: 4231 - - Fixed issue where the compilation of an INSERT statement with the - "literal_binds" option that also uses an explicit sequence and "inline" - generation, as on PostgreSQL and Oracle, would fail to accommodate the - extra keyword argument within the sequence processing routine. - - .. change:: - :tags: bug, orm - :tickets: 4241 - - Fixed issue in single-inheritance loading where the use of an aliased - entity against a single-inheritance subclass in conjunction with the - :meth:`_query.Query.select_from` method would cause the SQL to be rendered with - the unaliased table mixed in to the query, causing a cartesian product. In - particular this was affecting the new "selectin" loader when used against a - single-inheritance subclass. - -.. changelog:: - :version: 1.2.6 - :released: March 30, 2018 - - .. change:: - :tags: bug, mssql - :tickets: 4227 - - Adjusted the SQL Server version detection for pyodbc to only allow for - numeric tokens, filtering out non-integers, since the dialect does tuple- - numeric comparisons with this value. This is normally true for all known - SQL Server / pyodbc drivers in any case. - - .. change:: - :tags: feature, postgresql - - Added support for "PARTITION BY" in PostgreSQL table definitions, - using "postgresql_partition_by". Pull request courtesy - Vsevolod Solovyov. - - .. change:: - :tags: bug, sql - :tickets: 4204 - - Fixed a regression that occurred from the previous fix to :ticket:`4204` in - version 1.2.5, where a CTE that refers to itself after the - :meth:`_expression.CTE.alias` method has been called would not refer to itself - correctly. - - .. change:: - :tags: bug, engine - :tickets: 4225 - - Fixed bug in connection pool where a connection could be present in the - pool without all of its "connect" event handlers called, if a previous - "connect" handler threw an exception; note that the dialects themselves - have connect handlers that emit SQL, such as those which set transaction - isolation, which can fail if the database is in a non-available state, but - still allows a connection. The connection is now invalidated first if any - of the connect handlers fail. - - .. change:: - :tags: bug, oracle - :tickets: 4211 - - The minimum cx_Oracle version supported is 5.2 (June 2015). Previously, - the dialect asserted against version 5.0 but as of 1.2.2 we are using some - symbols that did not appear until 5.2. - - .. change:: - :tags: bug, declarative - :tickets: 4221 - - Removed a warning that would be emitted when calling upon - ``__table_args__``, ``__mapper_args__`` as named with a ``@declared_attr`` - method, when called from a non-mapped declarative mixin. Calling these - directly is documented as the approach to use when one is overriding one - of these methods on a mapped class. The warning still emits for regular - attribute names. - - .. change:: - :tags: bug, orm - :tickets: 4215 - - Fixed bug where using :meth:`.Mutable.associate_with` or - :meth:`.Mutable.as_mutable` in conjunction with a class that has non- - primary mappers set up with alternatively-named attributes would produce an - attribute error. Since non-primary mappers are not used for persistence, - the mutable extension now excludes non-primary mappers from its - instrumentation steps. - - -.. changelog:: - :version: 1.2.5 - :released: March 6, 2018 - - .. change:: - :tags: bug, sql - :tickets: 4210 - - Fixed bug in :class:.`CTE` construct along the same lines as that of - :ticket:`4204` where a :class:`_expression.CTE` that was aliased would not copy itself - correctly during a "clone" operation as is frequent within the ORM as well - as when using the :meth:`_expression.ClauseElement.params` method. - - .. change:: - :tags: bug, orm - :tickets: 4199 - - Fixed bug in new "polymorphic selectin" loading when a selection of - polymorphic objects were to be partially loaded from a relationship - lazy loader, leading to an "empty IN" condition within the load that - raises an error for the "inline" form of "IN". - - .. change:: - :tags: bug, sql - :tickets: 4204 - - Fixed bug in CTE rendering where a :class:`_expression.CTE` that was also turned into - an :class:`_expression.Alias` would not render its "ctename AS aliasname" clause - appropriately if there were more than one reference to the CTE in a FROM - clause. - - .. change:: - :tags: bug, orm - :tickets: 4209 - - Fixed 1.2 regression where a mapper option that contains an - :class:`.AliasedClass` object, as is typical when using the - :meth:`.QueryableAttribute.of_type` method, could not be pickled. 1.1's - behavior was to omit the aliased class objects from the path, so this - behavior is restored. - - .. change:: - :tags: feature, orm - :versions: 1.3.0b1 - - Added new feature :meth:`_query.Query.only_return_tuples`. Causes the - :class:`_query.Query` object to return keyed tuple objects unconditionally even - if the query is against a single entity. Pull request courtesy Eric - Atkin. - - - .. change:: - :tags: bug, sql - :tickets: 4198 - - Fixed bug in new "expanding IN parameter" feature where the bind parameter - processors for values wasn't working at all, tests failed to cover this - pretty basic case which includes that ENUM values weren't working. - -.. changelog:: - :version: 1.2.4 - :released: February 22, 2018 - - .. change:: - :tags: bug, orm - :tickets: 4193 - - Fixed 1.2 regression in ORM versioning feature where a mapping against a - :func:`_expression.select` or :func:`.alias` that also used a versioning column - against the underlying table would fail due to the check added as part of - :ticket:`3673`. - - .. change:: - :tags: bug, engine - :tickets: 4190 - - Fixed regression caused in 1.2.3 due to fix from :ticket:`4181` where - the changes to the event system involving :class:`_engine.Engine` and - :class:`.OptionEngine` did not accommodate for event removals, which - would raise an ``AttributeError`` when invoked at the class - level. - - .. change:: - :tags: bug, sql - :tickets: 4197 - - Fixed bug where CTE expressions would not have their name or alias name - quoted when the given name is case sensitive or otherwise requires quoting. - Pull request courtesy Eric Atkin. - -.. changelog:: - :version: 1.2.3 - :released: February 16, 2018 - - .. change:: - :tags: bug, oracle - :tickets: 4182 - - Fixed bug in cx_Oracle disconnect detection, used by pre_ping and other - features, where an error could be raised as DatabaseError which includes a - numeric error code; previously we weren't checking in this case for a - disconnect code. - - .. change:: - :tags: bug, sqlite - - Fixed the import error raised when a platform - has neither pysqlite2 nor sqlite3 installed, such - that the sqlite3-related import error is raised, - not the pysqlite2 one which is not the actual - failure mode. Pull request courtesy Robin. - - .. change:: - :tags: bug, orm - :tickets: 4175 - - Fixed bug where the :class:`.Bundle` object did not - correctly report upon the primary :class:`_orm.Mapper` object - represented by the bundle, if any. An immediate - side effect of this issue was that the new selectinload - loader strategy wouldn't work with the horizontal sharding - extension. - - .. change:: - :tags: bug, sql - :tickets: 4180 - - Fixed bug where the :class:`.Enum` type wouldn't handle - enum "aliases" correctly, when more than one key refers to the - same value. Pull request courtesy Daniel Knell. - - - .. change:: - :tags: bug, engine - :tickets: 4181 - - Fixed bug where events associated with an :class:`Engine` - at the class level would be doubled when the - :meth:`_engine.Engine.execution_options` method were used. To - achieve this, the semi-private class :class:`.OptionEngine` - no longer accepts events directly at the class level - and will raise an error; the class only propagates class-level - events from its parent :class:`_engine.Engine`. Instance-level - events continue to work as before. - - .. change:: - :tags: bug, tests - :tickets: 3265 - - A test added in 1.2 thought to confirm a Python 2.7 behavior turns out to - be confirming the behavior only as of Python 2.7.8. Python bug #8743 still - impacts set comparison in Python 2.7.7 and earlier, so the test in question - involving AssociationSet no longer runs for these older Python 2.7 - versions. - - .. change:: - :tags: feature, oracle - - The ON DELETE options for foreign keys are now part of - Oracle reflection. Oracle does not support ON UPDATE - cascades. Pull request courtesy Miroslav Shubernetskiy. - - - - .. change:: - :tags: bug, orm - :tickets: 4188 - - Fixed bug in concrete inheritance mapping where user-defined - attributes such as hybrid properties that mirror the names - of mapped attributes from sibling classes would be overwritten by - the mapper as non-accessible at the instance level. Additionally - ensured that user-bound descriptors are not implicitly invoked at the class - level during the mapper configuration stage. - - .. change:: - :tags: bug, orm - :tickets: 4178 - - Fixed bug where the :func:`_orm.reconstructor` event - helper would not be recognized if it were applied to the - ``__init__()`` method of the mapped class. - - .. change:: - :tags: bug, engine - :tickets: 4170 - - The :class:`.URL` object now allows query keys to be specified multiple - times where their values will be joined into a list. This is to support - the plugins feature documented at :class:`.CreateEnginePlugin` which - documents that "plugin" can be passed multiple times. Additionally, the - plugin names can be passed to :func:`_sa.create_engine` outside of the URL - using the new :paramref:`_sa.create_engine.plugins` parameter. - - .. change:: - :tags: feature, sql - :tickets: 3906 - - Added support for :class:`.Enum` to persist the values of the enumeration, - rather than the keys, when using a Python pep-435 style enumerated object. - The user supplies a callable function that will return the string values to - be persisted. This allows enumerations against non-string values to be - value-persistable as well. Pull request courtesy Jon Snyder. - - .. change:: - :tags: feature, orm - - Added new argument :paramref:`.attributes.set_attribute.inititator` - to the :func:`.attributes.set_attribute` function, allowing an - event token received from a listener function to be propagated - to subsequent set events. - -.. changelog:: - :version: 1.2.2 - :released: January 24, 2018 - - .. change:: - :tags: bug, mssql - :tickets: 4164 - - Added ODBC error code 10054 to the list of error - codes that count as a disconnect for ODBC / MSSQL server. - - - .. change:: - :tags: bug, orm - :tickets: 4171 - - Fixed 1.2 regression regarding new bulk_replace event - where a backref would fail to remove an object from the - previous owner when a bulk-assignment assigned the - object to a new owner. - - .. change:: - :tags: bug, oracle - :tickets: 4163 - - The cx_Oracle dialect now calls setinputsizes() with cx_Oracle.NCHAR - unconditionally when the NVARCHAR2 datatype, in SQLAlchemy corresponding - to sqltypes.Unicode(), is in use. Per cx_Oracle's author this allows - the correct conversions to occur within the Oracle client regardless - of the setting for NLS_NCHAR_CHARACTERSET. - - .. change:: - :tags: bug, mysql - - Added more MySQL 8.0 reserved words to the MySQL dialect - for quoting purposes. Pull request courtesy - Riccardo Magliocchetti. - -.. changelog:: - :version: 1.2.1 - :released: January 15, 2018 - - .. change:: - :tags: bug, orm - :tickets: 4159 - - Fixed regression where pickle format of a Load / _UnboundLoad object (e.g. - loader options) changed and ``__setstate__()`` was raising an - UnboundLocalError for an object received from the legacy format, even - though an attempt was made to do so. tests are now added to ensure this - works. - - .. change:: - :tags: bug, ext - :tickets: 4150 - - Fixed regression in association proxy due to :ticket:`3769` - (allow for chained any() / has()) where contains() against - an association proxy chained in the form - (o2m relationship, associationproxy(m2o relationship, m2o relationship)) - would raise an error regarding the re-application of contains() - on the final link of the chain. - - .. change:: - :tags: bug, orm - :tickets: 4153 - - Fixed regression caused by new lazyload caching scheme in :ticket:`3954` - where a query that makes use of loader options with of_type would cause - lazy loads of unrelated paths to fail with a TypeError. - - .. change:: - :tags: bug, oracle - :tickets: 4157 - - Fixed regression where the removal of most setinputsizes - rules from cx_Oracle dialect impacted the TIMESTAMP - datatype's ability to retrieve fractional seconds. - - - - .. change:: - :tags: bug, tests - - Removed an oracle-specific requirements rule from the public - test suite that was interfering with third party dialect - suites. - - .. change:: - :tags: bug, mssql - :tickets: 4154 - - Fixed regression in 1.2 where newly repaired quoting - of collation names in :ticket:`3785` breaks SQL Server, - which explicitly does not understand a quoted collation - name. Whether or not mixed-case collation names are - quoted or not is now deferred down to a dialect-level - decision so that each dialect can prepare these identifiers - directly. - - .. change:: - :tags: bug, orm - :tickets: 4156 - - Fixed bug in new "selectin" relationship loader where the loader could try - to load a non-existent relationship when loading a collection of - polymorphic objects, where only some of the mappers include that - relationship, typically when :meth:`.PropComparator.of_type` is being used. - - .. change:: - :tags: bug, tests - - Added a new exclusion rule group_by_complex_expression - which disables tests that use "GROUP BY ", which seems - to be not viable for at least two third party dialects. - - .. change:: - :tags: bug, oracle - - Fixed regression in Oracle imports where a missing comma caused - an undefined symbol to be present. Pull request courtesy - Miroslav Shubernetskiy. - -.. changelog:: - :version: 1.2.0 - :released: December 27, 2017 - - .. change:: - :tags: orm, feature - :tickets: 4137 - - Added a new data member to the identity key tuple - used by the ORM's identity map, known as the - "identity_token". This token defaults to None but - may be used by database sharding schemes to differentiate - objects in memory with the same primary key that come - from different databases. The horizontal sharding - extension integrates this token applying the shard - identifier to it, thus allowing primary keys to be - duplicated across horizontally sharded backends. - - .. seealso:: - - :ref:`change_4137` - - .. change:: - :tags: bug, mysql - :tickets: 4115 - - Fixed regression from issue 1.2.0b3 where "MariaDB" version comparison can - fail for some particular MariaDB version strings under Python 3. - - .. change:: - :tags: enhancement, sql - :tickets: 959 - - Implemented "DELETE..FROM" syntax for PostgreSQL, MySQL, MS SQL Server - (as well as within the unsupported Sybase dialect) in a manner similar - to how "UPDATE..FROM" works. A DELETE statement that refers to more than - one table will switch into "multi-table" mode and render the appropriate - "USING" or multi-table "FROM" clause as understood by the database. - Pull request courtesy Pieter Mulder. - - .. seealso:: - - :ref:`change_959` - - .. change:: - :tags: bug, sql - :tickets: 2694 - - Reworked the new "autoescape" feature introduced in - :ref:`change_2694` in 1.2.0b2 to be fully automatic; the escape - character now defaults to a forwards slash ``"/"`` and - is applied to percent, underscore, as well as the escape - character itself, for fully automatic escaping. The - character can also be changed using the "escape" parameter. - - .. seealso:: - - :ref:`change_2694` - - - .. change:: - :tags: bug, sql - :tickets: 4147 - - Fixed bug where the :meth:`_schema.Table.tometadata` method would not properly - accommodate :class:`.Index` objects that didn't consist of simple - column expressions, such as indexes against a :func:`_expression.text` construct, - indexes that used SQL expressions or :attr:`.func`, etc. The routine - now copies expressions fully to a new :class:`.Index` object while - substituting all table-bound :class:`_schema.Column` objects for those - of the target table. - - .. change:: - :tags: bug, sql - :tickets: 4142 - - Changed the "visit name" of :class:`_expression.ColumnElement` from "column" to - "column_element", so that when this element is used as the basis for a - user-defined SQL element, it is not assumed to behave like a table-bound - :class:`.ColumnClause` when processed by various SQL traversal utilities, - as are commonly used by the ORM. - - .. change:: - :tags: bug, sql, ext - :tickets: 4141 - - Fixed issue in :class:`_types.ARRAY` datatype which is essentially the same - issue as that of :ticket:`3832`, except not a regression, where - column attachment events on top of :class:`_types.ARRAY` would not fire - correctly, thus interfering with systems which rely upon this. A key - use case that was broken by this is the use of mixins to declare - columns that make use of :meth:`.MutableList.as_mutable`. - - .. change:: - :tags: feature, engine - :tickets: 4089 - - The "password" attribute of the :class:`.url.URL` object can now be - any user-defined or user-subclassed string object that responds to the - Python ``str()`` builtin. The object passed will be maintained as the - datamember :attr:`.url.URL.password_original` and will be consulted - when the :attr:`.url.URL.password` attribute is read to produce the - string value. - - .. change:: - :tags: bug, orm - :tickets: 4130 - - Fixed bug in :func:`.contains_eager` query option where making use of a - path that used :meth:`.PropComparator.of_type` to refer to a subclass - across more than one level of joins would also require that the "alias" - argument were provided with the same subtype in order to avoid adding - unwanted FROM clauses to the query; additionally, using - :func:`.contains_eager` across subclasses that use :func:`.aliased` objects - of subclasses as the :meth:`.PropComparator.of_type` argument will also - render correctly. - - - - - .. change:: - :tags: feature, postgresql - - Added new :class:`_postgresql.MONEY` datatype. Pull request courtesy - Cleber J Santos. - - .. change:: - :tags: bug, sql - :tickets: 4140 - - Fixed bug in new "expanding bind parameter" feature whereby if multiple - params were used in one statement, the regular expression would not - match the parameter name correctly. - - .. change:: - :tags: enhancement, ext - :tickets: 4135 - - Added new method :meth:`.baked.Result.with_post_criteria` to baked - query system, allowing non-SQL-modifying transformations to take place - after the query has been pulled from the cache. Among other things, - this method can be used with :class:`.horizontal_shard.ShardedQuery` - to set the shard identifier. :class:`.horizontal_shard.ShardedQuery` - has also been modified such that its :meth:`.ShardedQuery.get` method - interacts correctly with that of :class:`_baked.Result`. - - .. change:: - :tags: bug, oracle - :tickets: 4064 - - Added some additional rules to fully handle ``Decimal('Infinity')``, - ``Decimal('-Infinity')`` values with cx_Oracle numerics when using - ``asdecimal=True``. - - .. change:: - :tags: bug, mssql - :tickets: 4121 - - Fixed bug where sqltypes.BINARY and sqltypes.VARBINARY datatypes - would not include correct bound-value handlers for pyodbc, - which allows the pyodbc.NullParam value to be passed that - helps with FreeTDS. - - - - - .. change:: - :tags: feature, misc - - Added a new errors section to the documentation with background - about common error messages. Selected exceptions within SQLAlchemy - will include a link in their string output to the relevant section - within this page. - - .. change:: - :tags: bug, orm - :tickets: 4032 - - The :meth:`_query.Query.exists` method will now disable eager loaders for when - the query is rendered. Previously, joined-eager load joins would be rendered - unnecessarily as well as subquery eager load queries would be needlessly - generated. The new behavior matches that of the :meth:`_query.Query.subquery` - method. - -.. changelog:: - :version: 1.2.0b3 - :released: December 27, 2017 - :released: October 13, 2017 - - .. change:: - :tags: feature, postgresql - :tickets: 4109 - - Added a new flag ``use_batch_mode`` to the psycopg2 dialect. This flag - enables the use of psycopg2's ``psycopg2.extras.execute_batch`` - extension when the :class:`_engine.Engine` calls upon - ``cursor.executemany()``. This extension provides a critical - performance increase by over an order of magnitude when running INSERT - statements in batch. The flag is False by default as it is considered - to be experimental for now. - - .. seealso:: - - :ref:`change_4109` - - .. change:: - :tags: bug, mssql - :tickets: 4061 - - SQL Server supports what SQLAlchemy calls "native boolean" - with its BIT type, as this type only accepts 0 or 1 and the - DBAPIs return its value as True/False. So the SQL Server - dialects now enable "native boolean" support, in that a - CHECK constraint is not generated for a :class:`.Boolean` - datatype. The only difference vs. other native boolean - is that there are no "true" / "false" constants so "1" and - "0" are still rendered here. - - - .. change:: - :tags: bug, oracle - :tickets: 4064 - - Partial support for persisting and retrieving the Oracle value - "infinity" is implemented with cx_Oracle, using Python float values - only, e.g. ``float("inf")``. Decimal support is not yet fulfilled by - the cx_Oracle DBAPI driver. - - .. change:: - :tags: bug, oracle - - The cx_Oracle dialect has been reworked and modernized to take advantage of - new patterns that weren't present in the old 4.x series of cx_Oracle. This - includes that the minimum cx_Oracle version is the 5.x series and that - cx_Oracle 6.x is now fully tested. The most significant change involves - type conversions, primarily regarding the numeric / floating point and LOB - datatypes, making more effective use of cx_Oracle type handling hooks to - simplify how bind parameter and result data is processed. - - .. seealso:: - - :ref:`change_cxoracle_12` - - .. change:: - :tags: bug, oracle - :tickets: 3997 - - two phase support for cx_Oracle has been completely removed for all - versions of cx_Oracle, whereas in 1.2.0b1 this change only took effect for - the 6.x series of cx_Oracle. This feature never worked correctly - in any version of cx_Oracle and in cx_Oracle 6.x, the API which SQLAlchemy - relied upon was removed. - - .. seealso:: - - :ref:`change_cxoracle_12` - - .. change:: - :tags: bug, oracle - - The column keys present in a result set when using :meth:`_expression.Insert.returning` - with the cx_Oracle backend now use the correct column / label names - like that of all other dialects. Previously, these came out as - ``ret_nnn``. - - .. seealso:: - - :ref:`change_cxoracle_12` - - .. change:: - :tags: bug, oracle - - Several parameters to the cx_Oracle dialect are now deprecated and will - have no effect: ``auto_setinputsizes``, ``exclude_setinputsizes``, - ``allow_twophase``. - - .. seealso:: - - :ref:`change_cxoracle_12` - - - .. change:: - :tags: bug, sql - :tickets: 4075 - - Added a new method :meth:`.DefaultExecutionContext.get_current_parameters` - which is used within a function-based default value generator in - order to retrieve the current parameters being passed to the statement. - The new function differs from the - :attr:`.DefaultExecutionContext.current_parameters` attribute in - that it also provides for optional grouping of parameters that - correspond to a multi-valued "insert" construct. Previously it was not - possible to identify the subset of parameters that were relevant to - the function call. - - .. seealso:: - - :ref:`change_4075` - - :ref:`context_default_functions` - - .. change:: - :tags: bug, orm - :tickets: 4050 - - Fixed regression introduced in 1.2.0b1 due to :ticket:`3934` where the - :class:`.Session` would fail to "deactivate" the transaction, if a - rollback failed (the target issue is when MySQL loses track of a SAVEPOINT). - This would cause a subsequent call to :meth:`.Session.rollback` to raise - an error a second time, rather than completing and bringing the - :class:`.Session` back to ACTIVE. - - .. change:: - :tags: bug, postgresql - :tickets: 4041 - - Fixed bug where the pg8000 driver would fail if using - :meth:`_schema.MetaData.reflect` with a schema name, since the schema name would - be sent as a "quoted_name" object that's a string subclass, which pg8000 - doesn't recognize. The quoted_name type is added to pg8000's - py_types collection on connect. - - .. change:: - :tags: bug, postgresql - :tickets: 4016 - - Enabled UUID support for the pg8000 driver, which supports native Python - uuid round trips for this datatype. Arrays of UUID are still not supported, - however. - - .. change:: - :tags: mssql, bug - :tickets: 4057 - - Fixed the pymssql dialect so that percent signs in SQL text, such - as used in modulus expressions or literal textual values, are - **not** doubled up, as seems to be what pymssql expects. This is - despite the fact that the pymssql DBAPI uses the "pyformat" parameter - style which itself considers the percent sign to be significant. - - .. change:: - :tags: bug, orm, declarative - :tickets: 4091 - - A warning is emitted if a subclass attempts to override an attribute - that was declared on a superclass using ``@declared_attr.cascading`` - that the overridden attribute will be ignored. This use - case cannot be fully supported down to further subclasses without more - complex development efforts, so for consistency the "cascading" is - honored all the way down regardless of overriding attributes. - - .. change:: - :tags: bug, orm, declarative - :tickets: 4092 - - A warning is emitted if the ``@declared_attr.cascading`` attribute is - used with a special declarative name such as ``__tablename__``, as this - has no effect. - - .. change:: - :tags: feature, engine - :tickets: 4077 - - Added ``__next__()`` and ``next()`` methods to :class:`_engine.ResultProxy`, - so that the ``next()`` builtin function works on the object directly. - :class:`_engine.ResultProxy` has long had an ``__iter__()`` method which already - allows it to respond to the ``iter()`` builtin. The implementation - for ``__iter__()`` is unchanged, as performance testing has indicated - that iteration using a ``__next__()`` method with ``StopIteration`` - is about 20% slower in both Python 2.7 and 3.6. - - .. change:: - :tags: feature, mssql - :tickets: 4086 - - Added a new :class:`_mssql.TIMESTAMP` datatype, that - correctly acts like a binary datatype for SQL Server - rather than a datetime type, as SQL Server breaks the - SQL standard here. Also added :class:`_mssql.ROWVERSION`, - as the "TIMESTAMP" type in SQL Server is deprecated in - favor of ROWVERSION. - - .. change:: - :tags: bug, orm - :tickets: 4084 - - Fixed issue where the :func:`.make_transient_to_detached` function - would expire all attributes on the target object, including "deferred" - attributes, which has the effect of the attribute being undeferred - for the next refresh, causing an unexpected load of the attribute. - - .. change:: - :tags: bug, orm - :tickets: 4040 - - Fixed bug involving delete-orphan cascade where a related item - that becomes an orphan before the parent object is part of a - session is still tracked as moving into orphan status, which results - in it being expunged from the session rather than being flushed. - - .. note:: This fix was inadvertently merged during the 1.2.0b3 - release and was **not added to the changelog** at that time. - This changelog note was added to the release retroactively as of - version 1.2.13. - - .. change:: - :tags: bug, orm - :tickets: 4026 - - Fixed bug in :ref:`change_3948` which prevented "selectin" and - "inline" settings in a multi-level class hierarchy from interacting - together as expected. - - .. change:: - :tags: bug, oracle - :tickets: 4042 - - Fixed bug where an index reflected under Oracle with an expression like - "column DESC" would not be returned, if the table also had no primary - key, as a result of logic that attempts to filter out the - index implicitly added by Oracle onto the primary key columns. - - .. change:: - :tags: bug, orm - :tickets: 4071 - - Removed the warnings that are emitted when the LRU caches employed - by the mapper as well as loader strategies reach their threshold; the - purpose of this warning was at first a guard against excess cache keys - being generated but became basically a check on the "creating many - engines" antipattern. While this is still an antipattern, the presence - of test suites which both create an engine per test as well as raise - on all warnings will be an inconvenience; it should not be critical - that such test suites change their architecture just for this warning - (though engine-per-test suite is always better). - - .. change:: - :tags: bug, orm - :tickets: 4049 - - Fixed regression where the use of a :func:`.undefer_group` option - in conjunction with a lazy loaded relationship option would cause - an attribute error, due to a bug in the SQL cache key generation - added in 1.2 as part of :ticket:`3954`. - - .. change:: - :tags: bug, oracle - :tickets: 4045 - - Fixed more regressions caused by cx_Oracle 6.0; at the moment, the only - behavioral change for users is disconnect detection now detects for - cx_Oracle.DatabaseError in addition to cx_Oracle.InterfaceError, as - this behavior seems to have changed. Other issues regarding numeric - precision and uncloseable connections are pending with the upstream - cx_Oracle issue tracker. - - .. change:: - :tags: bug, mssql - :tickets: 4060 - - Fixed bug where the SQL Server dialect could pull columns from multiple - schemas when reflecting a self-referential foreign key constraint, if - multiple schemas contained a constraint of the same name against a - table of the same name. - - - .. change:: - :tags: feature, mssql - :tickets: 4058 - - Added support for "AUTOCOMMIT" isolation level, as established - via :meth:`_engine.Connection.execution_options`, to the - PyODBC and pymssql dialects. This isolation level sets the - appropriate DBAPI-specific flags on the underlying - connection object. - - .. change:: - :tags: bug, orm - :tickets: 4073 - - Modified the change made to the ORM update/delete evaluator in - :ticket:`3366` such that if an unmapped column expression is present - in the update or delete, if the evaluator can match its name to the - mapped columns of the target class, a warning is emitted, rather than - raising UnevaluatableError. This is essentially the pre-1.2 behavior, - and is to allow migration for applications that are currently relying - upon this pattern. However, if the given attribute name cannot be - matched to the columns of the mapper, the UnevaluatableError is - still raised, which is what was fixed in :ticket:`3366`. - - .. change:: - :tags: bug, sql - :tickets: 4087 - - Fixed bug in new SQL comments feature where table and column comment - would not be copied when using :meth:`_schema.Table.tometadata`. - - .. change:: - :tags: bug, sql - :tickets: 4102 - - In release 1.1, the :class:`.Boolean` type was broken in that - boolean coercion via ``bool()`` would occur for backends that did not - feature "native boolean", but would not occur for native boolean backends, - meaning the string ``"0"`` now behaved inconsistently. After a poll, a - consensus was reached that non-boolean values should be raising an error, - especially in the ambiguous case of string ``"0"``; so the :class:`.Boolean` - datatype will now raise ``ValueError`` if an incoming value is not - within the range ``None, True, False, 1, 0``. - - .. seealso:: - - :ref:`change_4102` - - .. change:: - :tags: bug, sql - :tickets: 4063 - - Refined the behavior of :meth:`.Operators.op` such that in all cases, - if the :paramref:`.Operators.op.is_comparison` flag is set to True, - the return type of the resulting expression will be - :class:`.Boolean`, and if the flag is False, the return type of the - resulting expression will be the same type as that of the left-hand - expression, which is the typical default behavior of other operators. - Also added a new parameter :paramref:`.Operators.op.return_type` as well - as a helper method :meth:`.Operators.bool_op`. - - .. seealso:: - - :ref:`change_4063` - - .. change:: - :tags: bug, mysql - :tickets: 4072 - - Changed the name of the ``.values`` attribute of the new MySQL - INSERT..ON DUPLICATE KEY UPDATE construct to ``.inserted``, as - :class:`_expression.Insert` already has a method called :meth:`_expression.Insert.values`. - The ``.inserted`` attribute ultimately renders the MySQL ``VALUES()`` - function. - - .. change:: - :tags: bug, mssql, orm - :tickets: 4062 - - Added a new class of "rowcount support" for dialects that is specific to - when "RETURNING", which on SQL Server looks like "OUTPUT inserted", is in - use, as the PyODBC backend isn't able to give us rowcount on an UPDATE or - DELETE statement when OUTPUT is in effect. This primarily affects the ORM - when a flush is updating a row that contains server-calculated values, - raising an error if the backend does not return the expected row count. - PyODBC now states that it supports rowcount except if OUTPUT.inserted is - present, which is taken into account by the ORM during a flush as to - whether it will look for a rowcount. - - .. change:: - :tags: bug, sql - :tickets: 4088 - - Internal refinements to the :class:`.Enum`, :class:`.Interval`, and - :class:`.Boolean` types, which now extend a common mixin - :class:`.Emulated` that indicates a type that provides Python-side - emulation of a DB native type, switching out to the DB native type when - a supporting backend is in use. The PostgreSQL - :class:`_postgresql.INTERVAL` type when used directly will now include - the correct type coercion rules for SQL expressions that also take - effect for :class:`_types.Interval` (such as adding a date to an - interval yields a datetime). - - - .. change:: - :tags: bug, mssql, orm - - Enabled the "sane_rowcount" flag for the pymssql dialect, indicating - that the DBAPI now reports the correct number of rows affected from - an UPDATE or DELETE statement. This impacts mostly the ORM versioning - feature in that it now can verify the number of rows affected on a - target version. - - .. change:: 4028 - :tags: bug, engine - :tickets: 4028 - - Made some adjustments to :class:`_pool.Pool` and :class:`_engine.Connection` such - that recovery logic is not run underneath exception catches for - ``pool.Empty``, ``AttributeError``, since when the recovery operation - itself fails, Python 3 creates a misleading stack trace referring to the - ``Empty`` / ``AttributeError`` as the cause, when in fact these exception - catches are part of control flow. - - - .. change:: - :tags: bug, oracle - :tickets: 4076 - - Fixed bug where Oracle 8 "non ansi" join mode would not add the - ``(+)`` operator to expressions that used an operator other than the - ``=`` operator. The ``(+)`` needs to be on all columns that are part - of the right-hand side. - - .. change:: - :tags: bug, mssql - :tickets: 4059 - - Added a rule to SQL Server index reflection to ignore the so-called - "heap" index that is implicitly present on a table that does not - specify a clustered index. - - -.. changelog:: - :version: 1.2.0b2 - :released: December 27, 2017 - :released: July 24, 2017 - - .. change:: 4033 - :tags: bug, orm - :tickets: 4033 - - Fixed regression from 1.1.11 where adding additional non-entity - columns to a query that includes an entity with subqueryload - relationships would fail, due to an inspection added in 1.1.11 as a - result of :ticket:`4011`. - - -.. changelog:: - :version: 1.2.0b1 - :released: December 27, 2017 - :released: July 10, 2017 - - .. change:: scoped_autocommit - :tags: feature, orm - - Added ``.autocommit`` attribute to :class:`.scoped_session`, proxying - the ``.autocommit`` attribute of the underling :class:`.Session` - currently assigned to the thread. Pull request courtesy - Ben Fagin. - - .. change:: 4009 - :tags: feature, mysql - :tickets: 4009 - - Added support for MySQL's ON DUPLICATE KEY UPDATE - MySQL-specific :class:`.mysql.dml.Insert` object. - Pull request courtesy Michael Doronin. - - .. seealso:: - - :ref:`change_4009` - - .. change:: 4018 - :tags: bug, sql - :tickets: 4018 - - The rules for type coercion between :class:`.Numeric`, :class:`.Integer`, - and date-related types now include additional logic that will attempt - to preserve the settings of the incoming type on the "resolved" type. - Currently the target for this is the ``asdecimal`` flag, so that - a math operation between :class:`.Numeric` or :class:`.Float` and - :class:`.Integer` will preserve the "asdecimal" flag as well as - if the type should be the :class:`.Float` subclass. - - .. seealso:: - - :ref:`change_floats_12` - - .. change:: 4020 - :tags: bug, sql, mysql - :tickets: 4020 - - The result processor for the :class:`.Float` type now unconditionally - runs values through the ``float()`` processor if the dialect - specifies that it also supports "native decimal" mode. While most - backends will deliver Python ``float`` objects for a floating point - datatype, the MySQL backends in some cases lack the typing information - in order to provide this and return ``Decimal`` unless the float - conversion is done. - - .. seealso:: - - :ref:`change_floats_12` - - .. change:: 4017 - :tags: bug, sql - :tickets: 4017 - - Added some extra strictness to the handling of Python "float" values - passed to SQL statements. A "float" value will be associated with the - :class:`.Float` datatype and not the Decimal-coercing :class:`.Numeric` - datatype as was the case before, eliminating a confusing warning - emitted on SQLite as well as unnecessary coercion to Decimal. - - .. seealso:: - - :ref:`change_floats_12` - - .. change:: 3058 - :tags: feature, orm - :tickets: 3058 - - Added a new feature :func:`_orm.with_expression` that allows an ad-hoc - SQL expression to be added to a specific entity in a query at result - time. This is an alternative to the SQL expression being delivered as - a separate element in the result tuple. - - .. seealso:: - - :ref:`change_3058` - - .. change:: 3496 - :tags: bug, orm - :tickets: 3496 - - An UPDATE emitted as a result of the - :paramref:`_orm.relationship.post_update` feature will now integrate with - the versioning feature to both bump the version id of the row as well - as assert that the existing version number was matched. - - .. seealso:: - - :ref:`change_3496` - - .. change:: 3769 - :tags: bug, ext - :tickets: 3769 - - The :meth:`.AssociationProxy.any`, :meth:`.AssociationProxy.has` - and :meth:`.AssociationProxy.contains` comparison methods now support - linkage to an attribute that is itself also an - :class:`.AssociationProxy`, recursively. - - .. seealso:: - - :ref:`change_3769` - - .. change:: 3853 - :tags: bug, ext - :tickets: 3853 - - Implemented in-place mutation operators ``__ior__``, ``__iand__``, - ``__ixor__`` and ``__isub__`` for :class:`.mutable.MutableSet` - and ``__iadd__`` for :class:`.mutable.MutableList` so that change - events are fired off when these mutator methods are used to alter the - collection. - - .. seealso:: - - :ref:`change_3853` - - .. change:: 3847 - :tags: bug, declarative - :tickets: 3847 - - A warning is emitted if the :attr:`.declared_attr.cascading` modifier - is used with a declarative attribute that is itself declared on - a class that is to be mapped, as opposed to a declarative mixin - class or ``__abstract__`` class. The :attr:`.declared_attr.cascading` - modifier currently only applies to mixin/abstract classes. - - .. change:: 4003 - :tags: feature, oracle - :tickets: 4003 - - The Oracle dialect now inspects unique and check constraints when using - :meth:`_reflection.Inspector.get_unique_constraints`, - :meth:`_reflection.Inspector.get_check_constraints`. - As Oracle does not have unique constraints that are separate from a unique - :class:`.Index`, a :class:`_schema.Table` that's reflected will still continue - to not have :class:`.UniqueConstraint` objects associated with it. - Pull requests courtesy Eloy Felix. - - .. seealso:: - - :ref:`change_4003` - - .. change:: 3948 - :tags: feature, orm - :tickets: 3948 - - Added a new style of mapper-level inheritance loading - "polymorphic selectin". This style of loading - emits queries for each subclass in an inheritance - hierarchy subsequent to the load of the base - object type, using IN to specify the desired - primary key values. - - .. seealso:: - - :ref:`change_3948` - - .. change:: 3472 - :tags: bug, orm - :tickets: 3471, 3472 - - Repaired several use cases involving the - :paramref:`_orm.relationship.post_update` feature when used in conjunction - with a column that has an "onupdate" value. When the UPDATE emits, - the corresponding object attribute is now expired or refreshed so that - the newly generated "onupdate" value can populate on the object; - previously the stale value would remain. Additionally, if the target - attribute is set in Python for the INSERT of the object, the value is - now re-sent during the UPDATE so that the "onupdate" does not overwrite - it (note this works just as well for server-generated onupdates). - Finally, the :meth:`.SessionEvents.refresh_flush` event is now emitted - for these attributes when refreshed within the flush. - - .. seealso:: - - :ref:`change_3471` - - .. change:: 3996 - :tags: bug, orm - :tickets: 3996 - - Fixed bug where programmatic version_id counter in conjunction with - joined table inheritance would fail if the version_id counter - were not actually incremented and no other values on the base table - were modified, as the UPDATE would have an empty SET clause. Since - programmatic version_id where version counter is not incremented - is a documented use case, this specific condition is now detected - and the UPDATE now sets the version_id value to itself, so that - concurrency checks still take place. - - .. change:: 3848 - :tags: bug, orm, declarative - :tickets: 3848 - - Fixed bug where using :class:`.declared_attr` on an - :class:`.AbstractConcreteBase` where a particular return value were some - non-mapped symbol, including ``None``, would cause the attribute - to hard-evaluate just once and store the value to the object - dictionary, not allowing it to invoke for subclasses. This behavior - is normal when :class:`.declared_attr` is on a mapped class, and - does not occur on a mixin or abstract class. Since - :class:`.AbstractConcreteBase` is both "abstract" and actually - "mapped", a special exception case is made here so that the - "abstract" behavior takes precedence for :class:`.declared_attr`. - - .. change:: 3673 - :tags: bug, orm - :tickets: 3673 - - The versioning feature does not support NULL for the version counter. - An exception is now raised if the version id is programmatic and - was set to NULL for an UPDATE. Pull request courtesy Diana Clarke. - - .. change:: 3999 - :tags: bug, sql - :tickets: 3999 - - The operator precedence for all comparison operators such as LIKE, IS, - IN, MATCH, equals, greater than, less than, etc. has all been merged - into one level, so that expressions which make use of these against - each other will produce parentheses between them. This suits the - stated operator precedence of databases like Oracle, MySQL and others - which place all of these operators as equal precedence, as well as - PostgreSQL as of 9.5 which has also flattened its operator precedence. - - .. seealso:: - - :ref:`change_3999` - - - .. change:: 3796 - :tags: bug, orm - :tickets: 3796 - - Removed a very old keyword argument from :class:`.scoped_session` - called ``scope``. This keyword was never documented and was an - early attempt at allowing for variable scopes. - - .. seealso:: - - :ref:`change_3796` - - .. change:: 3871 - :tags: bug, mysql - :tickets: 3871 - - Added support for views that are unreflectable due to stale - table definitions, when calling :meth:`_schema.MetaData.reflect`; a warning - is emitted for the table that cannot respond to ``DESCRIBE``, - but the operation succeeds. - - .. change:: baked_opts - :tags: feature, ext - - Added new flag :paramref:`.Session.enable_baked_queries` to the - :class:`.Session` to allow baked queries to be disabled - session-wide, reducing memory use. Also added new :class:`.Bakery` - wrapper so that the bakery returned by :paramref:`.BakedQuery.bakery` - can be inspected. - - .. change:: 3988 - :tags: bug, orm - :tickets: 3988 - - Fixed bug where combining a "with_polymorphic" load in conjunction - with subclass-linked relationships that specify joinedload with - innerjoin=True, would fail to demote those "innerjoins" to - "outerjoins" to suit the other polymorphic classes that don't - support that relationship. This applies to both a single and a - joined inheritance polymorphic load. - - .. change:: 3991 - :tags: bug, orm - :tickets: 3991 - - Added new argument :paramref:`.with_for_update` to the - :meth:`.Session.refresh` method. When the :meth:`_query.Query.with_lockmode` - method were deprecated in favor of :meth:`_query.Query.with_for_update`, - the :meth:`.Session.refresh` method was never updated to reflect - the new option. - - .. seealso:: - - :ref:`change_3991` - - .. change:: 3984 - :tags: bug, orm - :tickets: 3984 - - Fixed bug where a :func:`.column_property` that is also marked as - "deferred" would be marked as "expired" during a flush, causing it - to be loaded along with the unexpiry of regular attributes even - though this attribute was never accessed. - - .. change:: 3873 - :tags: bug, sql - :tickets: 3873 - - Repaired issue where the type of an expression that used - :meth:`.ColumnOperators.is_` or similar would not be a "boolean" type, - instead the type would be "nulltype", as well as when using custom - comparison operators against an untyped expression. This typing can - impact how the expression behaves in larger contexts as well as - in result-row-handling. - - .. change:: 3941 - :tags: bug, ext - :tickets: 3941 - - Improved the association proxy list collection so that premature - autoflush against a newly created association object can be prevented - in the case where ``list.append()`` is being used, and a lazy load - would be invoked when the association proxy accesses the endpoint - collection. The endpoint collection is now accessed first before - the creator is invoked to produce the association object. - - .. change:: 3969 - :tags: bug, sql - :tickets: 3969 - - Fixed the negation of a :class:`.Label` construct so that the - inner element is negated correctly, when the :func:`.not_` modifier - is applied to the labeled expression. - - .. change:: 3944 - :tags: feature, orm - :tickets: 3944 - - Added a new kind of eager loading called "selectin" loading. This - style of loading is very similar to "subquery" eager loading, - except that it uses an IN expression given a list of primary key - values from the loaded parent objects, rather than re-stating the - original query. This produces a more efficient query that is - "baked" (e.g. the SQL string is cached) and also works in the - context of :meth:`_query.Query.yield_per`. - - .. seealso:: - - :ref:`change_3944` - - .. change:: - :tags: bug, orm - :tickets: 3967 - - Fixed bug in subquery eager loading where the "join_depth" parameter - for self-referential relationships would not be correctly honored, - loading all available levels deep rather than correctly counting - the specified number of levels for eager loading. - - .. change:: - :tags: bug, orm - - Added warnings to the LRU "compiled cache" used by the :class:`_orm.Mapper` - (and ultimately will be for other ORM-based LRU caches) such that - when the cache starts hitting its size limits, the application will - emit a warning that this is a performance-degrading situation that - may require attention. The LRU caches can reach their size limits - primarily if an application is making use of an unbounded number - of :class:`_engine.Engine` objects, which is an antipattern. Otherwise, - this may suggest an issue that should be brought to the SQLAlchemy - developer's attention. - - .. change:: 3964 - :tags: bug, postgresql - :tickets: 3964 - - Fixed bug where the base :class:`_types.ARRAY` datatype would not - invoke the bind/result processors of :class:`_postgresql.ARRAY`. - - .. change:: 3963 - :tags: bug, orm - :tickets: 3963 - - Fixed bug to improve upon the specificity of loader options that - take effect subsequent to the lazy load of a related entity, so - that the loader options will match to an aliased or non-aliased - entity more specifically if those options include entity information. - - .. change:: 3954 - :tags: feature, orm - :tickets: 3954 - - The ``lazy="select"`` loader strategy now makes used of the - :class:`.BakedQuery` query caching system in all cases. This - removes most overhead of generating a :class:`_query.Query` object and - running it into a :func:`_expression.select` and then string SQL statement from - the process of lazy-loading related collections and objects. The - "baked" lazy loader has also been improved such that it can now - cache in most cases where query load options are used. - - .. seealso:: - - :ref:`change_3954` - - .. change:: 3740 - :tags: bug, sql - :tickets: 3740 - - The system by which percent signs in SQL statements are "doubled" - for escaping purposes has been refined. The "doubling" of percent - signs mostly associated with the :obj:`_expression.literal_column` construct - as well as operators like :meth:`.ColumnOperators.contains` now - occurs based on the stated paramstyle of the DBAPI in use; for - percent-sensitive paramstyles as are common with the PostgreSQL - and MySQL drivers the doubling will occur, for others like that - of SQLite it will not. This allows more database-agnostic use - of the :obj:`_expression.literal_column` construct to be possible. - - .. seealso:: - - :ref:`change_3740` - - .. change:: 3959 - :tags: bug, postgresql - :tickets: 3959 - - Added support for all possible "fields" identifiers when reflecting the - PostgreSQL ``INTERVAL`` datatype, e.g. "YEAR", "MONTH", "DAY TO - MINUTE", etc.. In addition, the :class:`_postgresql.INTERVAL` - datatype itself now includes a new parameter - :paramref:`.postgresql.INTERVAL.fields` where these qualifiers can be - specified; the qualifier is also reflected back into the resulting - datatype upon reflection / inspection. - - .. seealso:: - - :ref:`change_3959` - - .. change:: 3957 - :tags: bug, sql - :tickets: 3957 - - Fixed bug where a column-level :class:`.CheckConstraint` would fail - to compile the SQL expression using the underlying dialect compiler - as well as apply proper flags to generate literal values as - inline, in the case that the sqltext is a Core expression and - not just a plain string. This was long-ago fixed for table-level - check constraints in 0.9 as part of :ticket:`2742`, which more commonly - feature Core SQL expressions as opposed to plain string expressions. - - .. change:: 2626 - :tags: bug, mssql - :tickets: 2626 - - The SQL Server dialect now allows for a database and/or owner name - with a dot inside of it, using brackets explicitly in the string around - the owner and optionally the database name as well. In addition, - sending the :class:`.quoted_name` construct for the schema name will - not split on the dot and will deliver the full string as the "owner". - :class:`.quoted_name` is also now available from the ``sqlalchemy.sql`` - import space. - - .. seealso:: - - :ref:`change_2626` - - .. change:: 3953 - :tags: feature, sql - :tickets: 3953 - - Added a new kind of :func:`.bindparam` called "expanding". This is - for use in ``IN`` expressions where the list of elements is rendered - into individual bound parameters at statement execution time, rather - than at statement compilation time. This allows both a single bound - parameter name to be linked to an IN expression of multiple elements, - as well as allows query caching to be used with IN expressions. The - new feature allows the related features of "select in" loading and - "polymorphic in" loading to make use of the baked query extension - to reduce call overhead. This feature should be considered to be - **experimental** for 1.2. - - .. seealso:: - - :ref:`change_3953` - - .. change:: 3923 - :tags: bug, sql - :tickets: 3923 - - Fixed bug where a SQL-oriented Python-side column default could fail to - be executed properly upon INSERT in the "pre-execute" codepath, if the - SQL itself were an untyped expression, such as plain text. The "pre- - execute" codepath is fairly uncommon however can apply to non-integer - primary key columns with SQL defaults when RETURNING is not used. - - .. change:: 3785 - :tags: bug, sql - :tickets: 3785 - - The expression used for COLLATE as rendered by the column-level - :func:`_expression.collate` and :meth:`.ColumnOperators.collate` is now - quoted as an identifier when the name is case sensitive, e.g. has - uppercase characters. Note that this does not impact type-level - collation, which is already quoted. - - .. seealso:: - - :ref:`change_3785` - - .. change:: 3229 - :tags: feature, orm, ext - :tickets: 3229 - - The :meth:`_query.Query.update` method can now accommodate both - hybrid attributes as well as composite attributes as a source - of the key to be placed in the SET clause. For hybrids, an - additional decorator :meth:`.hybrid_property.update_expression` - is supplied for which the user supplies a tuple-returning function. - - .. seealso:: - - :ref:`change_3229` - - .. change:: 3753 - :tags: bug, orm - :tickets: 3753 - - The :func:`.attributes.flag_modified` function now raises - :class:`.InvalidRequestError` if the named attribute key is not - present within the object, as this is assumed to be present - in the flush process. To mark an object "dirty" for a flush - without referring to any specific attribute, the - :func:`.attributes.flag_dirty` function may be used. - - .. seealso:: - - :ref:`change_3753` - - .. change:: 3911_3912 - :tags: bug, ext - :tickets: 3911, 3912 - - The :class:`sqlalchemy.ext.hybrid.hybrid_property` class now supports - calling mutators like ``@setter``, ``@expression`` etc. multiple times - across subclasses, and now provides a ``@getter`` mutator, so that - a particular hybrid can be repurposed across subclasses or other - classes. This now matches the behavior of ``@property`` in standard - Python. - - .. seealso:: - - :ref:`change_3911_3912` - - - - .. change:: 1546 - :tags: feature, sql, postgresql, mysql, oracle - :tickets: 1546 - - Added support for SQL comments on :class:`_schema.Table` and :class:`_schema.Column` - objects, via the new :paramref:`_schema.Table.comment` and - :paramref:`_schema.Column.comment` arguments. The comments are included - as part of DDL on table creation, either inline or via an appropriate - ALTER statement, and are also reflected back within table reflection, - as well as via the :class:`_reflection.Inspector`. Supported backends currently - include MySQL, PostgreSQL, and Oracle. Many thanks to Frazer McLean - for a large amount of effort on this. - - .. seealso:: - - :ref:`change_1546` - - .. change:: 3919 - :tags: feature, engine - :tickets: 3919 - - Added native "pessimistic disconnection" handling to the :class:`_pool.Pool` - object. The new parameter :paramref:`_pool.Pool.pre_ping`, available from - the engine as :paramref:`_sa.create_engine.pool_pre_ping`, applies an - efficient form of the "pre-ping" recipe featured in the pooling - documentation, which upon each connection check out, emits a simple - statement, typically "SELECT 1", to test the connection for liveness. - If the existing connection is no longer able to respond to commands, - the connection is transparently recycled, and all other connections - made prior to the current timestamp are invalidated. - - .. seealso:: - - :ref:`pool_disconnects_pessimistic` - - :ref:`change_3919` - - .. change:: 3939 - :tags: bug, sql - :tickets: 3939 - - Fixed bug where the use of an :class:`_expression.Alias` object in a column - context would raise an argument error when it tried to group itself - into a parenthesized expression. Using :class:`_expression.Alias` in this way - is not yet a fully supported API, however it applies to some end-user - recipes and may have a more prominent role in support of some - future PostgreSQL features. - - .. change:: 3366 - :tags: bug, orm - :tickets: 3366 - - The "evaluate" strategy used by :meth:`_query.Query.update` and - :meth:`_query.Query.delete` can now accommodate a simple - object comparison from a many-to-one relationship to an instance, - when the attribute names of the primary key / foreign key columns - don't match the actual names of the columns. Previously this would - do a simple name-based match and fail with an AttributeError. - - .. change:: 3896_a - :tags: feature, orm - :tickets: 3896 - - Added new attribute event :meth:`.AttributeEvents.bulk_replace`. - This event is triggered when a collection is assigned to a - relationship, before the incoming collection is compared with the - existing one. This early event allows for conversion of incoming - non-ORM objects as well. The event is integrated with the - ``@validates`` decorator. - - .. seealso:: - - :ref:`change_3896_event` - - .. change:: 3896_b - :tags: bug, orm - :tickets: 3896 - - The ``@validates`` decorator now allows the decorated method to receive - objects from a "bulk collection set" operation that have not yet - been compared to the existing collection. This allows incoming values - to be converted to compatible ORM objects as is already allowed - from an "append" event. Note that this means that the - ``@validates`` method is called for **all** values during a collection - assignment, rather than just the ones that are new. - - .. seealso:: - - :ref:`change_3896_validates` - - .. change:: 3938 - :tags: bug, engine - :tickets: 3938 - - Fixed bug where in the unusual case of passing a - :class:`.Compiled` object directly to :meth:`_engine.Connection.execute`, - the dialect with which the :class:`.Compiled` object were generated - was not consulted for the paramstyle of the string statement, instead - assuming it would match the dialect-level paramstyle, causing - mismatches to occur. - - .. change:: 3303 - :tags: feature, orm - :tickets: 3303 - - Added new event handler :meth:`.AttributeEvents.modified` which is - triggered when the func:`.attributes.flag_modified` function is - invoked, which is common when using the :mod:`sqlalchemy.ext.mutable` - extension module. - - .. seealso:: - - :ref:`change_3303` - - .. change:: 3918 - :tags: bug, ext - :tickets: 3918 - - Fixed a bug in the ``sqlalchemy.ext.serializer`` extension whereby - an "annotated" SQL element (as produced by the ORM for many types - of SQL expressions) could not be reliably serialized. Also bumped - the default pickle level for the serializer to "HIGHEST_PROTOCOL". - - .. change:: 3891 - :tags: bug, orm - :tickets: 3891 - - Fixed bug in single-table inheritance where the select_from() - argument would not be taken into account when limiting rows - to a subclass. Previously, only expressions in the - columns requested would be taken into account. - - .. seealso:: - - :ref:`change_3891` - - .. change:: 3913 - :tags: bug, orm - :tickets: 3913 - - When assigning a collection to an attribute mapped by a relationship, - the previous collection is no longer mutated. Previously, the old - collection would be emptied out in conjunction with the "item remove" - events that fire off; the events now fire off without affecting - the old collection. - - .. seealso:: - - :ref:`change_3913` - - .. change:: 3932 - :tags: bug, oracle - :tickets: 3932 - - The cx_Oracle dialect now supports "sane multi rowcount", that is, - when a series of parameter sets are executed via DBAPI - ``cursor.executemany()``, we can make use of ``cursor.rowcount`` to - verify the number of rows matched. This has an impact within the - ORM when detecting concurrent modification scenarios, in that - some simple conditions can now be detected even when the ORM - is batching statements, as well as when the more strict versioning - feature is used, the ORM can still use statement batching. The - flag is enabled for cx_Oracle assuming at least version 5.0, which - is now commonplace. - - .. change:: 3907 - :tags: feature, sql - :tickets: 3907 - - The longstanding behavior of the :meth:`.ColumnOperators.in_` and - :meth:`.ColumnOperators.notin_` operators emitting a warning when - the right-hand condition is an empty sequence has been revised; - a simple "static" expression of "1 != 1" or "1 = 1" is now rendered - by default, rather than pulling in the original left-hand - expression. This causes the result for a NULL column comparison - against an empty set to change from NULL to true/false. The - behavior is configurable, and the old behavior can be enabled - using the :paramref:`_sa.create_engine.empty_in_strategy` parameter - to :func:`_sa.create_engine`. - - .. seealso:: - - :ref:`change_3907` - - .. change:: 3276 - :tags: bug, oracle - :tickets: 3276 - - Oracle reflection now "normalizes" the name given to a foreign key - constraint, that is, returns it as all lower case for a case - insensitive name. This was already the behavior for indexes - and primary key constraints as well as all table and column names. - This will allow Alembic autogenerate scripts to compare and render - foreign key constraint names correctly when initially specified - as case insensitive. - - .. seealso:: - - :ref:`change_3276` - - .. change:: 2694 - :tags: feature, sql - :tickets: 2694 - - Added a new option ``autoescape`` to the "startswith" and - "endswith" classes of comparators; this supplies an escape character - also applies it to all occurrences of the wildcard characters "%" - and "_" automatically. Pull request courtesy Diana Clarke. - - .. note:: This feature has been changed as of 1.2.0 from its initial - implementation in 1.2.0b2 such that autoescape is now passed as a - boolean value, rather than a specific character to use as the escape - character. - - .. seealso:: - - :ref:`change_2694` - - .. change:: 3934 - :tags: bug, orm - :tickets: 3934 - - The state of the :class:`.Session` is now present when the - :meth:`.SessionEvents.after_rollback` event is emitted, that is, the - attribute state of objects prior to their being expired. This is now - consistent with the behavior of the - :meth:`.SessionEvents.after_commit` event which also emits before the - attribute state of objects is expired. - - .. seealso:: - - :ref:`change_3934` - - .. change:: 3607 - :tags: bug, orm - :tickets: 3607 - - Fixed bug where :meth:`_query.Query.with_parent` would not work if the - :class:`_query.Query` were against an :func:`.aliased` construct rather than - a regular mapped class. Also adds a new parameter - :paramref:`.util.with_parent.from_entity` to the standalone - :func:`.util.with_parent` function as well as - :meth:`_query.Query.with_parent`. diff --git a/doc/build/changelog/changelog_13.rst b/doc/build/changelog/changelog_13.rst deleted file mode 100644 index 60394996921..00000000000 --- a/doc/build/changelog/changelog_13.rst +++ /dev/null @@ -1,4010 +0,0 @@ -============= -1.3 Changelog -============= - -.. changelog_imports:: - - .. include:: changelog_12.rst - :start-line: 5 - - .. include:: changelog_11.rst - :start-line: 5 - -.. changelog:: - :version: 1.3.25 - :include_notes_from: unreleased_13 - -.. changelog:: - :version: 1.3.24 - :released: March 30, 2021 - - .. change:: - :tags: bug, schema - :tickets: 6152 - - Fixed bug first introduced in as some combination of :ticket:`2892`, - :ticket:`2919` nnd :ticket:`3832` where the attachment events for a - :class:`_types.TypeDecorator` would be doubled up against the "impl" class, - if the "impl" were also a :class:`_types.SchemaType`. The real-world case - is any :class:`_types.TypeDecorator` against :class:`_types.Enum` or - :class:`_types.Boolean` would get a doubled - :class:`_schema.CheckConstraint` when the ``create_constraint=True`` flag - is set. - - - .. change:: - :tags: bug, schema, sqlite - :tickets: 6007 - :versions: 1.4.0 - - Fixed issue where the CHECK constraint generated by :class:`_types.Boolean` - or :class:`_types.Enum` would fail to render the naming convention - correctly after the first compilation, due to an unintended change of state - within the name given to the constraint. This issue was first introduced in - 0.9 in the fix for issue #3067, and the fix revises the approach taken at - that time which appears to have been more involved than what was needed. - - .. change:: - :tags: bug, orm - :tickets: 5983 - :versions: 1.4.0 - - Removed very old warning that states that passive_deletes is not intended - for many-to-one relationships. While it is likely that in many cases - placing this parameter on a many-to-one relationship is not what was - intended, there are use cases where delete cascade may want to be - disallowed following from such a relationship. - - - - .. change:: - :tags: bug, postgresql - :tickets: 5989 - :versions: 1.4.0 - - Fixed issue where using :class:`_postgresql.aggregate_order_by` would - return ARRAY(NullType) under certain conditions, interfering with - the ability of the result object to return data correctly. - - .. change:: - :tags: bug, schema - :tickets: 5919 - :versions: 1.4.0 - - Repaired / implemented support for primary key constraint naming - conventions that use column names/keys/etc as part of the convention. In - particular, this includes that the :class:`.PrimaryKeyConstraint` object - that's automatically associated with a :class:`.schema.Table` will update - its name as new primary key :class:`_schema.Column` objects are added to - the table and then to the constraint. Internal failure modes related to - this constraint construction process including no columns present, no name - present or blank name present are now accommodated. - - .. change:: - :tags: bug, schema - :tickets: 6071 - :versions: 1.4.3 - - Adjusted the logic that emits DROP statements for :class:`_schema.Sequence` - objects among the dropping of multiple tables, such that all - :class:`_schema.Sequence` objects are dropped after all tables, even if the - given :class:`_schema.Sequence` is related only to a :class:`_schema.Table` - object and not directly to the overall :class:`_schema.MetaData` object. - The use case supports the same :class:`_schema.Sequence` being associated - with more than one :class:`_schema.Table` at a time. - - .. change:: - :tags: bug, orm - :tickets: 5952 - :versions: 1.4.0 - - Fixed issue where the process of joining two tables could fail if one of - the tables had an unrelated, unresolvable foreign key constraint which - would raise :class:`_exc.NoReferenceError` within the join process, which - nonetheless could be bypassed to allow the join to complete. The logic - which tested the exception for significance within the process would make - assumptions about the construct which would fail. - - - .. change:: - :tags: bug, postgresql, reflection - :tickets: 6161 - :versions: 1.4.4 - - Fixed issue in PostgreSQL reflection where a column expressing "NOT NULL" - will supersede the nullability of a corresponding domain. - - .. change:: - :tags: bug, engine - :tickets: 5929 - :versions: 1.4.0 - - Fixed bug where the "schema_translate_map" feature failed to be taken into - account for the use case of direct execution of - :class:`_schema.DefaultGenerator` objects such as sequences, which included - the case where they were "pre-executed" in order to generate primary key - values when implicit_returning was disabled. - - .. change:: - :tags: bug, orm - :tickets: 6001 - :versions: 1.4.0 - - Fixed issue where the :class:`_mutable.MutableComposite` construct could be - placed into an invalid state when the parent object was already loaded, and - then covered by a subsequent query, due to the composite properties' - refresh handler replacing the object with a new one not handled by the - mutable extension. - - - .. change:: - :tags: bug, types, postgresql - :tickets: 6023 - :versions: 1.4.3 - - Adjusted the psycopg2 dialect to emit an explicit PostgreSQL-style cast for - bound parameters that contain ARRAY elements. This allows the full range of - datatypes to function correctly within arrays. The asyncpg dialect already - generated these internal casts in the final statement. This also includes - support for array slice updates as well as the PostgreSQL-specific - :meth:`_postgresql.ARRAY.contains` method. - - .. change:: - :tags: bug, mssql, reflection - :tickets: 5921 - - Fixed issue regarding SQL Server reflection for older SQL Server 2005 - version, a call to sp_columns would not proceed correctly without being - prefixed with the EXEC keyword. This method is not used in current 1.4 - series. - - -.. changelog:: - :version: 1.3.23 - :released: February 1, 2021 - - .. change:: - :tags: bug, ext - :tickets: 5836 - - Fixed issue where the stringification that is sometimes called when - attempting to generate the "key" for the ``.c`` collection on a selectable - would fail if the column were an unlabeled custom SQL construct using the - ``sqlalchemy.ext.compiler`` extension, and did not provide a default - compilation form; while this seems like an unusual case, it can get invoked - for some ORM scenarios such as when the expression is used in an "order by" - in combination with joined eager loading. The issue is that the lack of a - default compiler function was raising :class:`.CompileError` and not - :class:`.UnsupportedCompilationError`. - - .. change:: - :tags: bug, postgresql - :tickets: 5645 - - For SQLAlchemy 1.3 only, setup.py pins pg8000 to a version lower than - 1.16.6. Version 1.16.6 and above is supported by SQLAlchemy 1.4. Pull - request courtesy Giuseppe Lumia. - - .. change:: - :tags: bug, postgresql - :tickets: 5850 - - Fixed issue where using :meth:`_schema.Table.to_metadata` (called - :meth:`_schema.Table.tometadata` in 1.3) in conjunction with a PostgreSQL - :class:`_postgresql.ExcludeConstraint` that made use of ad-hoc column - expressions would fail to copy correctly. - - .. change:: - :tags: bug, sql - :tickets: 5816 - - Fixed bug where making use of the :meth:`.TypeEngine.with_variant` method - on a :class:`.TypeDecorator` type would fail to take into account the - dialect-specific mappings in use, due to a rule in :class:`.TypeDecorator` - that was instead attempting to check for chains of :class:`.TypeDecorator` - instances. - - - .. change:: - :tags: bug, mysql, reflection - :tickets: 5860 - - Fixed bug where MySQL server default reflection would fail for numeric - values with a negation symbol present. - - - .. change:: - :tags: bug, oracle - :tickets: 5813 - - Fixed regression in Oracle dialect introduced by :ticket:`4894` in - SQLAlchemy 1.3.11 where use of a SQL expression in RETURNING for an UPDATE - would fail to compile, due to a check for "server_default" when an - arbitrary SQL expression is not a column. - - - .. change:: - :tags: usecase, mysql - :tickets: 5808 - - Casting to ``FLOAT`` is now supported in MySQL >= (8, 0, 17) and - MariaDb >= (10, 4, 5). - - .. change:: - :tags: bug, mysql - :tickets: 5898 - - Fixed long-lived bug in MySQL dialect where the maximum identifier length - of 255 was too long for names of all types of constraints, not just - indexes, all of which have a size limit of 64. As metadata naming - conventions can create too-long names in this area, apply the limit to the - identifier generator within the DDL compiler. - - .. change:: - :tags: bug, oracle - :tickets: 5812 - - Fixed bug in Oracle dialect where retrieving a CLOB/BLOB column via - :meth:`_dml.Insert.returning` would fail as the LOB value would need to be - read when returned; additionally, repaired support for retrieval of Unicode - values via RETURNING under Python 2. - - .. change:: - :tags: bug, mysql - :tickets: 5821 - - Fixed deprecation warnings that arose as a result of the release of PyMySQL - 1.0, including deprecation warnings for the "db" and "passwd" parameters - now replaced with "database" and "password". - - - .. change:: - :tags: bug, mysql - :tickets: 5800 - - Fixed regression from SQLAlchemy 1.3.20 caused by the fix for - :ticket:`5462` which adds double-parenthesis for MySQL functional - expressions in indexes, as is required by the backend, this inadvertently - extended to include arbitrary :func:`_sql.text` expressions as well as - Alembic's internal textual component, which are required by Alembic for - arbitrary index expressions which don't imply double parenthesis. The - check has been narrowed to include only binary/ unary/functional - expressions directly. - -.. changelog:: - :version: 1.3.22 - :released: December 18, 2020 - - .. change:: - :tags: bug, oracle - :tickets: 5784 - :versions: 1.4.0b2 - - Fixed regression which occurred due to :ticket:`5755` which implemented - isolation level support for Oracle. It has been reported that many Oracle - accounts don't actually have permission to query the ``v$transaction`` - view so this feature has been altered to gracefully fallback when it fails - upon database connect, where the dialect will assume "READ COMMITTED" is - the default isolation level as was the case prior to SQLAlchemy 1.3.21. - However, explicit use of the :meth:`_engine.Connection.get_isolation_level` - method must now necessarily raise an exception, as Oracle databases with - this restriction explicitly disallow the user from reading the current - isolation level. - -.. changelog:: - :version: 1.3.21 - :released: December 17, 2020 - - .. change:: - :tags: bug, orm - :tickets: 5774 - :versions: 1.4.0b2 - - Added a comprehensive check and an informative error message for the case - where a mapped class, or a string mapped class name, is passed to - :paramref:`_orm.relationship.secondary`. This is an extremely common error - which warrants a clear message. - - Additionally, added a new rule to the class registry resolution such that - with regards to the :paramref:`_orm.relationship.secondary` parameter, if a - mapped class and its table are of the identical string name, the - :class:`.Table` will be favored when resolving this parameter. In all - other cases, the class continues to be favored if a class and table - share the identical name. - - .. change:: - :tags: sqlite, usecase - :tickets: 5685 - - Added ``sqlite_with_rowid=False`` dialect keyword to enable creating - tables as ``CREATE TABLE … WITHOUT ROWID``. Patch courtesy Sean Anderson. - - .. change:: - :tags: bug, sql - :tickets: 5691 - - A warning is emitted if a returning() method such as - :meth:`_sql.Insert.returning` is called multiple times, as this does not - yet support additive operation. Version 1.4 will support additive - operation for this. Additionally, any combination of the - :meth:`_sql.Insert.returning` and :meth:`_sql.ValuesBase.return_defaults` - methods now raises an error as these methods are mutually exclusive; - previously the operation would fail silently. - - - .. change:: - :tags: bug, mssql - :tickets: 5751 - - Fixed bug where a CREATE INDEX statement was rendered incorrectly when - both ``mssql-include`` and ``mssql_where`` were specified. Pull request - courtesy @Adiorz. - - .. change:: - :tags: bug, postgresql, mysql - :tickets: 5729 - :versions: 1.4.0b2 - - Fixed regression introduced in 1.3.2 for the PostgreSQL dialect, also - copied out to the MySQL dialect's feature in 1.3.18, where usage of a non - :class:`_schema.Table` construct such as :func:`_sql.text` as the argument - to :paramref:`_sql.Select.with_for_update.of` would fail to be accommodated - correctly within the PostgreSQL or MySQL compilers. - - - .. change:: - :tags: bug, mssql - :tickets: 5646 - - Added SQL Server code "01000" to the list of disconnect codes. - - - .. change:: - :tags: usecase, postgresql - :tickets: 5604 - :versions: 1.4.0b2 - - Added new parameter :paramref:`_postgresql.ExcludeConstraint.ops` to the - :class:`_postgresql.ExcludeConstraint` object, to support operator class - specification with this constraint. Pull request courtesy Alon Menczer. - - .. change:: - :tags: bug, mysql, reflection - :tickets: 5744 - :versions: 1.4.0b2 - - Fixed issue where reflecting a server default on MariaDB only that - contained a decimal point in the value would fail to be reflected - correctly, leading towards a reflected table that lacked any server - default. - - - .. change:: - :tags: bug, orm - :tickets: 5664 - - Fixed bug in :meth:`_query.Query.update` where objects in the - :class:`_ormsession.Session` that were already expired would be - unnecessarily SELECTed individually when they were refreshed by the - "evaluate"synchronize strategy. - - .. change:: - :tags: usecase, oracle - :tickets: 5755 - - Implemented support for the SERIALIZABLE isolation level for Oracle - databases, as well as a real implementation for - :meth:`_engine.Connection.get_isolation_level`. - - .. seealso:: - - :ref:`oracle_isolation_level` - - .. change:: - :tags: mysql, sql - :tickets: 5696 - - Added missing keywords to the ``RESERVED_WORDS`` list for the MySQL - dialect: ``action``, ``level``, ``mode``, ``status``, ``text``, ``time``. - Pull request courtesy Oscar Batori. - - .. change:: - :tags: bug, orm - :tickets: 5737 - :versions: 1.4.0b2 - - Fixed bug involving the ``restore_load_context`` option of ORM events such - as :meth:`_ormevent.InstanceEvents.load` such that the flag would not be - carried along to subclasses which were mapped after the event handler were - first established. - - - - .. change:: - :tags: bug, sql - :tickets: 5656 - - Fixed structural compiler issue where some constructs such as MySQL / - PostgreSQL "on conflict / on duplicate key" would rely upon the state of - the :class:`_sql.Compiler` object being fixed against their statement as - the top level statement, which would fail in cases where those statements - are branched from a different context, such as a DDL construct linked to a - SQL statement. - - - .. change:: - :tags: mssql, sqlite, reflection - :tickets: 5661 - - Fixed issue with composite primary key columns not being reported - in the correct order. Patch courtesy @fulpm. - -.. changelog:: - :version: 1.3.20 - :released: October 12, 2020 - - .. change:: - :tags: bug, orm - :tickets: 4428 - - An :class:`.ArgumentError` with more detail is now raised if the target - parameter for :meth:`_query.Query.join` is set to an unmapped object. - Prior to this change a less detailed ``AttributeError`` was raised. - Pull request courtesy Ramon Williams. - - .. change:: - :tags: bug, mysql - :tickets: 5568 - - The "skip_locked" keyword used with ``with_for_update()`` will emit a - warning when used on MariaDB backends, and will then be ignored. This is - a deprecated behavior that will raise in SQLAlchemy 1.4, as an application - that requests "skip locked" is looking for a non-blocking operation which - is not available on those backends. - - - - .. change:: - :tags: bug, engine - :tickets: 5599 - - Fixed issue where a non-string object sent to - :class:`_exc.SQLAlchemyError` or a subclass, as occurs with some third - party dialects, would fail to stringify correctly. Pull request - courtesy Andrzej Bartosiński. - - .. change:: - :tags: bug, sql - :tickets: 5644 - - Fixed issue where the ``pickle.dumps()`` operation against - :class:`_expression.Over` construct would produce a recursion overflow. - - .. change:: - :tags: postgresql, usecase - :tickets: 4392 - - The psycopg2 dialect now support PostgreSQL multiple host connections, by - passing host/port combinations to the query string. Pull request courtesy - Ramon Williams. - - .. seealso:: - - :ref:`psycopg2_multi_host` - - .. change:: - :tags: bug, mysql - :tickets: 5617 - - Fixed bug where an UPDATE statement against a JOIN using MySQL multi-table - format would fail to include the table prefix for the target table if the - statement had no WHERE clause, as only the WHERE clause were scanned to - detect a "multi table update" at that particular point. The target - is now also scanned if it's a JOIN to get the leftmost table as the - primary table and the additional entries as additional FROM entries. - - - .. change:: - :tags: bug, postgresql - :tickets: 5518 - - Adjusted the :meth:`_types.ARRAY.Comparator.any` and - :meth:`_types.ARRAY.Comparator.all` methods to implement a straight "NOT" - operation for negation, rather than negating the comparison operator. - - .. change:: - :tags: bug, pool - :tickets: 5582 - - Fixed issue where the following pool parameters were not being propagated - to the new pool created when :meth:`_engine.Engine.dispose` were called: - ``pre_ping``, ``use_lifo``. Additionally the ``recycle`` and - ``reset_on_return`` parameter is now propagated for the - :class:`_engine.AssertionPool` class. - - .. change:: - :tags: bug, ext, associationproxy - :tickets: 5541, 5542 - - An informative error is now raised when attempting to use an association - proxy element as a plain column expression to be SELECTed from or used in a - SQL function; this use case is not currently supported. - - - .. change:: - :tags: bug, sql - :tickets: 5618 - - Fixed bug where an error was not raised in the case where a - :func:`_sql.column` were added to more than one :func:`_sql.table` at a - time. This raised correctly for the :class:`_schema.Column` and - :class:`_schema.Table` objects. An :class:`_exc.ArgumentError` is now - raised when this occurs. - - .. change:: - :tags: bug, orm - :tickets: 4589 - - Fixed issue where using a loader option against a string attribute name - that is not actually a mapped attribute, such as a plain Python descriptor, - would raise an uninformative AttributeError; a descriptive error is now - raised. - - - - .. change:: - :tags: mysql, usecase - :tickets: 5462 - - Adjusted the MySQL dialect to correctly parenthesize functional index - expressions as accepted by MySQL 8. Pull request courtesy Ramon Williams. - - .. change:: - :tags: bug, engine - :tickets: 5632 - - Repaired a function-level import that was not using SQLAlchemy's standard - late-import system within the sqlalchemy.exc module. - - - .. change:: - :tags: change, mysql - :tickets: 5539 - - Add new MySQL reserved words: ``cube``, ``lateral`` added in MySQL 8.0.1 - and 8.0.14, respectively; this indicates that these terms will be quoted if - used as table or column identifier names. - - .. change:: - :tags: bug, mssql - :tickets: 5592 - - Fixed issue where a SQLAlchemy connection URI for Azure DW with - ``authentication=ActiveDirectoryIntegrated`` (and no username+password) - was not constructing the ODBC connection string in a way that was - acceptable to the Azure DW instance. - - .. change:: - :tags: bug, postgresql - :tickets: 5520 - - Fixed issue where the :class:`_postgresql.ENUM` type would not consult the - schema translate map when emitting a CREATE TYPE or DROP TYPE during the - test to see if the type exists or not. Additionally, repaired an issue - where if the same enum were encountered multiple times in a single DDL - sequence, the "check" query would run repeatedly rather than relying upon a - cached value. - - - .. change:: - :tags: bug, tests - :tickets: 5635 - - Fixed incompatibilities in the test suite when running against Pytest 6.x. - - -.. changelog:: - :version: 1.3.19 - :released: August 17, 2020 - - .. change:: - :tags: usecase, py3k - :tickets: #5357 - - Added a ``**kw`` argument to the :meth:`.DeclarativeMeta.__init__` method. - This allows a class to support the :pep:`487` metaclass hook - ``__init_subclass__``. Pull request courtesy Ewen Gillies. - - - .. change:: - :tags: bug, sql - :tickets: 5470 - - Repaired an issue where the "ORDER BY" clause rendering a label name rather - than a complete expression, which is particularly important for SQL Server, - would fail to occur if the expression were enclosed in a parenthesized - grouping in some cases. This case has been added to test support. The - change additionally adjusts the "automatically add ORDER BY columns when - DISTINCT is present" behavior of ORM query, deprecated in 1.4, to more - accurately detect column expressions that are already present. - - .. change:: - :tags: usecase, mysql - :tickets: 5481 - - The MySQL dialect will render FROM DUAL for a SELECT statement that has no - FROM clause but has a WHERE clause. This allows things like "SELECT 1 WHERE - EXISTS (subquery)" kinds of queries to be used as well as other use cases. - - - .. change:: - :tags: bug, mssql, sql - :tickets: 5467 - - Fixed bug where the mssql dialect incorrectly escaped object names that - contained ']' character(s). - - .. change:: - :tags: bug, reflection, sqlite, mssql - :tickets: 5456 - - Applied a sweep through all included dialects to ensure names that contain - single or double quotes are properly escaped when querying system tables, - for all :class:`.Inspector` methods that accept object names as an argument - (e.g. table names, view names, etc). SQLite and MSSQL contained two - quoting issues that were repaired. - - .. change:: - :tags: bug, mysql - :tickets: 5411 - - Fixed an issue where CREATE TABLE statements were not specifying the - COLLATE keyword correctly. - - .. change:: - :tags: bug, datatypes, sql - :tickets: 4733 - - The ``LookupError`` message will now provide the user with up to four - possible values that a column is constrained to via the :class:`.Enum`. - Values longer than 11 characters will be truncated and replaced with - ellipses. Pull request courtesy Ramon Williams. - - .. change:: - :tags: bug, postgresql - :tickets: 5476 - - Fixed issue where the return type for the various RANGE comparison - operators would itself be the same RANGE type rather than BOOLEAN, which - would cause an undesirable result in the case that a - :class:`.TypeDecorator` that defined result-processing behavior were in - use. Pull request courtesy Jim Bosch. - - - - .. change:: - :tags: bug, mysql - :tickets: 5493 - - Added MariaDB code 1927 to the list of "disconnect" codes, as recent - MariaDB versions apparently use this code when the database server was - stopped. - - .. change:: - :tags: usecase, declarative, orm - :tickets: 5513 - - The name of the virtual column used when using the - :class:`_declarative.AbstractConcreteBase` and - :class:`_declarative.ConcreteBase` classes can now be customized, to allow - for models that have a column that is actually named ``type``. Pull - request courtesy Jesse-Bakker. - - .. change:: - :tags: usecase, orm - :tickets: 5494 - - Adjusted the workings of the :meth:`_orm.Mapper.all_orm_descriptors` - accessor to represent the attributes in the order that they are located in - a deterministic way, assuming the use of Python 3.6 or higher which - maintains the sorting order of class attributes based on how they were - declared. This sorting is not guaranteed to match the declared order of - attributes in all cases however; see the method documentation for the exact - scheme. - - - - .. change:: - :tags: bug, sql - :tickets: 5500 - - Fixed issue where the - :paramref:`_engine.Connection.execution_options.schema_translate_map` - feature would not take effect when the :meth:`_schema.Sequence.next_value` - function function for a :class:`_schema.Sequence` were used in the - :paramref:`_schema.Column.server_default` parameter and the create table - DDL were emitted. - -.. changelog:: - :version: 1.3.18 - :released: June 25, 2020 - - .. change:: - :tags: bug, sqlite - :tickets: 5395 - - Added "exists" to the list of reserved words for SQLite so that this word - will be quoted when used as a label or column name. Pull request courtesy - Thodoris Sotiropoulos. - - .. change:: - :tags: bug, mssql - :tickets: 5366, 5364 - - Refined the logic used by the SQL Server dialect to interpret multi-part - schema names that contain many dots, to not actually lose any dots if the - name does not have bracking or quoting used, and additionally to support a - "dbname" token that has many parts including that it may have multiple, - independently-bracketed sections. - - - - .. change:: - :tags: bug, mssql, pyodbc - :tickets: 5346 - - Fixed an issue in the pyodbc connector such that a warning about pyodbc - "drivername" would be emitted when using a totally empty URL. Empty URLs - are normal when producing a non-connected dialect object or when using the - "creator" argument to create_engine(). The warning now only emits if the - driver name is missing but other parameters are still present. - - .. change:: - :tags: bug, mssql - :tickets: 5373 - - Fixed issue with assembling the ODBC connection string for the pyodbc - DBAPI. Tokens containing semicolons and/or braces "{}" were not being - correctly escaped, causing the ODBC driver to misinterpret the - connection string attributes. - - .. change:: - :tags: usecase, orm - :tickets: 5326 - - Improve error message when using :meth:`_query.Query.filter_by` in - a query where the first entity is not a mapped class. - - .. change:: - :tags: sql, schema - :tickets: 5324 - - Introduce :class:`.IdentityOptions` to store common parameters for - sequences and identity columns. - - .. change:: - :tags: usecase, sql - :tickets: 5309 - - Added a ".schema" parameter to the :func:`_expression.table` construct, - allowing ad-hoc table expressions to also include a schema name. - Pull request courtesy Dylan Modesitt. - - .. change:: - :tags: bug, mssql - :tickets: 5339 - - Fixed issue where ``datetime.time`` parameters were being converted to - ``datetime.datetime``, making them incompatible with comparisons like - ``>=`` against an actual :class:`_mssql.TIME` column. - - .. change:: - :tags: bug, mssql - :tickets: 5359 - - Fixed an issue where the ``is_disconnect`` function in the SQL Server - pyodbc dialect was incorrectly reporting the disconnect state when the - exception message had a substring that matched a SQL Server ODBC error - code. - - .. change:: - :tags: bug, engine - :tickets: 5326 - - Further refinements to the fixes to the "reset" agent fixed in - :ticket:`5326`, which now emits a warning when it is not being correctly - invoked and corrects for the behavior. Additional scenarios have been - identified and fixed where this warning was being emitted. - - - .. change:: - :tags: usecase, sqlite - :tickets: 5297 - - SQLite 3.31 added support for computed column. This change - enables their support in SQLAlchemy when targeting SQLite. - - .. change:: - :tags: bug, schema - :tickets: 5276 - - Fixed issue where ``dialect_options`` were omitted when a - database object (e.g., :class:`.Table`) was copied using - :func:`.tometadata`. - - .. change:: - :tags: bug, sql - :tickets: 5344 - - Correctly apply self_group in type_coerce element. - - The type coerce element did not correctly apply grouping rules when using - in an expression - - .. change:: - :tags: bug, oracle, reflection - :tickets: 5421 - - Fixed bug in Oracle dialect where indexes that contain the full set of - primary key columns would be mistaken as the primary key index itself, - which is omitted, even if there were multiples. The check has been refined - to compare the name of the primary key constraint against the index name - itself, rather than trying to guess based on the columns present in the - index. - - .. change:: - :tags: change, sql, sybase - :tickets: 5294 - - Added ``.offset`` support to sybase dialect. - Pull request courtesy Alan D. Snow. - - .. change:: - :tags: bug, engine - :tickets: 5341 - - Fixed issue in :class:`.URL` object where stringifying the object - would not URL encode special characters, preventing the URL from being - re-consumable as a real URL. Pull request courtesy Miguel Grinberg. - - .. change:: - :tags: usecase, mysql - :tickets: 4860 - - Implemented row-level locking support for mysql. Pull request courtesy - Quentin Somerville. - - .. change:: - :tags: change, mssql - :tickets: 5321 - - Moved the ``supports_sane_rowcount_returning = False`` requirement from - the ``PyODBCConnector`` level to the ``MSDialect_pyodbc`` since pyodbc - does work properly in some circumstances. - - .. change:: - :tags: change, examples - - Added new option ``--raw`` to the examples.performance suite - which will dump the raw profile test for consumption by any - number of profiling visualizer tools. Removed the "runsnake" - option as runsnake is very hard to build at this point; - - .. change:: - :tags: bug, sql - :tickets: 5353 - - Added :meth:`.Select.with_hint` output to the generic SQL string that is - produced when calling ``str()`` on a statement. Previously, this clause - would be omitted under the assumption that it was dialect specific. - The hint text is presented within brackets to indicate the rendering - of such hints varies among backends. - - - .. change:: - :tags: usecase, orm - :tickets: 5198 - - Added a new parameter :paramref:`_orm.query_expression.default_expr` to the - :func:`_orm.query_expression` construct, which will be appled to queries - automatically if the :func:`_orm.with_expression` option is not used. Pull - request courtesy Haoyu Sun. - -.. changelog:: - :version: 1.3.17 - :released: May 13, 2020 - - .. change:: - :tags: bug, oracle - :tickets: 5246 - - Some modifications to how the cx_oracle dialect sets up per-column - outputtype handlers for LOB and numeric datatypes to adjust for potential - changes coming in cx_Oracle 8. - - - .. change:: - :tags: bug, orm - :tickets: 5288 - - Fixed bug where using :func:`.with_polymorphic` as the target of a join via - :meth:`.RelationshipComparator.of_type` on a mapper that already has a - subquery-based with_polymorphic setting that's equivalent to the one - requested would not correctly alias the ON clause in the join. - - .. change:: - :tags: bug, oracle, performance - :tickets: 5314 - - Changed the implementation of fetching CLOB and BLOB objects to use - cx_Oracle's native implementation which fetches CLOB/BLOB objects inline - with other result columns, rather than performing a separate fetch. As - always, this can be disabled by setting auto_convert_lobs to False. - - As part of this change, the behavior of a CLOB that was given a blank - string on INSERT now returns None on SELECT, which is now consistent with - that of VARCHAR on Oracle. - - - .. change:: - :tags: usecase, postgresql - :tickets: 5265 - - Added support for columns or type :class:`_sqltypes.ARRAY` of :class:`.Enum`, - :class:`_postgresql.JSON` or :class:`_postgresql.JSONB` in PostgreSQL. - Previously a workaround was required in these use cases. - - - .. change:: - :tags: schema - :tickets: 4138 - - Add ``comment`` attribute to :class:`_schema.Column` ``__repr__`` method. - - .. change:: - :tags: bug, orm - :tickets: 5303 - - Fixed issue in the area of where loader options such as selectinload() - interact with the baked query system, such that the caching of a query is - not supposed to occur if the loader options themselves have elements such - as with_polymorphic() objects in them that currently are not - cache-compatible. The baked loader could sometimes not fully invalidate - itself in these some of these scenarios leading to missed eager loads. - - - .. change:: - :tags: bug, engine - :tickets: 5326 - - Fixed fairly critical issue where the DBAPI connection could be returned to - the connection pool while still in an un-rolled-back state. The reset agent - responsible for rolling back the connection could be corrupted in the case - that the transaction was "closed" without being rolled back or committed, - which can occur in some scenarios when using ORM sessions and emitting - .close() in a certain pattern involving savepoints. The fix ensures that - the reset agent is always active. - - - .. change:: - :tags: bug, orm - :tickets: 5304 - - Modified the internal "identity set" implementation, which is a set that - hashes objects on their id() rather than their hash values, to not actually - call the ``__hash__()`` method of the objects, which are typically - user-mapped objects. Some methods were calling this method as a side - effect of the implementation. - - - .. change:: - :tags: usecase, postgresql - :tickets: 5266 - - Raise an explicit :class:`.exc.CompileError` when adding a table with a - column of type :class:`_sqltypes.ARRAY` of :class:`.Enum` configured with - :paramref:`.Enum.native_enum` set to ``False`` when - :paramref:`.Enum.create_constraint` is not set to ``False`` - - .. change:: - :tags: bug, schema - :tickets: 5298 - - Fixed issue where an :class:`.Index` that is deferred in being associated - with a table, such as as when it contains a :class:`.Column` that is not - associated with any :class:`.Table` yet, would fail to attach correctly if - it also contained a non table-oriented expression. - - - .. change:: - :tags: change, firebird - :tickets: 5278 - - Adjusted dialect loading for ``firebird://`` URIs so the external - sqlalchemy-firebird dialect will be used if it has been installed, - otherwise fall back to the (now deprecated) internal Firebird dialect. - - .. change:: - :tags: bug, mssql, reflection - :tickets: 5255 - - Fix a regression introduced by the reflection of computed column in - MSSQL when using the legacy TDS version 4.2. The dialect will try - to detect the protocol version of first connect and run in compatibility - mode if it cannot detect it. - - .. change:: - :tags: bug, mssql, reflection - :tickets: 5271 - - Fix a regression introduced by the reflection of computed column in - MSSQL when using SQL server versions before 2012, which does not support - the ``concat`` function. - - .. change:: - :tags: bug, orm - :tickets: 5269 - - An informative error message is raised when an ORM many-to-one comparison - is attempted against an object that is not an actual mapped instance. - Comparisons such as those to scalar subqueries aren't supported; - generalized comparison with subqueries is better achieved using - :meth:`~.RelationshipProperty.Comparator.has`. - - - .. change:: - :tags: usecase, orm - :tickets: 5262 - - Added an accessor :attr:`.ColumnProperty.Comparator.expressions` which - provides access to the group of columns mapped under a multi-column - :class:`.ColumnProperty` attribute. - - - .. change:: - :tags: bug, schema - :tickets: 5316 - - A warning is emitted when making use of the :attr:`.MetaData.sorted_tables` - attribute as well as the :func:`_schema.sort_tables` function, and the - given tables cannot be correctly sorted due to a cyclic dependency between - foreign key constraints. In this case, the functions will no longer sort - the involved tables by foreign key, and a warning will be emitted. Other - tables that are not part of the cycle will still be returned in dependency - order. Previously, the sorted_table routines would return a collection that - would unconditionally omit all foreign keys when a cycle was detected, and - no warning was emitted. - - - .. change:: - :tags: orm, usecase - :tickets: 5237 - - Introduce :paramref:`_orm.relationship.sync_backref` flag in a relationship - to control if the synchronization events that mutate the in-Python - attributes are added. This supersedes the previous change :ticket:`5149`, - which warned that ``viewonly=True`` relationship target of a - back_populates or backref configuration would be disallowed. - -.. changelog:: - :version: 1.3.16 - :released: April 7, 2020 - - .. change:: - :tags: oracle, usecase - :tickets: 5200 - - Implemented AUTOCOMMIT isolation level for Oracle when using cx_Oracle. - Also added a fixed default isolation level of READ COMMITTED for Oracle. - - - .. change:: - :tags: bug, mysql - :tickets: 5239 - - Fixed issue in MySQL dialect when connecting to a pseudo-MySQL database - such as that provided by ProxySQL, the up front check for isolation level - when it returns no row will not prevent the dialect from continuing to - connect. A warning is emitted that the isolation level could not be - detected. - - - .. change:: - :tags: bug, tests - :tickets: 5201 - - Fixed an issue that prevented the test suite from running with the - recently released py.test 5.4.0. - - - .. change:: - :tags: bug, oracle, reflection - :tickets: 5146 - - Fixed regression / incorrect fix caused by fix for :ticket:`5146` where the - Oracle dialect reads from the "all_tab_comments" view to get table comments - but fails to accommodate for the current owner of the table being - requested, causing it to read the wrong comment if multiple tables of the - same name exist in multiple schemas. - - - .. change:: - :tags: types, enum - :tickets: 5183 - - The :class:`.Enum` type now supports the parameter :paramref:`.Enum.length` - to specify the length of the VARCHAR column to create when using - non native enums by setting :paramref:`.Enum.native_enum` to ``False`` - - .. change:: - :tags: bug, orm - :tickets: 5228 - - Fixed bug in :func:`_orm.selectinload` loading option where two or more - loaders that represent different relationships with the same string key - name as referenced from a single :func:`_orm.with_polymorphic` construct - with multiple subclass mappers would fail to invoke each subqueryload - separately, instead making use of a single string-based slot that would - prevent the other loaders from being invoked. - - - .. change:: - :tags: schema, reflection - :tickets: 5063 - - Added support for reflection of "computed" columns, which are now returned - as part of the structure returned by :meth:`_reflection.Inspector.get_columns`. - When reflecting full :class:`_schema.Table` objects, computed columns will - be represented using the :class:`.Computed` construct. - - .. change:: - :tags: orm, performance - :tickets: 5162 - - Modified the queries used by subqueryload and selectinload to no longer - ORDER BY the primary key of the parent entity; this ordering was there to - allow the rows as they come in to be copied into lists directly with a - minimal level of Python-side collation. However, these ORDER BY clauses - can negatively impact the performance of the query as in many scenarios - these columns are derived from a subquery or are otherwise not actual - primary key columns such that SQL planners cannot make use of indexes. The - Python-side collation uses the native itertools.group_by() to collate the - incoming rows, and has been modified to allow multiple - row-groups-per-parent to be assembled together using list.extend(), which - should still allow for relatively fast Python-side performance. There will - still be an ORDER BY present for a relationship that includes an explicit - order_by parameter, however this is the only ORDER BY that will be added to - the query for both kinds of loading. - - .. change:: - :tags: mssql, mysql, oracle, usecase - :tickets: 5137 - - Added support for :meth:`.ColumnOperators.is_distinct_from` and - :meth:`.ColumnOperators.isnot_distinct_from` to SQL Server, - MySQL, and Oracle. - - .. change:: - :tags: sqlite, usecase - :tickets: 5164 - - Implemented AUTOCOMMIT isolation level for SQLite when using pysqlite. - - .. change:: - :tags: bug, postgresql - :tickets: 5205 - - Fixed issue where a "covering" index, e.g. those which have an INCLUDE - clause, would be reflected including all the columns in INCLUDE clause as - regular columns. A warning is now emitted if these additional columns are - detected indicating that they are currently ignored. Note that full - support for "covering" indexes is part of :ticket:`4458`. Pull request - courtesy Marat Sharafutdinov. - - .. change:: - :tags: sql, types - :tickets: 5052 - - Add ability to literal compile a :class:`DateTime`, :class:`Date` - or :class:`Time` when using the string dialect for debugging purposes. - This change does not impact real dialect implementation that retain - their current behavior. - - .. change:: - :tags: installer - :tickets: 5207 - - Ensured that the "pyproject.toml" file is not included in builds, as the - presence of this file indicates to pip that a pep-517 installation process - should be used. As this mode of operation appears to be not well supported - by current tools / distros, these problems are avoided within the scope - of SQLAlchemy installation by omitting the file. - - - .. change:: - :tags: bug, orm - :tickets: 5210 - - Fixed issue where a lazyload that uses session-local "get" against a target - many-to-one relationship where an object with the correct primary key is - present, however it's an instance of a sibling class, does not correctly - return None as is the case when the lazy loader actually emits a load for - that row. - - .. change:: - :tags: bug, orm, declarative - :tickets: 5238 - - The string argument accepted as the first positional argument by the - :func:`_orm.relationship` function when using the Declarative API is no longer - interpreted using the Python ``eval()`` function; instead, the name is dot - separated and the names are looked up directly in the name resolution - dictionary without treating the value as a Python expression. However, - passing a string argument to the other :func:`_orm.relationship` parameters - that necessarily must accept Python expressions will still use ``eval()``; - the documentation has been clarified to ensure that there is no ambiguity - that this is in use. - - .. seealso:: - - :ref:`declarative_relationship_eval` - details on string evaluation - -.. changelog:: - :version: 1.3.15 - :released: March 11, 2020 - - .. change:: - :tags: bug, orm - :tickets: 5194 - - Adjusted the error message emitted by :meth:`_query.Query.join` when a left hand - side can't be located that the :meth:`_query.Query.select_from` method is the - best way to resolve the issue. Also, within the 1.3 series, used a - deterministic ordering when determining the FROM clause from a given column - entity passed to :class:`_query.Query` so that the same expression is determined - each time. - - - .. change:: - :tags: orm, bug - :tickets: 5196 - - Fixed regression in 1.3.14 due to :ticket:`4849` where a sys.exc_info() - call failed to be invoked correctly when a flush error would occur. Test - coverage has been added for this exception case. - - -.. changelog:: - :version: 1.3.14 - :released: March 10, 2020 - - .. change:: - :tags: bug, sql, postgresql - :tickets: 5181 - - Fixed bug where a CTE of an INSERT/UPDATE/DELETE that also uses RETURNING - could then not be SELECTed from directly, as the internal state of the - compiler would try to treat the outer SELECT as a DELETE statement itself - and access nonexistent state. - - - .. change:: - :tags: bug, orm - :tickets: 5110 - - Fixed regression caused in 1.3.13 by :ticket:`5056` where a refactor of the - ORM path registry system made it such that a path could no longer be - compared to an empty tuple, which can occur in a particular kind of joined - eager loading path. The "empty tuple" use case has been resolved so that - the path registry is compared to a path registry in all cases; the - :class:`.PathRegistry` object itself now implements ``__eq__()`` and - ``__ne__()`` methods which will take place for all equality comparisons and - continue to succeed in the not anticipated case that a non- - :class:`.PathRegistry` object is compared, while emitting a warning that - this object should not be the subject of the comparison. - - - - .. change:: - :tags: bug, orm - :tickets: 5149 - - Setting a relationship to viewonly=True which is also the target of a - back_populates or backref configuration will now emit a warning and - eventually be disallowed. back_populates refers specifically to mutation - of an attribute or collection, which is disallowed when the attribute is - subject to viewonly=True. The viewonly attribute is not subject to - persistence behaviors which means it will not reflect correct results - when it is locally mutated. - - .. change:: - :tags: bug, oracle - :tickets: 5146 - - Fixed a reflection bug where table comments could only be retrieved for - tables actually owned by the user but not for tables visible to the user - but owned by someone else. Pull request courtesy Dave Hirschfeld. - - .. change:: - :tags: bug, performance - :tickets: 5180 - - Revised an internal change to the test system added as a result of - :ticket:`5085` where a testing-related module per dialect would be loaded - unconditionally upon making use of that dialect, pulling in SQLAlchemy's - testing framework as well as the ORM into the module import space. This - would only impact initial startup time and memory to a modest extent, - however it's best that these additional modules aren't reverse-dependent on - straight Core usage. - - .. change:: - :tags: bug, installation - :tickets: 5138 - - Vendored the ``inspect.formatannotation`` function inside of - ``sqlalchemy.util.compat``, which is needed for the vendored version of - ``inspect.formatargspec``. The function is not documented in cPython and - is not guaranteed to be available in future Python versions. - - - .. change:: - :tags: bug, mssql - :tickets: 5132 - - Fixed issue where the :class:`_mssql.DATETIMEOFFSET` type would not - accommodate for the ``None`` value, introduced as part of the series of - fixes for this type first introduced in :ticket:`4983`, :ticket:`5045`. - Additionally, added support for passing a backend-specific date formatted - string through this type, as is typically allowed for date/time types on - most other DBAPIs. - - .. change:: - :tags: bug, engine - :tickets: 5182 - - Expanded the scope of cursor/connection cleanup when a statement is - executed to include when the result object fails to be constructed, or an - after_cursor_execute() event raises an error, or autocommit / autoclose - fails. This allows the DBAPI cursor to be cleaned up on failure and for - connectionless execution allows the connection to be closed out and - returned to the connection pool, where previously it waiting until garbage - collection would trigger a pool return. - - .. change:: - :tags: bug, postgresql - :tickets: 5158 - - Fixed issue where the "schema_translate_map" feature would not work with a - PostgreSQL native enumeration type (i.e. :class:`.Enum`, - :class:`_postgresql.ENUM`) in that while the "CREATE TYPE" statement would - be emitted with the correct schema, the schema would not be rendered in - the CREATE TABLE statement at the point at which the enumeration was - referenced. - - - .. change:: - :tags: usecase, ext - :tickets: 5114 - - Added keyword arguments to the :meth:`.MutableList.sort` function so that a - key function as well as the "reverse" keyword argument can be provided. - - - .. change:: - :tags: bug, general, py3k - :tickets: 4849 - - Applied an explicit "cause" to most if not all internally raised exceptions - that are raised from within an internal exception catch, to avoid - misleading stacktraces that suggest an error within the handling of an - exception. While it would be preferable to suppress the internally caught - exception in the way that the ``__suppress_context__`` attribute would, - there does not as yet seem to be a way to do this without suppressing an - enclosing user constructed context, so for now it exposes the internally - caught exception as the cause so that full information about the context - of the error is maintained. - - .. change:: - :tags: orm, bug - :tickets: 5121 - - Fixed an additional regression in the same area as that of :ticket:`5080` - introduced in 1.3.0b3 via :ticket:`4468` where the ability to create a - joined option across a :func:`.with_polymorphic` into a relationship - against the base class of that with_polymorphic, and then further into - regular mapped relationships would fail as the base class component would - not add itself to the load path in a way that could be located by the - loader strategy. The changes applied in :ticket:`5080` have been further - refined to also accommodate this scenario. - - .. change:: - :tags: bug, postgresql, reflection - :tickets: 5170 - - Fixed bug where PostgreSQL reflection of CHECK constraints would fail to - parse the constraint if the SQL text contained newline characters. The - regular expression has been adjusted to accommodate for this case. Pull - request courtesy Eric Borczuk. - - .. change:: - :tags: usecase, orm - :tickets: 5129 - - Added a new flag :paramref:`.InstanceEvents.restore_load_context` and - :paramref:`.SessionEvents.restore_load_context` which apply to the - :meth:`.InstanceEvents.load`, :meth:`.InstanceEvents.refresh`, and - :meth:`.SessionEvents.loaded_as_persistent` events, which when set will - restore the "load context" of the object after the event hook has been - called. This ensures that the object remains within the "loader context" - of the load operation that is already ongoing, rather than the object being - transferred to a new load context due to refresh operations which may have - occurred in the event. A warning is now emitted when this condition occurs, - which recommends use of the flag to resolve this case. The flag is - "opt-in" so that there is no risk introduced to existing applications. - - The change additionally adds support for the ``raw=True`` flag to - session lifecycle events. - - .. change:: - :tags: bug, mysql - :tickets: 5173 - - Fixed issue in MySQL :meth:`.mysql.Insert.on_duplicate_key_update` construct - where using a SQL function or other composed expression for a column argument - would not properly render the ``VALUES`` keyword surrounding the column - itself. - -.. changelog:: - :version: 1.3.13 - :released: January 22, 2020 - - .. change:: - :tags: bug, postgresql - :tickets: 5039 - - Fixed issue where the PostgreSQL dialect would fail to parse a reflected - CHECK constraint that was a boolean-valued function (as opposed to a - boolean-valued expression). - - .. change:: - :tags: bug, ext - :tickets: 5086 - - Fixed bug in sqlalchemy.ext.serializer where a unique - :class:`.BindParameter` object could conflict with itself if it were - present in the mapping itself, as well as the filter condition of the - query, as one side would be used against the non-deserialized version and - the other side would use the deserialized version. Logic is added to - :class:`.BindParameter` similar to its "clone" method which will uniquify - the parameter name upon deserialize so that it doesn't conflict with its - original. - - - .. change:: - :tags: usecase, sql - :tickets: 5079 - - A function created using :class:`.GenericFunction` can now specify that the - name of the function should be rendered with or without quotes by assigning - the :class:`.quoted_name` construct to the .name element of the object. - Prior to 1.3.4, quoting was never applied to function names, and some - quoting was introduced in :ticket:`4467` but no means to force quoting for - a mixed case name was available. Additionally, the :class:`.quoted_name` - construct when used as the name will properly register its lowercase name - in the function registry so that the name continues to be available via the - ``func.`` registry. - - .. seealso:: - - :class:`.GenericFunction` - - - .. change:: - :tags: bug, engine - :tickets: 5048 - - Fixed issue where the collection of value processors on a - :class:`.Compiled` object would be mutated when "expanding IN" parameters - were used with a datatype that has bind value processors; in particular, - this would mean that when using statement caching and/or baked queries, the - same compiled._bind_processors collection would be mutated concurrently. - Since these processors are the same function for a given bind parameter - namespace every time, there was no actual negative effect of this issue, - however, the execution of a :class:`.Compiled` object should never be - causing any changes in its state, especially given that they are intended - to be thread-safe and reusable once fully constructed. - - - .. change:: - :tags: tests, postgresql - :tickets: 5057 - - Improved detection of two phase transactions requirement for the PostgreSQL - database by testing that max_prepared_transactions is set to a value - greater than 0. Pull request courtesy Federico Caselli. - - - .. change:: - :tags: bug, orm, engine - :tickets: 5056, 5050, 5071 - - Added test support and repaired a wide variety of unnecessary reference - cycles created for short-lived objects, mostly in the area of ORM queries. - Thanks much to Carson Ip for the help on this. - - - .. change:: - :tags: orm, bug - :tickets: 5107 - - Fixed regression in loader options introduced in 1.3.0b3 via :ticket:`4468` - where the ability to create a loader option using - :meth:`.PropComparator.of_type` targeting an aliased entity that is an - inheriting subclass of the entity which the preceding relationship refers - to would fail to produce a matching path. See also :ticket:`5082` fixed - in this same release which involves a similar kind of issue. - - .. change:: - :tags: bug, tests - :tickets: 4946 - - Fixed a few test failures which would occur on Windows due to SQLite file - locking issues, as well as some timing issues in connection pool related - tests; pull request courtesy Federico Caselli. - - - .. change:: - :tags: orm, bug - :tickets: 5082 - - Fixed regression in joined eager loading introduced in 1.3.0b3 via - :ticket:`4468` where the ability to create a joined option across a - :func:`.with_polymorphic` into a polymorphic subclass using - :meth:`.RelationshipProperty.of_type` and then further along regular mapped - relationships would fail as the polymorphic subclass would not add itself - to the load path in a way that could be located by the loader strategy. A - tweak has been made to resolve this scenario. - - - .. change:: - :tags: performance, orm - - Identified a performance issue in the system by which a join is constructed - based on a mapped relationship. The clause adaption system would be used - for the majority of join expressions including in the common case where no - adaptation is needed. The conditions under which this adaptation occur - have been refined so that average non-aliased joins along a simple - relationship without a "secondary" table use about 70% less function calls. - - - .. change:: - :tags: usecase, postgresql - :tickets: 5040 - - Added support for prefixes to the :class:`_expression.CTE` construct, to allow - support for Postgresql 12 "MATERIALIZED" and "NOT MATERIALIZED" phrases. - Pull request courtesy Marat Sharafutdinov. - - .. seealso:: - - :meth:`_expression.HasCTE.cte` - - .. change:: - :tags: bug, mssql - :tickets: 5045 - - Fixed issue where a timezone-aware ``datetime`` value being converted to - string for use as a parameter value of a :class:`_mssql.DATETIMEOFFSET` - column was omitting the fractional seconds. - - .. change:: - :tags: bug, orm - :tickets: 5068 - - Repaired a warning in the ORM flush process that was not covered by test - coverage when deleting objects that use the "version_id" feature. This - warning is generally unreachable unless using a dialect that sets the - "supports_sane_rowcount" flag to False, which is not typically the case - however is possible for some MySQL configurations as well as older Firebird - drivers, and likely some third party dialects. - - .. change:: - :tags: bug, orm - :tickets: 5065 - - Fixed bug where usage of joined eager loading would not properly wrap the - query inside of a subquery when :meth:`_query.Query.group_by` were used against - the query. When any kind of result-limiting approach is used, such as - DISTINCT, LIMIT, OFFSET, joined eager loading embeds the row-limited query - inside of a subquery so that the collection results are not impacted. For - some reason, the presence of GROUP BY was never included in this criterion, - even though it has a similar effect as using DISTINCT. Additionally, the - bug would prevent using GROUP BY at all for a joined eager load query for - most database platforms which forbid non-aggregated, non-grouped columns - from being in the query, as the additional columns for the joined eager - load would not be accepted by the database. - - - -.. changelog:: - :version: 1.3.12 - :released: December 16, 2019 - - .. change:: - :tags: bug, sql - :tickets: 5028 - - Fixed bug where "distinct" keyword passed to :func:`_expression.select` would not - treat a string value as a "label reference" in the same way that the - :meth:`_expression.select.distinct` does; it would instead raise unconditionally. This - keyword argument and the others passed to :func:`_expression.select` will ultimately - be deprecated for SQLAlchemy 2.0. - - - .. change:: - :tags: bug, orm - :tickets: 4997 - - Fixed issue involving ``lazy="raise"`` strategy where an ORM delete of an - object would raise for a simple "use-get" style many-to-one relationship - that had lazy="raise" configured. This is inconsistent vs. the change - introduced in 1.3 as part of :ticket:`4353`, where it was established that - a history operation that does not expect emit SQL should bypass the - ``lazy="raise"`` check, and instead effectively treat it as - ``lazy="raise_on_sql"`` for this case. The fix adjusts the lazy loader - strategy to not raise for the case where the lazy load was instructed that - it should not emit SQL if the object were not present. - - .. change:: - :tags: bug, sql - - Changed the text of the exception for "Can't resolve label reference" to - include other kinds of label coercions, namely that "DISTINCT" is also in - this category under the PostgreSQL dialect. - - - .. change:: - :tags: bug, orm - :tickets: 5000 - - Fixed regression introduced in 1.3.0 related to the association proxy - refactor in :ticket:`4351` that prevented :func:`.composite` attributes - from working in terms of an association proxy that references them. - - .. change:: - :tags: bug, mssql - :tickets: 4983 - - Repaired support for the :class:`_mssql.DATETIMEOFFSET` datatype on PyODBC, - by adding PyODBC-level result handlers as it does not include native - support for this datatype. This includes usage of the Python 3 "timezone" - tzinfo subclass in order to set up a timezone, which on Python 2 makes - use of a minimal backport of "timezone" in sqlalchemy.util. - - - .. change:: - :tags: bug, orm - :tickets: 4993 - - Setting persistence-related flags on :func:`_orm.relationship` while also - setting viewonly=True will now emit a regular warning, as these flags do - not make sense for a viewonly=True relationship. In particular, the - "cascade" settings have their own warning that is generated based on the - individual values, such as "delete, delete-orphan", that should not apply - to a viewonly relationship. Note however that in the case of "cascade", - these settings are still erroneously taking effect even though the - relationship is set up as "viewonly". In 1.4, all persistence-related - cascade settings will be disallowed on a viewonly=True relationship in - order to resolve this issue. - - .. change:: - :tags: bug, sqlite - :tickets: 5014 - - Fixed issue to workaround SQLite's behavior of assigning "numeric" affinity - to JSON datatypes, first described at :ref:`change_3850`, which returns - scalar numeric JSON values as a number and not as a string that can be JSON - deserialized. The SQLite-specific JSON deserializer now gracefully - degrades for this case as an exception and bypasses deserialization for - single numeric values, as from a JSON perspective they are already - deserialized. - - - - .. change:: - :tags: bug, orm, py3k - :tickets: 4990 - - Fixed issue where when assigning a collection to itself as a slice, the - mutation operation would fail as it would first erase the assigned - collection inadvertently. As an assignment that does not change the - contents should not generate events, the operation is now a no-op. Note - that the fix only applies to Python 3; in Python 2, the ``__setitem__`` - hook isn't called in this case; ``__setslice__`` is used instead which - recreates the list item-by-item in all cases. - - .. change:: - :tags: bug, orm - :tickets: 5034 - - Fixed issue where by if the "begin" of a transaction failed at the Core - engine/connection level, such as due to network error or database is locked - for some transactional recipes, within the context of the :class:`.Session` - procuring that connection from the connection pool and then immediately - returning it, the ORM :class:`.Session` would not close the connection - despite this connection not being stored within the state of that - :class:`.Session`. This would lead to the connection being cleaned out by - the connection pool weakref handler within garbage collection which is an - unpreferred codepath that in some special configurations can emit errors in - standard error. - -.. changelog:: - :version: 1.3.11 - :released: November 11, 2019 - - .. change:: - :tags: bug, mssql - :tickets: 4973 - - Fixed issue in MSSQL dialect where an expression-based OFFSET value in a - SELECT would be rejected, even though the dialect can render this - expression inside of a ROW NUMBER-oriented LIMIT/OFFSET construct. - - - .. change:: - :tags: orm, usecase - :tickets: 4934 - - Added accessor :meth:`_query.Query.is_single_entity` to :class:`_query.Query`, which - will indicate if the results returned by this :class:`_query.Query` will be a - list of ORM entities, or a tuple of entities or column expressions. - SQLAlchemy hopes to improve upon the behavior of single entity / tuples in - future releases such that the behavior would be explicit up front, however - this attribute should be helpful with the current behavior. Pull request - courtesy Patrick Hayes. - - .. change:: - :tags: bug, mysql - :tickets: 4945 - - Added "Connection was killed" message interpreted from the base - pymysql.Error class in order to detect closed connection, based on reports - that this message is arriving via a pymysql.InternalError() object which - indicates pymysql is not handling it correctly. - - .. change:: - :tags: bug, orm - :tickets: 4954 - - The :paramref:`_orm.relationship.omit_join` flag was not intended to be - manually set to True, and will now emit a warning when this occurs. The - omit_join optimization is detected automatically, and the ``omit_join`` - flag was only intended to disable the optimization in the hypothetical case - that the optimization may have interfered with correct results, which has - not been observed with the modern version of this feature. Setting the - flag to True when it is not automatically detected may cause the selectin - load feature to not work correctly when a non-default primary join - condition is in use. - - - .. change:: - :tags: bug, orm - :tickets: 4915 - - A warning is emitted if a primary key value is passed to :meth:`_query.Query.get` - that consists of None for all primary key column positions. Previously, - passing a single None outside of a tuple would raise a ``TypeError`` and - passing a composite None (tuple of None values) would silently pass - through. The fix now coerces the single None into a tuple where it is - handled consistently with the other None conditions. Thanks to Lev - Izraelit for the help with this. - - - .. change:: - :tags: bug, orm - :tickets: 4947 - - The :class:`.BakedQuery` will not cache a query that was modified by a - :meth:`.QueryEvents.before_compile` event, so that compilation hooks that - may be applying ad-hoc modifications to queries will take effect on each - run. In particular this is helpful for events that modify queries used in - lazy loading as well as eager loading such as "select in" loading. In - order to re-enable caching for a query modified by this event, a new - flag ``bake_ok`` is added; see :ref:`baked_with_before_compile` for - details. - - A longer term plan to provide a new form of SQL caching should solve this - kind of issue more comprehensively. - - .. change:: - :tags: bug, tests - :tickets: 4920 - - Fixed test failures which would occur with newer SQLite as of version 3.30 - or greater, due to their addition of nulls ordering syntax as well as new - restrictions on aggregate functions. Pull request courtesy Nils Philippsen. - - - - .. change:: - :tags: bug, installation, windows - :tickets: 4967 - - Added a workaround for a setuptools-related failure that has been observed - as occurring on Windows installations, where setuptools is not correctly - reporting a build error when the MSVC build dependencies are not installed - and therefore not allowing graceful degradation into non C extensions - builds. - - .. change:: - :tags: bug, sql, py3k - :tickets: 4931 - - Changed the ``repr()`` of the :class:`.quoted_name` construct to use - regular string repr() under Python 3, rather than running it through - "backslashreplace" escaping, which can be misleading. - - .. change:: - :tags: bug, oracle, firebird - :tickets: 4931 - - Modified the approach of "name normalization" for the Oracle and Firebird - dialects, which converts from the UPPERCASE-as-case-insensitive convention - of these dialects into lowercase-as-case-insensitive for SQLAlchemy, to not - automatically apply the :class:`.quoted_name` construct to a name that - matches itself under upper or lower case conversion, as is the case for - many non-european characters. All names used within metadata structures - are converted to :class:`.quoted_name` objects in any case; the change - here would only affect the output of some inspection functions. - - .. change:: - :tags: bug, schema - :tickets: 4911 - - Fixed bug where a table that would have a column label overlap with a plain - column name, such as "foo.id AS foo_id" vs. "foo.foo_id", would prematurely - generate the ``._label`` attribute for a column before this overlap could - be detected due to the use of the ``index=True`` or ``unique=True`` flag on - the column in conjunction with the default naming convention of - ``"column_0_label"``. This would then lead to failures when ``._label`` - were used later to generate a bound parameter name, in particular those - used by the ORM when generating the WHERE clause for an UPDATE statement. - The issue has been fixed by using an alternate ``._label`` accessor for DDL - generation that does not affect the state of the :class:`_schema.Column`. The - accessor also bypasses the key-deduplication step as it is not necessary - for DDL, the naming is now consistently ``"_"`` - without any subsequent numeric symbols when used in DDL. - - - - .. change:: - :tags: bug, engine - :tickets: 4902 - - Fixed bug where parameter repr as used in logging and error reporting needs - additional context in order to distinguish between a list of parameters for - a single statement and a list of parameter lists, as the "list of lists" - structure could also indicate a single parameter list where the first - parameter itself is a list, such as for an array parameter. The - engine/connection now passes in an additional boolean indicating how the - parameters should be considered. The only SQLAlchemy backend that expects - arrays as parameters is that of psycopg2 which uses pyformat parameters, - so this issue has not been too apparent, however as other drivers that use - positional gain more features it is important that this be supported. It - also eliminates the need for the parameter repr function to guess based on - the parameter structure passed. - - .. change:: - :tags: usecase, schema - :tickets: 4894 - - Added DDL support for "computed columns"; these are DDL column - specifications for columns that have a server-computed value, either upon - SELECT (known as "virtual") or at the point of which they are INSERTed or - UPDATEd (known as "stored"). Support is established for Postgresql, MySQL, - Oracle SQL Server and Firebird. Thanks to Federico Caselli for lots of work - on this one. - - .. seealso:: - - :ref:`computed_ddl` - - - .. change:: - :tags: bug, engine, postgresql - :tickets: 4955 - - Fixed bug in :class:`_reflection.Inspector` where the cache key generation did not - take into account arguments passed in the form of tuples, such as the tuple - of view name styles to return for the PostgreSQL dialect. This would lead - the inspector to cache too generally for a more specific set of criteria. - The logic has been adjusted to include every keyword element in the cache, - as every argument is expected to be appropriate for a cache else the - caching decorator should be bypassed by the dialect. - - - .. change:: - :tags: bug, mssql - :tickets: 4923 - - Fixed an issue in the :meth:`_engine.Engine.table_names` method where it would - feed the dialect's default schema name back into the dialect level table - function, which in the case of SQL Server would interpret it as a - dot-tokenized schema name as viewed by the mssql dialect, which would - cause the method to fail in the case where the database username actually - had a dot inside of it. In 1.3, this method is still used by the - :meth:`_schema.MetaData.reflect` function so is a prominent codepath. In 1.4, - which is the current master development branch, this issue doesn't exist, - both because :meth:`_schema.MetaData.reflect` isn't using this method nor does the - method pass the default schema name explicitly. The fix nonetheless - guards against the default server name value returned by the dialect from - being interpreted as dot-tokenized name under any circumstances by - wrapping it in quoted_name(). - - .. change:: - :tags: bug, orm - :tickets: 4974 - - Fixed ORM bug where a "secondary" table that referred to a selectable which - in some way would refer to the local primary table would apply aliasing to - both sides of the join condition when a relationship-related join, either - via :meth:`_query.Query.join` or by :func:`_orm.joinedload`, were generated. The - "local" side is now excluded. - - .. change:: - :tags: usecase, sql - :tickets: 4276 - - Added new accessors to expressions of type :class:`_types.JSON` to allow for - specific datatype access and comparison, covering strings, integers, - numeric, boolean elements. This revises the documented approach of - CASTing to string when comparing values, instead adding specific - functionality into the PostgreSQL, SQlite, MySQL dialects to reliably - deliver these basic types in all cases. - - .. seealso:: - - :class:`_types.JSON` - - :meth:`_sqltypes.JSON.Comparator.as_string` - - :meth:`_sqltypes.JSON.Comparator.as_boolean` - - :meth:`_sqltypes.JSON.Comparator.as_float` - - :meth:`_sqltypes.JSON.Comparator.as_integer` - - .. change:: - :tags: usecase, oracle - :tickets: 4799 - - Added dialect-level flag ``encoding_errors`` to the cx_Oracle dialect, - which can be specified as part of :func:`_sa.create_engine`. This is passed - to SQLAlchemy's unicode decoding converter under Python 2, and to - cx_Oracle's ``cursor.var()`` object as the ``encodingErrors`` parameter - under Python 3, for the very unusual case that broken encodings are present - in the target database which cannot be fetched unless error handling is - relaxed. The value is ultimately one of the Python "encoding errors" - parameters passed to ``decode()``. - - .. change:: - :tags: usecase, sql - :tickets: 4933 - - The :func:`_expression.text` construct now supports "unique" bound parameters, which - will dynamically uniquify themselves on compilation thus allowing multiple - :func:`_expression.text` constructs with the same bound parameter names to be combined - together. - - - .. change:: - :tags: bug, oracle - :tickets: 4913 - - The :class:`_types.NCHAR` datatype will now bind to the - ``cx_Oracle.FIXED_NCHAR`` DBAPI data bindings when used in a bound - parameter, which supplies proper comparison behavior against a - variable-length string. Previously, the :class:`_types.NCHAR` datatype - would bind to ``cx_oracle.NCHAR`` which is not fixed length; the - :class:`_types.CHAR` datatype already binds to ``cx_Oracle.FIXED_CHAR`` - so it is now consistent that :class:`_types.NCHAR` binds to - ``cx_Oracle.FIXED_NCHAR``. - - - - .. change:: - :tags: bug, firebird - :tickets: 4903 - - Added additional "disconnect" message "Error writing data to the - connection" to Firebird disconnection detection. Pull request courtesy - lukens. - -.. changelog:: - :version: 1.3.10 - :released: October 9, 2019 - - .. change:: - :tags: bug, mssql - :tickets: 4857 - - Fixed bug in SQL Server dialect with new "max_identifier_length" feature - where the mssql dialect already featured this flag, and the implementation - did not accommodate for the new initialization hook correctly. - - - .. change:: - :tags: bug, oracle - :tickets: 4898, 4857 - - Fixed regression in Oracle dialect that was inadvertently using max - identifier length of 128 characters on Oracle server 12.2 and greater even - though the stated contract for the remainder of the 1.3 series is that - this value stays at 30 until version SQLAlchemy 1.4. Also repaired issues - with the retrieval of the "compatibility" version, and removed the warning - emitted when the "v$parameter" view was not accessible as this was causing - user confusion. - -.. changelog:: - :version: 1.3.9 - :released: October 4, 2019 - - .. change:: - :tags: usecase, engine - :tickets: 4857 - - Added new :func:`_sa.create_engine` parameter - :paramref:`_sa.create_engine.max_identifier_length`. This overrides the - dialect-coded "max identifier length" in order to accommodate for databases - that have recently changed this length and the SQLAlchemy dialect has - not yet been adjusted to detect for that version. This parameter interacts - with the existing :paramref:`_sa.create_engine.label_length` parameter in that - it establishes the maximum (and default) value for anonymously generated - labels. Additionally, post-connection detection of max identifier lengths - has been added to the dialect system. This feature is first being used - by the Oracle dialect. - - .. seealso:: - - :ref:`oracle_max_identifier_lengths` - in the Oracle dialect documentation - - .. change:: - :tags: usecase, oracle - :tickets: 4857 - - The Oracle dialect now emits a warning if Oracle version 12.2 or greater is - used, and the :paramref:`_sa.create_engine.max_identifier_length` parameter is - not set. The version in this specific case defaults to that of the - "compatibility" version set in the Oracle server configuration, not the - actual server version. In version 1.4, the default max_identifier_length - for 12.2 or greater will move to 128 characters. In order to maintain - forwards compatibility, applications should set - :paramref:`_sa.create_engine.max_identifier_length` to 30 in order to maintain - the same length behavior, or to 128 in order to test the upcoming behavior. - This length determines among other things how generated constraint names - are truncated for statements like ``CREATE CONSTRAINT`` and ``DROP - CONSTRAINT``, which means a the new length may produce a name-mismatch - against a name that was generated with the old length, impacting database - migrations. - - .. seealso:: - - :ref:`oracle_max_identifier_lengths` - in the Oracle dialect documentation - - .. change:: - :tags: usecase, sqlite - :tickets: 4863 - - Added support for sqlite "URI" connections, which allow for sqlite-specific - flags to be passed in the query string such as "read only" for Python - sqlite3 drivers that support this. - - .. seealso:: - - :ref:`pysqlite_uri_connections` - - .. change:: - :tags: bug, tests - :tickets: 4285 - - Fixed unit test regression released in 1.3.8 that would cause failure for - Oracle, SQL Server and other non-native ENUM platforms due to new - enumeration tests added as part of :ticket:`4285` enum sortability in the - unit of work; the enumerations created constraints that were duplicated on - name. - - .. change:: - :tags: bug, oracle - :tickets: 4886 - - Restored adding cx_Oracle.DATETIME to the setinputsizes() call when a - SQLAlchemy :class:`.Date`, :class:`.DateTime` or :class:`.Time` datatype is - used, as some complex queries require this to be present. This was removed - in the 1.2 series for arbitrary reasons. - - .. change:: - :tags: bug, mssql - :tickets: 4883 - - Added identifier quoting to the schema name applied to the "use" statement - which is invoked when a SQL Server multipart schema name is used within a - :class:`_schema.Table` that is being reflected, as well as for :class:`_reflection.Inspector` - methods such as :meth:`_reflection.Inspector.get_table_names`; this accommodates for - special characters or spaces in the database name. Additionally, the "use" - statement is not emitted if the current database matches the target owner - database name being passed. - - .. change:: - :tags: bug, orm - :tickets: 4872 - - Fixed regression in selectinload loader strategy caused by :ticket:`4775` - (released in version 1.3.6) where a many-to-one attribute of None would no - longer be populated by the loader. While this was usually not noticeable - due to the lazyloader populating None upon get, it would lead to a detached - instance error if the object were detached. - - .. change:: - :tags: bug, orm - :tickets: 4873 - - Passing a plain string expression to :meth:`.Session.query` is deprecated, - as all string coercions were removed in :ticket:`4481` and this one should - have been included. The :func:`_expression.literal_column` function may be used to - produce a textual column expression. - - .. change:: - :tags: usecase, sql - :tickets: 4847 - - Added an explicit error message for the case when objects passed to - :class:`_schema.Table` are not :class:`.SchemaItem` objects, rather than resolving - to an attribute error. - - - .. change:: - :tags: bug, orm - :tickets: 4890 - - A warning is emitted for a condition in which the :class:`.Session` may - implicitly swap an object out of the identity map for another one with the - same primary key, detaching the old one, which can be an observed result of - load operations which occur within the :meth:`.SessionEvents.after_flush` - hook. The warning is intended to notify the user that some special - condition has caused this to happen and that the previous object may not be - in the expected state. - - .. change:: - :tags: bug, sql - :tickets: 4837 - - Characters that interfere with "pyformat" or "named" formats in bound - parameters, namely ``%, (, )`` and the space character, as well as a few - other typically undesirable characters, are stripped early for a - :func:`.bindparam` that is using an anonymized name, which is typically - generated automatically from a named column which itself includes these - characters in its name and does not use a ``.key``, so that they do not - interfere either with the SQLAlchemy compiler's use of string formatting or - with the driver-level parsing of the parameter, both of which could be - demonstrated before the fix. The change only applies to anonymized - parameter names that are generated and consumed internally, not end-user - defined names, so the change should have no impact on any existing code. - Applies in particular to the psycopg2 driver which does not otherwise quote - special parameter names, but also strips leading underscores to suit Oracle - (but not yet leading numbers, as some anon parameters are currently - entirely numeric/underscore based); Oracle in any case continues to quote - parameter names that include special characters. - -.. changelog:: - :version: 1.3.8 - :released: August 27, 2019 - - .. change:: - :tags: bug, orm - :tickets: 4823 - - Fixed bug where :class:`_orm.Load` objects were not pickleable due to - mapper/relationship state in the internal context dictionary. These - objects are now converted to picklable using similar techniques as that of - other elements within the loader option system that have long been - serializable. - - .. change:: - :tags: bug, postgresql - :tickets: 4623 - - Revised the approach for the just added support for the psycopg2 - "execute_values()" feature added in 1.3.7 for :ticket:`4623`. The approach - relied upon a regular expression that would fail to match for a more - complex INSERT statement such as one which had subqueries involved. The - new approach matches exactly the string that was rendered as the VALUES - clause. - - .. change:: - :tags: usecase, orm - :tickets: 4285 - - Added support for the use of an :class:`.Enum` datatype using Python - pep-435 enumeration objects as values for use as a primary key column - mapped by the ORM. As these values are not inherently sortable, as - required by the ORM for primary keys, a new - :attr:`.TypeEngine.sort_key_function` attribute is added to the typing - system which allows any SQL type to implement a sorting for Python objects - of its type which is consulted by the unit of work. The :class:`.Enum` - type then defines this using the database value of a given enumeration. - The sorting scheme can be also be redefined by passing a callable to the - :paramref:`.Enum.sort_key_function` parameter. Pull request courtesy - Nicolas Caniart. - - .. change:: - :tags: bug, engine - :tickets: 4807 - - Fixed an issue whereby if the dialect "initialize" process which occurs on - first connect would encounter an unexpected exception, the initialize - process would fail to complete and then no longer attempt on subsequent - connection attempts, leaving the dialect in an un-initialized, or partially - initialized state, within the scope of parameters that need to be - established based on inspection of a live connection. The "invoke once" - logic in the event system has been reworked to accommodate for this - occurrence using new, private API features that establish an "exec once" - hook that will continue to allow the initializer to fire off on subsequent - connections, until it completes without raising an exception. This does not - impact the behavior of the existing ``once=True`` flag within the event - system. - - .. change:: - :tags: bug, sqlite, reflection - :tickets: 4810 - - Fixed bug where a FOREIGN KEY that was set up to refer to the parent table - by table name only without the column names would not correctly be - reflected as far as setting up the "referred columns", since SQLite's - PRAGMA does not report on these columns if they weren't given explicitly. - For some reason this was hardcoded to assume the name of the local column, - which might work for some cases but is not correct. The new approach - reflects the primary key of the referred table and uses the constraint - columns list as the referred columns list, if the remote column(s) aren't - present in the reflected pragma directly. - - - .. change:: - :tags: bug, postgresql - :tickets: 4822 - - Fixed bug where Postgresql operators such as - :meth:`.postgresql.ARRAY.Comparator.contains` and - :meth:`.postgresql.ARRAY.Comparator.contained_by` would fail to function - correctly for non-integer values when used against a - :class:`_postgresql.array` object, due to an erroneous assert statement. - - .. change:: - :tags: feature, engine - :tickets: 4815 - - Added new parameter :paramref:`_sa.create_engine.hide_parameters` which when - set to True will cause SQL parameters to no longer be logged, nor rendered - in the string representation of a :class:`.StatementError` object. - - - .. change:: - :tags: usecase, postgresql - :tickets: 4824 - - Added support for reflection of CHECK constraints that include the special - PostgreSQL qualifier "NOT VALID", which can be present for CHECK - constraints that were added to an existing table with the directive that - they not be applied to existing data in the table. The PostgreSQL - dictionary for CHECK constraints as returned by - :meth:`_reflection.Inspector.get_check_constraints` may include an additional entry - ``dialect_options`` which within will contain an entry ``"not_valid": - True`` if this symbol is detected. Pull request courtesy Bill Finn. - -.. changelog:: - :version: 1.3.7 - :released: August 14, 2019 - - .. change:: - :tags: bug, sql - :tickets: 4778 - - Fixed issue where :class:`.Index` object which contained a mixture of - functional expressions which were not resolvable to a particular column, - in combination with string-based column names, would fail to initialize - its internal state correctly leading to failures during DDL compilation. - - .. change:: - :tags: bug, sqlite - :tickets: 4798 - - The dialects that support json are supposed to take arguments - ``json_serializer`` and ``json_deserializer`` at the create_engine() level, - however the SQLite dialect calls them ``_json_serializer`` and - ``_json_deserilalizer``. The names have been corrected, the old names are - accepted with a change warning, and these parameters are now documented as - :paramref:`_sa.create_engine.json_serializer` and - :paramref:`_sa.create_engine.json_deserializer`. - - - .. change:: - :tags: bug, mysql - :tickets: 4804 - - The MySQL dialects will emit "SET NAMES" at the start of a connection when - charset is given to the MySQL driver, to appease an apparent behavior - observed in MySQL 8.0 that raises a collation error when a UNION includes - string columns unioned against columns of the form CAST(NULL AS CHAR(..)), - which is what SQLAlchemy's polymorphic_union function does. The issue - seems to have affected PyMySQL for at least a year, however has recently - appeared as of mysqlclient 1.4.4 based on changes in how this DBAPI creates - a connection. As the presence of this directive impacts three separate - MySQL charset settings which each have intricate effects based on their - presence, SQLAlchemy will now emit the directive on new connections to - ensure correct behavior. - - .. change:: - :tags: usecase, postgresql - :tickets: 4623 - - Added new dialect flag for the psycopg2 dialect, ``executemany_mode`` which - supersedes the previous experimental ``use_batch_mode`` flag. - ``executemany_mode`` supports both the "execute batch" and "execute values" - functions provided by psycopg2, the latter which is used for compiled - :func:`_expression.insert` constructs. Pull request courtesy Yuval Dinari. - - .. seealso:: - - :ref:`psycopg2_executemany_mode` - - - - - .. change:: - :tags: bug, sql - :tickets: 4787 - - Fixed bug where :meth:`.TypeEngine.column_expression` method would not be - applied to subsequent SELECT statements inside of a UNION or other - :class:`_selectable.CompoundSelect`, even though the SELECT statements are rendered at - the topmost level of the statement. New logic now differentiates between - rendering the column expression, which is needed for all SELECTs in the - list, vs. gathering the returned data type for the result row, which is - needed only for the first SELECT. - - .. change:: - :tags: bug, sqlite - :tickets: 4793 - - Fixed bug where usage of "PRAGMA table_info" in SQLite dialect meant that - reflection features to detect for table existence, list of table columns, - and list of foreign keys, would default to any table in any attached - database, when no schema name was given and the table did not exist in the - base schema. The fix explicitly runs PRAGMA for the 'main' schema and then - the 'temp' schema if the 'main' returned no rows, to maintain the behavior - of tables + temp tables in the "no schema" namespace, attached tables only - in the "schema" namespace. - - - .. change:: - :tags: bug, sql - :tickets: 4780 - - Fixed issue where internal cloning of SELECT constructs could lead to a key - error if the copy of the SELECT changed its state such that its list of - columns changed. This was observed to be occurring in some ORM scenarios - which may be unique to 1.3 and above, so is partially a regression fix. - - - - .. change:: - :tags: bug, orm - :tickets: 4777 - - Fixed regression caused by new selectinload for many-to-one logic where - a primaryjoin condition not based on real foreign keys would cause - KeyError if a related object did not exist for a given key value on the - parent object. - - .. change:: - :tags: usecase, mysql - :tickets: 4783 - - Added reserved words ARRAY and MEMBER to the MySQL reserved words list, as - MySQL 8.0 has now made these reserved. - - - .. change:: - :tags: bug, events - :tickets: 4794 - - Fixed issue in event system where using the ``once=True`` flag with - dynamically generated listener functions would cause event registration of - future events to fail if those listener functions were garbage collected - after they were used, due to an assumption that a listened function is - strongly referenced. The "once" wrapped is now modified to strongly - reference the inner function persistently, and documentation is updated - that using "once" does not imply automatic de-registration of listener - functions. - - .. change:: - :tags: bug, mysql - :tickets: 4751 - - Added another fix for an upstream MySQL 8 issue where a case sensitive - table name is reported incorrectly in foreign key constraint reflection, - this is an extension of the fix first added for :ticket:`4344` which - affects a case sensitive column name. The new issue occurs through MySQL - 8.0.17, so the general logic of the 88718 fix remains in place. - - .. seealso:: - - https://bugs.mysql.com/bug.php?id=96365 - upstream bug - - - .. change:: - :tags: usecase, mssql - :tickets: 4782 - - Added new :func:`_mssql.try_cast` construct for SQL Server which emits - "TRY_CAST" syntax. Pull request courtesy Leonel Atencio. - - .. change:: - :tags: bug, orm - :tickets: 4803 - - Fixed bug where using :meth:`_query.Query.first` or a slice expression in - conjunction with a query that has an expression based "offset" applied - would raise TypeError, due to an "or" conditional against "offset" that did - not expect it to be a SQL expression as opposed to an integer or None. - - -.. changelog:: - :version: 1.3.6 - :released: July 21, 2019 - - .. change:: - :tags: bug, engine - :tickets: 4754 - - Fixed bug where using reflection function such as :meth:`_schema.MetaData.reflect` - with an :class:`_engine.Engine` object that had execution options applied to it - would fail, as the resulting :class:`.OptionEngine` proxy object failed to - include a ``.engine`` attribute used within the reflection routines. - - .. change:: - :tags: bug, mysql - :tickets: 4743 - - Fixed bug where the special logic to render "NULL" for the - :class:`_types.TIMESTAMP` datatype when ``nullable=True`` would not work if the - column's datatype were a :class:`.TypeDecorator` or a :class:`.Variant`. - The logic now ensures that it unwraps down to the original - :class:`_types.TIMESTAMP` so that this special case NULL keyword is correctly - rendered when requested. - - .. change:: - :tags: performance, orm - :tickets: 4775 - - The optimization applied to selectin loading in :ticket:`4340` where a JOIN - is not needed to eagerly load related items is now applied to many-to-one - relationships as well, so that only the related table is queried for a - simple join condition. In this case, the related items are queried - based on the value of a foreign key column on the parent; if these columns - are deferred or otherwise not loaded on any of the parent objects in - the collection, the loader falls back to the JOIN method. - - - .. change:: - :tags: bug, orm - :tickets: 4773 - - Fixed regression caused by :ticket:`4365` where a join from an entity to - itself without using aliases no longer raises an informative error message, - instead failing on an assertion. The informative error condition has been - restored. - - - .. change:: - :tags: orm, feature - :tickets: 4736 - - Added new loader option method :meth:`_orm.Load.options` which allows loader - options to be constructed hierarchically, so that many sub-options can be - applied to a particular path without needing to call :func:`.defaultload` - many times. Thanks to Alessio Bogon for the idea. - - - .. change:: - :tags: usecase, postgresql - :tickets: 4771 - - Added support for reflection of indexes on PostgreSQL partitioned tables, - which was added to PostgreSQL as of version 11. - - .. change:: - :tags: bug, mysql - :tickets: 4624 - - Enhanced MySQL/MariaDB version string parsing to accommodate for exotic - MariaDB version strings where the "MariaDB" word is embedded among other - alphanumeric characters such as "MariaDBV1". This detection is critical in - order to correctly accommodate for API features that have split between MySQL - and MariaDB such as the "transaction_isolation" system variable. - - - .. change:: - :tags: bug, mssql - :tickets: 4745 - - Ensured that the queries used to reflect indexes and view definitions will - explicitly CAST string parameters into NVARCHAR, as many SQL Server drivers - frequently treat string values, particularly those with non-ascii - characters or larger string values, as TEXT which often don't compare - correctly against VARCHAR characters in SQL Server's information schema - tables for some reason. These CAST operations already take place for - reflection queries against SQL Server ``information_schema.`` tables but - were missing from three additional queries that are against ``sys.`` - tables. - - .. change:: - :tags: bug, orm - :tickets: 4713 - - Fixed an issue where the :meth:`.orm._ORMJoin.join` method, which is a - not-internally-used ORM-level method that exposes what is normally an - internal process of :meth:`_query.Query.join`, did not propagate the ``full`` and - ``outerjoin`` keyword arguments correctly. Pull request courtesy Denis - Kataev. - - .. change:: - :tags: bug, sql - :tickets: 4758 - - Adjusted the initialization for :class:`.Enum` to minimize how often it - invokes the ``.__members__`` attribute of a given PEP-435 enumeration - object, to suit the case where this attribute is expensive to invoke, as is - the case for some popular third party enumeration libraries. - - - .. change:: - :tags: bug, orm - :tickets: 4772 - - Fixed bug where a many-to-one relationship that specified ``uselist=True`` - would fail to update correctly during a primary key change where a related - column needs to change. - - - .. change:: - :tags: bug, orm - :tickets: 4772 - - Fixed bug where the detection for many-to-one or one-to-one use with a - "dynamic" relationship, which is an invalid configuration, would fail to - raise if the relationship were configured with ``uselist=True``. The - current fix is that it warns, instead of raises, as this would otherwise be - backwards incompatible, however in a future release it will be a raise. - - - .. change:: - :tags: bug, orm - :tickets: 4767 - - Fixed bug where a synonym created against a mapped attribute that does not - exist yet, as is the case when it refers to backref before mappers are - configured, would raise recursion errors when trying to test for attributes - on it which ultimately don't exist (as occurs when the classes are run - through Sphinx autodoc), as the unconfigured state of the synonym would put - it into an attribute not found loop. - - - .. change:: - :tags: usecase, postgresql - :tickets: 4756 - - Added support for multidimensional Postgresql array literals via nesting - the :class:`_postgresql.array` object within another one. The - multidimensional array type is detected automatically. - - .. seealso:: - - :class:`_postgresql.array` - - .. change:: - :tags: bug, sql, postgresql - :tickets: 4760 - - Fixed issue where the :class:`_functions.array_agg` construct in combination with - :meth:`.FunctionElement.filter` would not produce the correct operator - precedence in combination with the array index operator. - - - .. change:: - :tags: bug, sql - :tickets: 4747 - - Fixed an unlikely issue where the "corresponding column" routine for unions - and other :class:`_selectable.CompoundSelect` objects could return the wrong column in - some overlapping column situations, thus potentially impacting some ORM - operations when set operations are in use, if the underlying - :func:`_expression.select` constructs were used previously in other similar kinds of - routines, due to a cached value not being cleared. - - .. change:: - :tags: usecase, sqlite - :tickets: 4766 - - Added support for composite (tuple) IN operators with SQLite, by rendering - the VALUES keyword for this backend. As other backends such as DB2 are - known to use the same syntax, the syntax is enabled in the base compiler - using a dialect-level flag ``tuple_in_values``. The change also includes - support for "empty IN tuple" expressions for SQLite when using "in_()" - between a tuple value and an empty set. - - -.. changelog:: - :version: 1.3.5 - :released: June 17, 2019 - - .. change:: - :tags: bug, mysql - :tickets: 4715 - - Fixed bug where MySQL ON DUPLICATE KEY UPDATE would not accommodate setting - a column to the value NULL. Pull request courtesy Lukáš Banič. - - .. change:: - :tags: bug, orm - :tickets: 4723 - - Fixed a series of related bugs regarding joined table inheritance more than - two levels deep, in conjunction with modification to primary key values, - where those primary key columns are also linked together in a foreign key - relationship as is typical for joined table inheritance. The intermediary - table in a three-level inheritance hierarchy will now get its UPDATE if - only the primary key value has changed and passive_updates=False (e.g. - foreign key constraints not being enforced), whereas before it would be - skipped; similarly, with passive_updates=True (e.g. ON UPDATE CASCADE in - effect), the third-level table will not receive an UPDATE statement as was - the case earlier which would fail since CASCADE already modified it. In a - related issue, a relationship linked to a three-level inheritance hierarchy - on the primary key of an intermediary table of a joined-inheritance - hierarchy will also correctly have its foreign key column updated when the - parent object's primary key is modified, even if that parent object is a - subclass of the linked parent class, whereas before these classes would - not be counted. - - .. change:: - :tags: bug, orm - :tickets: 4729 - - Fixed bug where the :attr:`_orm.Mapper.all_orm_descriptors` accessor would - return an entry for the :class:`_orm.Mapper` itself under the declarative - ``__mapper__`` key, when this is not a descriptor. The ``.is_attribute`` - flag that's present on all :class:`.InspectionAttr` objects is now - consulted, which has also been modified to be ``True`` for an association - proxy, as it was erroneously set to False for this object. - - .. change:: - :tags: bug, orm - :tickets: 4704 - - Fixed regression in :meth:`_query.Query.join` where the ``aliased=True`` flag - would not properly apply clause adaptation to filter criteria, if a - previous join were made to the same entity. This is because the adapters - were placed in the wrong order. The order has been reversed so that the - adapter for the most recent ``aliased=True`` call takes precedence as was - the case in 1.2 and earlier. This broke the "elementtree" examples among - other things. - - .. change:: - :tags: bug, orm, py3k - :tickets: 4674 - - Replaced the Python compatibility routines for ``getfullargspec()`` with a - fully vendored version from Python 3.3. Originally, Python was emitting - deprecation warnings for this function in Python 3.8 alphas. While this - change was reverted, it was observed that Python 3 implementations for - ``getfullargspec()`` are an order of magnitude slower as of the 3.4 series - where it was rewritten against ``Signature``. While Python plans to - improve upon this situation, SQLAlchemy projects for now are using a simple - replacement to avoid any future issues. - - .. change:: - :tags: bug, orm - :tickets: 4694 - - Reworked the attribute mechanics used by :class:`.AliasedClass` to no - longer rely upon calling ``__getattribute__`` on the MRO of the wrapped - class, and to instead resolve the attribute normally on the wrapped class - using getattr(), and then unwrap/adapt that. This allows a greater range - of attribute styles on the mapped class including special ``__getattr__()`` - schemes; but it also makes the code simpler and more resilient in general. - - .. change:: - :tags: usecase, postgresql - :tickets: 4717 - - Added support for column sorting flags when reflecting indexes for - PostgreSQL, including ASC, DESC, NULLSFIRST, NULLSLAST. Also adds this - facility to the reflection system in general which can be applied to other - dialects in future releases. Pull request courtesy Eli Collins. - - .. change:: - :tags: bug, postgresql - :tickets: 4701 - - Fixed bug where PostgreSQL dialect could not correctly reflect an ENUM - datatype that has no members, returning a list with ``None`` for the - ``get_enums()`` call and raising a TypeError when reflecting a column which - has such a datatype. The inspection now returns an empty list. - - .. change:: - :tags: bug, sql - :tickets: 4730 - - Fixed a series of quoting issues which all stemmed from the concept of the - :func:`_expression.literal_column` construct, which when being "proxied" through a - subquery to be referred towards by a label that matches its text, the label - would not have quoting rules applied to it, even if the string in the - :class:`.Label` were set up as a :class:`.quoted_name` construct. Not - applying quoting to the text of the :class:`.Label` is a bug because this - text is strictly a SQL identifier name and not a SQL expression, and the - string should not have quotes embedded into it already unlike the - :func:`_expression.literal_column` which it may be applied towards. The existing - behavior of a non-labeled :func:`_expression.literal_column` being propagated as is on - the outside of a subquery is maintained in order to help with manual - quoting schemes, although it's not clear if valid SQL can be generated for - such a construct in any case. - -.. changelog:: - :version: 1.3.4 - :released: May 27, 2019 - - .. change:: - :tags: feature, mssql - :tickets: 4657 - - Added support for SQL Server filtered indexes, via the ``mssql_where`` - parameter which works similarly to that of the ``postgresql_where`` index - function in the PostgreSQL dialect. - - .. seealso:: - - :ref:`mssql_index_where` - - .. change:: - :tags: bug, misc - :tickets: 4625 - - Removed errant "sqla_nose.py" symbol from MANIFEST.in which created an - undesirable warning message. - - .. change:: - :tags: bug, sql - :tickets: 4653 - - Fixed that the :class:`.GenericFunction` class was inadvertently - registering itself as one of the named functions. Pull request courtesy - Adrien Berchet. - - .. change:: - :tags: bug, engine, postgresql - :tickets: 4663 - - Moved the "rollback" which occurs during dialect initialization so that it - occurs after additional dialect-specific initialize steps, in particular - those of the psycopg2 dialect which would inadvertently leave transactional - state on the first new connection, which could interfere with some - psycopg2-specific APIs which require that no transaction is started. Pull - request courtesy Matthew Wilkes. - - - .. change:: - :tags: bug, orm - :tickets: 4695 - - Fixed issue where the :paramref:`.AttributeEvents.active_history` flag - would not be set for an event listener that propagated to a subclass via the - :paramref:`.AttributeEvents.propagate` flag. This bug has been present - for the full span of the :class:`.AttributeEvents` system. - - - .. change:: - :tags: bug, orm - :tickets: 4690 - - Fixed regression where new association proxy system was still not proxying - hybrid attributes when they made use of the ``@hybrid_property.expression`` - decorator to return an alternate SQL expression, or when the hybrid - returned an arbitrary :class:`.PropComparator`, at the expression level. - This involved further generalization of the heuristics used to detect the - type of object being proxied at the level of :class:`.QueryableAttribute`, - to better detect if the descriptor ultimately serves mapped classes or - column expressions. - - .. change:: - :tags: bug, orm - :tickets: 4686 - - Applied the mapper "configure mutex" against the declarative class mapping - process, to guard against the race which can occur if mappers are used - while dynamic module import schemes are still in the process of configuring - mappers for related classes. This does not guard against all possible race - conditions, such as if the concurrent import has not yet encountered the - dependent classes as of yet, however it guards against as much as possible - within the SQLAlchemy declarative process. - - .. change:: - :tags: bug, mssql - :tickets: 4680 - - Added error code 20047 to "is_disconnect" for pymssql. Pull request - courtesy Jon Schuff. - - - .. change:: - :tags: bug, postgresql, orm - :tickets: 4661 - - Fixed an issue where the "number of rows matched" warning would emit even if - the dialect reported "supports_sane_multi_rowcount=False", as is the case - for psycogp2 with ``use_batch_mode=True`` and others. - - - .. change:: - :tags: bug, sql - :tickets: 4618 - - Fixed issue where double negation of a boolean column wouldn't reset - the "NOT" operator. - - .. change:: - :tags: mysql, bug - :tickets: 4650 - - Added support for DROP CHECK constraint which is required by MySQL 8.0.16 - to drop a CHECK constraint; MariaDB supports plain DROP CONSTRAINT. The - logic distinguishes between the two syntaxes by checking the server version - string for MariaDB presence. Alembic migrations has already worked - around this issue by implementing its own DROP for MySQL / MariaDB CHECK - constraints, however this change implements it straight in Core so that its - available for general use. Pull request courtesy Hannes Hansen. - - .. change:: - :tags: bug, orm - :tickets: 4647 - - A warning is now emitted for the case where a transient object is being - merged into the session with :meth:`.Session.merge` when that object is - already transient in the :class:`.Session`. This warns for the case where - the object would normally be double-inserted. - - - .. change:: - :tags: bug, orm - :tickets: 4676 - - Fixed regression in new relationship m2o comparison logic first introduced - at :ref:`change_4359` when comparing to an attribute that is persisted as - NULL and is in an un-fetched state in the mapped instance. Since the - attribute has no explicit default, it needs to default to NULL when - accessed in a persistent setting. - - - .. change:: - :tags: bug, sql - :tickets: 4569 - - The :class:`.GenericFunction` namespace is being migrated so that function - names are looked up in a case-insensitive manner, as SQL functions do not - collide on case sensitive differences nor is this something which would - occur with user-defined functions or stored procedures. Lookups for - functions declared with :class:`.GenericFunction` now use a case - insensitive scheme, however a deprecation case is supported which allows - two or more :class:`.GenericFunction` objects with the same name of - different cases to exist, which will cause case sensitive lookups to occur - for that particular name, while emitting a warning at function registration - time. Thanks to Adrien Berchet for a lot of work on this complicated - feature. - - -.. changelog:: - :version: 1.3.3 - :released: April 15, 2019 - - .. change:: - :tags: bug, postgresql - :tickets: 4601 - - Fixed regression from release 1.3.2 caused by :ticket:`4562` where a URL - that contained only a query string and no hostname, such as for the - purposes of specifying a service file with connection information, would no - longer be propagated to psycopg2 properly. The change in :ticket:`4562` - has been adjusted to further suit psycopg2's exact requirements, which is - that if there are any connection parameters whatsoever, the "dsn" parameter - is no longer required, so in this case the query string parameters are - passed alone. - - .. change:: - :tags: bug, pool - :tickets: 4585 - - Fixed behavioral regression as a result of deprecating the "use_threadlocal" - flag for :class:`_pool.Pool`, where the :class:`.SingletonThreadPool` no longer - makes use of this option which causes the "rollback on return" logic to take - place when the same :class:`_engine.Engine` is used multiple times in the context - of a transaction to connect or implicitly execute, thereby cancelling the - transaction. While this is not the recommended way to work with engines - and connections, it is nonetheless a confusing behavioral change as when - using :class:`.SingletonThreadPool`, the transaction should stay open - regardless of what else is done with the same engine in the same thread. - The ``use_threadlocal`` flag remains deprecated however the - :class:`.SingletonThreadPool` now implements its own version of the same - logic. - - - .. change:: - :tags: bug, orm - :tickets: 4584 - - Fixed 1.3 regression in new "ambiguous FROMs" query logic introduced in - :ref:`change_4365` where a :class:`_query.Query` that explicitly places an entity - in the FROM clause with :meth:`_query.Query.select_from` and also joins to it - using :meth:`_query.Query.join` would later cause an "ambiguous FROM" error if - that entity were used in additional joins, as the entity appears twice in - the "from" list of the :class:`_query.Query`. The fix resolves this ambiguity by - folding the standalone entity into the join that it's already a part of in - the same way that ultimately happens when the SELECT statement is rendered. - - .. change:: - :tags: bug, ext - :tickets: 4603 - - Fixed bug where using ``copy.copy()`` or ``copy.deepcopy()`` on - :class:`.MutableList` would cause the items within the list to be - duplicated, due to an inconsistency in how Python pickle and copy both make - use of ``__getstate__()`` and ``__setstate__()`` regarding lists. In order - to resolve, a ``__reduce_ex__`` method had to be added to - :class:`.MutableList`. In order to maintain backwards compatibility with - existing pickles based on ``__getstate__()``, the ``__setstate__()`` method - remains as well; the test suite asserts that pickles made against the old - version of the class can still be deserialized by the pickle module. - - .. change:: - :tags: bug, orm - :tickets: 4606 - - Adjusted the :meth:`_query.Query.filter_by` method to not call :func:`.and()` - internally against multiple criteria, instead passing it off to - :meth:`_query.Query.filter` as a series of criteria, instead of a single criteria. - This allows :meth:`_query.Query.filter_by` to defer to :meth:`_query.Query.filter`'s - treatment of variable numbers of clauses, including the case where the list - is empty. In this case, the :class:`_query.Query` object will not have a - ``.whereclause``, which allows subsequent "no whereclause" methods like - :meth:`_query.Query.select_from` to behave consistently. - - .. change:: - :tags: bug, mssql - :tickets: 4587 - - Fixed issue in SQL Server dialect where if a bound parameter were present in - an ORDER BY expression that would ultimately not be rendered in the SQL - Server version of the statement, the parameters would still be part of the - execution parameters, leading to DBAPI-level errors. Pull request courtesy - Matt Lewellyn. - -.. changelog:: - :version: 1.3.2 - :released: April 2, 2019 - - .. change:: - :tags: bug, documentation, sql - :tickets: 4580 - - Thanks to :ref:`change_3981`, we no longer need to rely on recipes that - subclass dialect-specific types directly, :class:`.TypeDecorator` can now - handle all cases. Additionally, the above change made it slightly less - likely that a direct subclass of a base SQLAlchemy type would work as - expected, which could be misleading. Documentation has been updated to use - :class:`.TypeDecorator` for these examples including the PostgreSQL - "ArrayOfEnum" example datatype and direct support for the "subclass a type - directly" has been removed. - - .. change:: - :tags: bug, postgresql - :tickets: 4550 - - Modified the :paramref:`.Select.with_for_update.of` parameter so that if a - join or other composed selectable is passed, the individual :class:`_schema.Table` - objects will be filtered from it, allowing one to pass a join() object to - the parameter, as occurs normally when using joined table inheritance with - the ORM. Pull request courtesy Raymond Lu. - - - .. change:: - :tags: feature, postgresql - :tickets: 4562 - - Added support for parameter-less connection URLs for the psycopg2 dialect, - meaning, the URL can be passed to :func:`_sa.create_engine` as - ``"postgresql+psycopg2://"`` with no additional arguments to indicate an - empty DSN passed to libpq, which indicates to connect to "localhost" with - no username, password, or database given. Pull request courtesy Julian - Mehnle. - - .. change:: - :tags: bug, orm, ext - :tickets: 4574, 4573 - - Restored instance-level support for plain Python descriptors, e.g. - ``@property`` objects, in conjunction with association proxies, in that if - the proxied object is not within ORM scope at all, it gets classified as - "ambiguous" but is proxed directly. For class level access, a basic class - level``__get__()`` now returns the - :class:`.AmbiguousAssociationProxyInstance` directly, rather than raising - its exception, which is the closest approximation to the previous behavior - that returned the :class:`.AssociationProxy` itself that's possible. Also - improved the stringification of these objects to be more descriptive of - current state. - - .. change:: - :tags: bug, orm - :tickets: 4537 - - Fixed bug where use of :func:`.with_polymorphic` or other aliased construct - would not properly adapt when the aliased target were used as the - :meth:`_expression.Select.correlate_except` target of a subquery used inside of a - :func:`.column_property`. This required a fix to the clause adaption - mechanics to properly handle a selectable that shows up in the "correlate - except" list, in a similar manner as which occurs for selectables that show - up in the "correlate" list. This is ultimately a fairly fundamental bug - that has lasted for a long time but it is hard to come across it. - - - .. change:: - :tags: bug, orm - :tickets: 4566 - - Fixed regression where a new error message that was supposed to raise when - attempting to link a relationship option to an AliasedClass without using - :meth:`.PropComparator.of_type` would instead raise an ``AttributeError``. - Note that in 1.3, it is no longer valid to create an option path from a - plain mapper relationship to an :class:`.AliasedClass` without using - :meth:`.PropComparator.of_type`. - -.. changelog:: - :version: 1.3.1 - :released: March 9, 2019 - - .. change:: - :tags: bug, mssql - :tickets: 4525 - - Fixed regression in SQL Server reflection due to :ticket:`4393` where the - removal of open-ended ``**kw`` from the :class:`.Float` datatype caused - reflection of this type to fail due to a "scale" argument being passed. - - .. change:: - :tags: bug, orm, ext - :tickets: 4522 - - Fixed regression where an association proxy linked to a synonym would no - longer work, both at instance level and at class level. - -.. changelog:: - :version: 1.3.0 - :released: March 4, 2019 - - .. change:: - :tags: feature, schema - :tickets: 4517 - - Added new parameters :paramref:`_schema.Table.resolve_fks` and - :paramref:`.MetaData.reflect.resolve_fks` which when set to False will - disable the automatic reflection of related tables encountered in - :class:`_schema.ForeignKey` objects, which can both reduce SQL overhead for omitted - tables as well as avoid tables that can't be reflected for database-specific - reasons. Two :class:`_schema.Table` objects present in the same :class:`_schema.MetaData` - collection can still refer to each other even if the reflection of the two - tables occurred separately. - - - .. change:: - :tags: feature, orm - :tickets: 4316 - - The :meth:`_query.Query.get` method can now accept a dictionary of attribute keys - and values as a means of indicating the primary key value to load; is - particularly useful for composite primary keys. Pull request courtesy - Sanjana S. - - .. change:: - :tags: feature, orm - :tickets: 3133 - - A SQL expression can now be assigned to a primary key attribute for an ORM - flush in the same manner as ordinary attributes as described in - :ref:`flush_embedded_sql_expressions` where the expression will be evaluated - and then returned to the ORM using RETURNING, or in the case of pysqlite, - works using the cursor.lastrowid attribute.Requires either a database that - supports RETURNING (e.g. Postgresql, Oracle, SQL Server) or pysqlite. - - .. change:: - :tags: bug, sql - :tickets: 4509 - - The :class:`_expression.Alias` class and related subclasses :class:`_expression.CTE`, - :class:`_expression.Lateral` and :class:`_expression.TableSample` have been reworked so that it is - not possible for a user to construct the objects directly. These constructs - require that the standalone construction function or selectable-bound method - be used to instantiate new objects. - - - .. change:: - :tags: feature, engine - :tickets: 4500 - - Revised the formatting for :class:`.StatementError` when stringified. Each - error detail is broken up over multiple newlines instead of spaced out on a - single line. Additionally, the SQL representation now stringifies the SQL - statement rather than using ``repr()``, so that newlines are rendered as is. - Pull request courtesy Nate Clark. - - .. seealso:: - - :ref:`change_4500` - -.. changelog:: - :version: 1.3.0b3 - :released: March 4, 2019 - :released: February 8, 2019 - - .. change:: - :tags: bug, ext - :tickets: 2642 - - Implemented a more comprehensive assignment operation (e.g. "bulk replace") - when using association proxy with sets or dictionaries. Fixes the problem - of redundant proxy objects being created to replace the old ones, which - leads to excessive events and SQL and in the case of unique constraints - will cause the flush to fail. - - .. seealso:: - - :ref:`change_2642` - - .. change:: - :tags: bug, postgresql - :tickets: 4473 - - Fixed issue where using an uppercase name for an index type (e.g. GIST, - BTREE, etc. ) or an EXCLUDE constraint would treat it as an identifier to - be quoted, rather than rendering it as is. The new behavior converts these - types to lowercase and ensures they contain only valid SQL characters. - - .. change:: - :tags: bug, orm - :tickets: 4469 - - Improved the behavior of :func:`_orm.with_polymorphic` in conjunction with - loader options, in particular wildcard operations as well as - :func:`_orm.load_only`. The polymorphic object will be more accurately - targeted so that column-level options on the entity will correctly take - effect.The issue is a continuation of the same kinds of things fixed in - :ticket:`4468`. - - - .. change:: - :tags: bug, sql - :tickets: 4481 - - Fully removed the behavior of strings passed directly as components of a - :func:`_expression.select` or :class:`_query.Query` object being coerced to :func:`_expression.text` - constructs automatically; the warning that has been emitted is now an - ArgumentError or in the case of order_by() / group_by() a CompileError. - This has emitted a warning since version 1.0 however its presence continues - to create concerns for the potential of mis-use of this behavior. - - Note that public CVEs have been posted for order_by() / group_by() which - are resolved by this commit: CVE-2019-7164 CVE-2019-7548 - - - .. seealso:: - - :ref:`change_4481` - - .. change:: - :tags: bug, sql - :tickets: 4467 - - Quoting is applied to :class:`.Function` names, those which are usually but - not necessarily generated from the :attr:`_expression.func` construct, at compile - time if they contain illegal characters, such as spaces or punctuation. The - names are as before treated as case insensitive however, meaning if the - names contain uppercase or mixed case characters, that alone does not - trigger quoting. The case insensitivity is currently maintained for - backwards compatibility. - - - .. change:: - :tags: bug, sql - :tickets: 4481 - - Added "SQL phrase validation" to key DDL phrases that are accepted as plain - strings, including :paramref:`_schema.ForeignKeyConstraint.on_delete`, - :paramref:`_schema.ForeignKeyConstraint.on_update`, - :paramref:`.ExcludeConstraint.using`, - :paramref:`_schema.ForeignKeyConstraint.initially`, for areas where a series of SQL - keywords only are expected.Any non-space characters that suggest the phrase - would need to be quoted will raise a :class:`.CompileError`. This change - is related to the series of changes committed as part of :ticket:`4481`. - - .. change:: - :tags: bug, orm, declarative - :tickets: 4470 - - Added some helper exceptions that invoke when a mapping based on - :class:`.AbstractConcreteBase`, :class:`.DeferredReflection`, or - :class:`.AutoMap` is used before the mapping is ready to be used, which - contain descriptive information on the class, rather than falling through - into other failure modes that are less informative. - - - .. change:: - :tags: change, tests - :tickets: 4460 - - The test system has removed support for Nose, which is unmaintained for - several years and is producing warnings under Python 3. The test suite is - currently standardized on Pytest. Pull request courtesy Parth Shandilya. - -.. changelog:: - :version: 1.3.0b2 - :released: March 4, 2019 - :released: January 25, 2019 - - .. change:: - :tags: bug, ext - :tickets: 4401 - - Fixed a regression in 1.3.0b1 caused by :ticket:`3423` where association - proxy objects that access an attribute that's only present on a polymorphic - subclass would raise an ``AttributeError`` even though the actual instance - being accessed was an instance of that subclass. - - .. change:: - :tags: bug, orm - :tickets: 1103 - - Fixed long-standing issue where duplicate collection members would cause a - backref to delete the association between the member and its parent object - when one of the duplicates were removed, as occurs as a side effect of - swapping two objects in one statement. - - .. seealso:: - - :ref:`change_1103` - - .. change:: - :tags: bug, mssql - :tickets: 4442 - - The ``literal_processor`` for the :class:`.Unicode` and - :class:`.UnicodeText` datatypes now render an ``N`` character in front of - the literal string expression as required by SQL Server for Unicode string - values rendered in SQL expressions. - - .. change:: - :tags: feature, orm - :tickets: 4423 - - Implemented a new feature whereby the :class:`.AliasedClass` construct can - now be used as the target of a :func:`_orm.relationship`. This allows the - concept of "non primary mappers" to no longer be necessary, as the - :class:`.AliasedClass` is much easier to configure and automatically inherits - all the relationships of the mapped class, as well as preserves the - ability for loader options to work normally. - - .. seealso:: - - :ref:`change_4423` - - .. change:: - :tags: bug, orm - :tickets: 4373 - - Extended the fix first made as part of :ticket:`3287`, where a loader option - made against a subclass using a wildcard would extend itself to include - application of the wildcard to attributes on the super classes as well, to a - "bound" loader option as well, e.g. in an expression like - ``Load(SomeSubClass).load_only('foo')``. Columns that are part of the - parent class of ``SomeSubClass`` will also be excluded in the same way as if - the unbound option ``load_only('foo')`` were used. - - .. change:: - :tags: bug, orm - :tickets: 4433 - - Improved error messages emitted by the ORM in the area of loader option - traversal. This includes early detection of mis-matched loader strategies - along with a clearer explanation why these strategies don't match. - - - .. change:: - :tags: change, orm - :tickets: 4412 - - Added a new function :func:`.close_all_sessions` which takes - over the task of the :meth:`.Session.close_all` method, which - is now deprecated as this is confusing as a classmethod. - Pull request courtesy Augustin Trancart. - - .. change:: - :tags: feature, orm - :tickets: 4397 - - Added new :meth:`.MapperEvents.before_mapper_configured` event. This - event complements the other "configure" stage mapper events with a per - mapper event that receives each :class:`_orm.Mapper` right before its - configure step, and additionally may be used to prevent or delay the - configuration of specific :class:`_orm.Mapper` objects using a new - return value :attr:`.orm.interfaces.EXT_SKIP`. See the - documentation link for an example. - - .. seealso:: - - :meth:`.MapperEvents.before_mapper_configured` - - - - .. change:: - :tags: bug, orm - - The "remove" event for collections is now called before the item is removed - in the case of the ``collection.remove()`` method, as is consistent with the - behavior for most other forms of collection item removal (such as - ``__delitem__``, replacement under ``__setitem__``). For ``pop()`` methods, - the remove event still fires after the operation. - - .. change:: - :tags: bug, orm declarative - :tickets: 4372 - - Added a ``__clause_element__()`` method to :class:`.ColumnProperty` which - can allow the usage of a not-fully-declared column or deferred attribute in - a declarative mapped class slightly more friendly when it's used in a - constraint or other column-oriented scenario within the class declaration, - though this still can't work in open-ended expressions; prefer to call the - :attr:`.ColumnProperty.expression` attribute if receiving ``TypeError``. - - .. change:: - :tags: bug, orm, engine - :tickets: 4464 - - Added accessors for execution options to Core and ORM, via - :meth:`_query.Query.get_execution_options`, - :meth:`_engine.Connection.get_execution_options`, - :meth:`_engine.Engine.get_execution_options`, and - :meth:`.Executable.get_execution_options`. PR courtesy Daniel Lister. - - .. change:: - :tags: bug, orm - :tickets: 4446 - - Fixed issue in association proxy due to :ticket:`3423` which caused the use - of custom :class:`.PropComparator` objects with hybrid attributes, such as - the one demonstrated in the ``dictlike-polymorphic`` example to not - function within an association proxy. The strictness that was added in - :ticket:`3423` has been relaxed, and additional logic to accommodate for - an association proxy that links to a custom hybrid have been added. - - .. change:: - :tags: change, general - :tickets: 4393 - - A large change throughout the library has ensured that all objects, - parameters, and behaviors which have been noted as deprecated or legacy now - emit ``DeprecationWarning`` warnings when invoked.As the Python 3 - interpreter now defaults to displaying deprecation warnings, as well as that - modern test suites based on tools like tox and pytest tend to display - deprecation warnings, this change should make it easier to note what API - features are obsolete. A major rationale for this change is so that long- - deprecated features that nonetheless still see continue to see real world - use can finally be removed in the near future; the biggest example of this - are the :class:`.SessionExtension` and :class:`.MapperExtension` classes as - well as a handful of other pre-event extension hooks, which have been - deprecated since version 0.7 but still remain in the library. Another is - that several major longstanding behaviors are to be deprecated as well, - including the threadlocal engine strategy, the convert_unicode flag, and non - primary mappers. - - .. seealso:: - - :ref:`change_4393_general` - - - .. change:: - :tags: change, engine - :tickets: 4393 - - The "threadlocal" engine strategy which has been a legacy feature of - SQLAlchemy since around version 0.2 is now deprecated, along with the - :paramref:`_pool.Pool.threadlocal` parameter of :class:`_pool.Pool` which has no - effect in most modern use cases. - - .. seealso:: - - :ref:`change_4393_threadlocal` - - .. change:: - :tags: change, sql - :tickets: 4393 - - The :paramref:`_sa.create_engine.convert_unicode` and - :paramref:`.String.convert_unicode` parameters have been deprecated. These - parameters were built back when most Python DBAPIs had little to no support - for Python Unicode objects, and SQLAlchemy needed to take on the very - complex task of marshalling data and SQL strings between Unicode and - bytestrings throughout the system in a performant way. Thanks to Python 3, - DBAPIs were compelled to adapt to Unicode-aware APIs and today all DBAPIs - supported by SQLAlchemy support Unicode natively, including on Python 2, - allowing this long-lived and very complicated feature to finally be (mostly) - removed. There are still of course a few Python 2 edge cases where - SQLAlchemy has to deal with Unicode however these are handled automatically; - in modern use, there should be no need for end-user interaction with these - flags. - - .. seealso:: - - :ref:`change_4393_convertunicode` - - .. change:: - :tags: bug, orm - :tickets: 3777 - - Implemented the ``.get_history()`` method, which also implies availability - of :attr:`.AttributeState.history`, for :func:`.synonym` attributes. - Previously, trying to access attribute history via a synonym would raise an - ``AttributeError``. - - .. change:: - :tags: feature, engine - :tickets: 3689 - - Added public accessor :meth:`.QueuePool.timeout` that returns the configured - timeout for a :class:`.QueuePool` object. Pull request courtesy Irina Delamare. - - .. change:: - :tags: feature, sql - :tickets: 4386 - - Amended the :class:`.AnsiFunction` class, the base of common SQL - functions like ``CURRENT_TIMESTAMP``, to accept positional arguments - like a regular ad-hoc function. This to suit the case that many of - these functions on specific backends accept arguments such as - "fractional seconds" precision and such. If the function is created - with arguments, it renders the parenthesis and the arguments. If - no arguments are present, the compiler generates the non-parenthesized form. - -.. changelog:: - :version: 1.3.0b1 - :released: March 4, 2019 - :released: November 16, 2018 - - .. change:: - :tags: bug, ext - :tickets: 3423 - - Reworked :class:`.AssociationProxy` to store state that's specific to a - parent class in a separate object, so that a single - :class:`.AssociationProxy` can serve for multiple parent classes, as is - intrinsic to inheritance, without any ambiguity in the state returned by it. - A new method :meth:`.AssociationProxy.for_class` is added to allow - inspection of class-specific state. - - .. seealso:: - - :ref:`change_3423` - - - .. change:: - :tags: bug, oracle - :tickets: 4369 - - Updated the parameters that can be sent to the cx_Oracle DBAPI to both allow - for all current parameters as well as for future parameters not added yet. - In addition, removed unused parameters that were deprecated in version 1.2, - and additionally we are now defaulting "threaded" to False. - - .. seealso:: - - :ref:`change_4369` - - .. change:: - :tags: bug, oracle - :tickets: 4242 - - The Oracle dialect will no longer use the NCHAR/NCLOB datatypes - represent generic unicode strings or clob fields in conjunction with - :class:`.Unicode` and :class:`.UnicodeText` unless the flag - ``use_nchar_for_unicode=True`` is passed to :func:`_sa.create_engine` - - this includes CREATE TABLE behavior as well as ``setinputsizes()`` for - bound parameters. On the read side, automatic Unicode conversion under - Python 2 has been added to CHAR/VARCHAR/CLOB result rows, to match the - behavior of cx_Oracle under Python 3. In order to mitigate the performance - hit under Python 2, SQLAlchemy's very performant (when C extensions - are built) native Unicode handlers are used under Python 2. - - .. seealso:: - - :ref:`change_4242` - - .. change:: - :tags: bug, orm - :tickets: 3844 - - Fixed issue regarding passive_deletes="all", where the foreign key - attribute of an object is maintained with its value even after the object - is removed from its parent collection. Previously, the unit of work would - set this to NULL even though passive_deletes indicated it should not be - modified. - - .. seealso:: - - :ref:`change_3844` - - .. change:: - :tags: bug, ext - :tickets: 4268 - - The long-standing behavior of the association proxy collection maintaining - only a weak reference to the parent object is reverted; the proxy will now - maintain a strong reference to the parent for as long as the proxy - collection itself is also in memory, eliminating the "stale association - proxy" error. This change is being made on an experimental basis to see if - any use cases arise where it causes side effects. - - .. seealso:: - - :ref:`change_4268` - - - .. change:: - :tags: bug, sql - :tickets: 4302 - - Added "like" based operators as "comparison" operators, including - :meth:`.ColumnOperators.startswith` :meth:`.ColumnOperators.endswith` - :meth:`.ColumnOperators.ilike` :meth:`.ColumnOperators.notilike` among many - others, so that all of these operators can be the basis for an ORM - "primaryjoin" condition. - - - .. change:: - :tags: feature, sqlite - :tickets: 3850 - - Added support for SQLite's json functionality via the new - SQLite implementation for :class:`_types.JSON`, :class:`_sqlite.JSON`. - The name used for the type is ``JSON``, following an example found at - SQLite's own documentation. Pull request courtesy Ilja Everilä. - - .. seealso:: - - :ref:`change_3850` - - .. change:: - :tags: feature, engine - - Added new "lifo" mode to :class:`.QueuePool`, typically enabled by setting - the flag :paramref:`_sa.create_engine.pool_use_lifo` to True. "lifo" mode - means the same connection just checked in will be the first to be checked - out again, allowing excess connections to be cleaned up from the server - side during periods of the pool being only partially utilized. Pull request - courtesy Taem Park. - - .. seealso:: - - :ref:`change_pr467` - - .. change:: - :tags: bug, orm - :tickets: 4359 - - Improved the behavior of a relationship-bound many-to-one object expression - such that the retrieval of column values on the related object are now - resilient against the object being detached from its parent - :class:`.Session`, even if the attribute has been expired. New features - within the :class:`.InstanceState` are used to memoize the last known value - of a particular column attribute before its expired, so that the expression - can still evaluate when the object is detached and expired at the same - time. Error conditions are also improved using modern attribute state - features to produce more specific messages as needed. - - .. seealso:: - - :ref:`change_4359` - - .. change:: - :tags: feature, mysql - :tickets: 4219 - - Support added for the "WITH PARSER" syntax of CREATE FULLTEXT INDEX - in MySQL, using the ``mysql_with_parser`` keyword argument. Reflection - is also supported, which accommodates MySQL's special comment format - for reporting on this option as well. Additionally, the "FULLTEXT" and - "SPATIAL" index prefixes are now reflected back into the ``mysql_prefix`` - index option. - - - - .. change:: - :tags: bug, orm, mysql, postgresql - :tickets: 4246 - - The ORM now doubles the "FOR UPDATE" clause within the subquery that - renders in conjunction with joined eager loading in some cases, as it has - been observed that MySQL does not lock the rows from a subquery. This - means the query renders with two FOR UPDATE clauses; note that on some - backends such as Oracle, FOR UPDATE clauses on subqueries are silently - ignored since they are unnecessary. Additionally, in the case of the "OF" - clause used primarily with PostgreSQL, the FOR UPDATE is rendered only on - the inner subquery when this is used so that the selectable can be targeted - to the table within the SELECT statement. - - .. seealso:: - - :ref:`change_4246` - - .. change:: - :tags: feature, mssql - :tickets: 4158 - - Added ``fast_executemany=True`` parameter to the SQL Server pyodbc dialect, - which enables use of pyodbc's new performance feature of the same name - when using Microsoft ODBC drivers. - - .. seealso:: - - :ref:`change_4158` - - .. change:: - :tags: bug, ext - :tickets: 4308 - - Fixed multiple issues regarding de-association of scalar objects with the - association proxy. ``del`` now works, and additionally a new flag - :paramref:`.AssociationProxy.cascade_scalar_deletes` is added, which when - set to True indicates that setting a scalar attribute to ``None`` or - deleting via ``del`` will also set the source association to ``None``. - - .. seealso:: - - :ref:`change_4308` - - - .. change:: - :tags: feature, ext - :tickets: 4318 - - Added new feature :meth:`.BakedQuery.to_query`, which allows for a - clean way of using one :class:`.BakedQuery` as a subquery inside of another - :class:`.BakedQuery` without needing to refer explicitly to a - :class:`.Session`. - - - .. change:: - :tags: feature, sqlite - :tickets: 4360 - - Implemented the SQLite ``ON CONFLICT`` clause as understood at the DDL - level, e.g. for primary key, unique, and CHECK constraints as well as - specified on a :class:`_schema.Column` to satisfy inline primary key and NOT NULL. - Pull request courtesy Denis Kataev. - - .. seealso:: - - :ref:`change_4360` - - .. change:: - :tags: feature, postgresql - :tickets: 4237 - - Added rudimental support for reflection of PostgreSQL - partitioned tables, e.g. that relkind='p' is added to reflection - queries that return table information. - - .. seealso:: - - :ref:`change_4237` - - .. change:: - :tags: feature, ext - :tickets: 4351 - - The :class:`.AssociationProxy` now has standard column comparison operations - such as :meth:`.ColumnOperators.like` and - :meth:`.ColumnOperators.startswith` available when the target attribute is a - plain column - the EXISTS expression that joins to the target table is - rendered as usual, but the column expression is then use within the WHERE - criteria of the EXISTS. Note that this alters the behavior of the - ``.contains()`` method on the association proxy to make use of - :meth:`.ColumnOperators.contains` when used on a column-based attribute. - - .. seealso:: - - :ref:`change_4351` - - - .. change:: - :tags: feature, orm - - Added new flag :paramref:`.Session.bulk_save_objects.preserve_order` to the - :meth:`.Session.bulk_save_objects` method, which defaults to True. When set - to False, the given mappings will be grouped into inserts and updates per - each object type, to allow for greater opportunities to batch common - operations together. Pull request courtesy Alessandro Cucci. - - .. change:: - :tags: bug, orm - :tickets: 4365 - - Refactored :meth:`_query.Query.join` to further clarify the individual components - of structuring the join. This refactor adds the ability for - :meth:`_query.Query.join` to determine the most appropriate "left" side of the - join when there is more than one element in the FROM list or the query is - against multiple entities. If more than one FROM/entity matches, an error - is raised that asks for an ON clause to be specified to resolve the - ambiguity. In particular this targets the regression we saw in - :ticket:`4363` but is also of general use. The codepaths within - :meth:`_query.Query.join` are now easier to follow and the error cases are - decided more specifically at an earlier point in the operation. - - .. seealso:: - - :ref:`change_4365` - - .. change:: - :tags: bug, sql - :tickets: 3981 - - Fixed issue with :meth:`.TypeEngine.bind_expression` and - :meth:`.TypeEngine.column_expression` methods where these methods would not - work if the target type were part of a :class:`.Variant`, or other target - type of a :class:`.TypeDecorator`. Additionally, the SQL compiler now - calls upon the dialect-level implementation when it renders these methods - so that dialects can now provide for SQL-level processing for built-in - types. - - .. seealso:: - - :ref:`change_3981` - - - .. change:: - :tags: bug, orm - :tickets: 4304 - - Fixed long-standing issue in :class:`_query.Query` where a scalar subquery such - as produced by :meth:`_query.Query.exists`, :meth:`_query.Query.as_scalar` and other - derivations from :attr:`_query.Query.statement` would not correctly be adapted - when used in a new :class:`_query.Query` that required entity adaptation, such as - when the query were turned into a union, or a from_self(), etc. The change - removes the "no adaptation" annotation from the :func:`_expression.select` object - produced by the :attr:`_query.Query.statement` accessor. - - .. change:: - :tags: bug, orm, declarative - :tickets: 4133 - - Fixed bug where declarative would not update the state of the - :class:`_orm.Mapper` as far as what attributes were present, when additional - attributes were added or removed after the mapper attribute collections had - already been called and memoized. Additionally, a ``NotImplementedError`` - is now raised if a fully mapped attribute (e.g. column, relationship, etc.) - is deleted from a class that is currently mapped, since the mapper will not - function correctly if the attribute has been removed. - - .. change:: - :tags: bug, mssql - :tickets: 4362 - - Deprecated the use of :class:`.Sequence` with SQL Server in order to affect - the "start" and "increment" of the IDENTITY value, in favor of new - parameters ``mssql_identity_start`` and ``mssql_identity_increment`` which - set these parameters directly. :class:`.Sequence` will be used to generate - real ``CREATE SEQUENCE`` DDL with SQL Server in a future release. - - .. seealso:: - - :ref:`change_4362` - - - .. change:: - :tags: feature, mysql - - Added support for the parameters in an ON DUPLICATE KEY UPDATE statement on - MySQL to be ordered, since parameter order in a MySQL UPDATE clause is - significant, in a similar manner as that described at - :ref:`tutorial_parameter_ordered_updates`. Pull request courtesy Maxim Bublis. - - .. seealso:: - - :ref:`change_mysql_ondupordering` - - .. change:: - :tags: feature, sql - :tickets: 4144 - - Added :class:`.Sequence` to the "string SQL" system that will render a - meaningful string expression (``""``) - when stringifying without a dialect a statement that includes a "sequence - nextvalue" expression, rather than raising a compilation error. - - - - .. change:: - :tags: bug, orm - :tickets: 4232 - - An informative exception is re-raised when a primary key value is not - sortable in Python during an ORM flush under Python 3, such as an ``Enum`` - that has no ``__lt__()`` method; normally Python 3 raises a ``TypeError`` - in this case. The flush process sorts persistent objects by primary key - in Python so the values must be sortable. - - - .. change:: - :tags: orm, bug - :tickets: 3604 - - Removed the collection converter used by the :class:`.MappedCollection` - class. This converter was used only to assert that the incoming dictionary - keys matched that of their corresponding objects, and only during a bulk set - operation. The converter can interfere with a custom validator or - :meth:`.AttributeEvents.bulk_replace` listener that wants to convert - incoming values further. The ``TypeError`` which would be raised by this - converter when an incoming key didn't match the value is removed; incoming - values during a bulk assignment will be keyed to their value-generated key, - and not the key that's explicitly present in the dictionary. - - Overall, @converter is superseded by the - :meth:`.AttributeEvents.bulk_replace` event handler added as part of - :ticket:`3896`. - - .. change:: - :tags: feature, sql - :tickets: 3989 - - Added new naming convention tokens ``column_0N_name``, ``column_0_N_name``, - etc., which will render the names / keys / labels for all columns referenced - by a particular constraint in a sequence. In order to accommodate for the - length of such a naming convention, the SQL compiler's auto-truncation - feature now applies itself to constraint names as well, which creates a - shortened, deterministically generated name for the constraint that will - apply to a target backend without going over the character limit of that - backend. - - The change also repairs two other issues. One is that the ``column_0_key`` - token wasn't available even though this token was documented, the other was - that the ``referred_column_0_name`` token would inadvertently render the - ``.key`` and not the ``.name`` of the column if these two values were - different. - - .. seealso:: - - :ref:`change_3989` - - - .. change:: - :tags: feature, ext - :tickets: 4196 - - Added support for bulk :meth:`_query.Query.update` and :meth:`_query.Query.delete` - to the :class:`.ShardedQuery` class within the horizontal sharding - extension. This also adds an additional expansion hook to the - bulk update/delete methods :meth:`_query.Query._execute_crud`. - - .. seealso:: - - :ref:`change_4196` - - .. change:: - :tags: feature, sql - :tickets: 4271 - - Added new logic to the "expanding IN" bound parameter feature whereby if - the given list is empty, a special "empty set" expression that is specific - to different backends is generated, thus allowing IN expressions to be - fully dynamic including empty IN expressions. - - .. seealso:: - - :ref:`change_4271` - - - - .. change:: - :tags: feature, mysql - - The "pre-ping" feature of the connection pool now uses - the ``ping()`` method of the DBAPI connection in the case of - mysqlclient, PyMySQL and mysql-connector-python. Pull request - courtesy Maxim Bublis. - - .. seealso:: - - :ref:`change_mysql_ping` - - .. change:: - :tags: feature, orm - :tickets: 4340 - - The "selectin" loader strategy now omits the JOIN in the case of a simple - one-to-many load, where it instead relies loads only from the related - table, relying upon the foreign key columns of the related table in order - to match up to primary keys in the parent table. This optimization can be - disabled by setting the :paramref:`_orm.relationship.omit_join` flag to False. - Many thanks to Jayson Reis for the efforts on this. - - .. seealso:: - - :ref:`change_4340` - - .. change:: - :tags: bug, orm - :tickets: 4353 - - Added new behavior to the lazy load that takes place when the "old" value of - a many-to-one is retrieved, such that exceptions which would be raised due - to either ``lazy="raise"`` or a detached session error are skipped. - - .. seealso:: - - :ref:`change_4353` - - .. change:: - :tags: feature, sql - - The Python builtin ``dir()`` is now supported for a SQLAlchemy "properties" - object, such as that of a Core columns collection (e.g. ``.c``), - ``mapper.attrs``, etc. Allows iPython autocompletion to work as well. - Pull request courtesy Uwe Korn. - - .. change:: - :tags: feature, orm - :tickets: 4257 - - Added ``.info`` dictionary to the :class:`.InstanceState` class, the object - that comes from calling :func:`_sa.inspect` on a mapped object. - - .. seealso:: - - :ref:`change_4257` - - .. change:: - :tags: feature, sql - :tickets: 3831 - - Added new feature :meth:`.FunctionElement.as_comparison` which allows a SQL - function to act as a binary comparison operation that can work within the - ORM. - - .. seealso:: - - :ref:`change_3831` - - .. change:: - :tags: bug, orm - :tickets: 4354 - - A long-standing oversight in the ORM, the ``__delete__`` method for a many- - to-one relationship was non-functional, e.g. for an operation such as ``del - a.b``. This is now implemented and is equivalent to setting the attribute - to ``None``. - - .. seealso:: - - :ref:`change_4354` diff --git a/doc/build/changelog/changelog_14.rst b/doc/build/changelog/changelog_14.rst index 1e80c684c56..6d78899fb6a 100644 --- a/doc/build/changelog/changelog_14.rst +++ b/doc/build/changelog/changelog_14.rst @@ -7,11 +7,6 @@ This document details individual issue-level changes made throughout :ref:`migration_14_toplevel`. -.. changelog_imports:: - - .. include:: changelog_13.rst - :start-line: 5 - .. changelog:: :version: 1.4.48 @@ -995,7 +990,7 @@ This document details individual issue-level changes made throughout .. seealso:: - :ref:`azure_synapse_ignore_no_transaction_on_rollback` + ref_azure_synapse_ignore_no_transaction_on_rollback .. change:: :tags: bug, mypy @@ -1832,7 +1827,7 @@ This document details individual issue-level changes made throughout .. seealso:: - :ref:`postgresql_constraint_options` + ref_postgresql_constraint_options .. change:: :tags: bug, orm, regression @@ -1982,7 +1977,7 @@ This document details individual issue-level changes made throughout .. seealso:: - :ref:`asyncio_events_run_async` + ref_asyncio_events_run_async .. change:: @@ -3229,7 +3224,7 @@ This document details individual issue-level changes made throughout .. seealso:: - :ref:`asyncmy` + ref_asyncmy .. change:: :tags: bug, asyncio @@ -4014,7 +4009,7 @@ This document details individual issue-level changes made throughout .. seealso:: - :ref:`asyncio_scoped_session` + ref_asyncio_scoped_session .. change:: :tags: usecase, mysql @@ -5293,7 +5288,7 @@ This document details individual issue-level changes made throughout .. seealso:: - :ref:`mypy_declarative_mixins` + ref_mypy_declarative_mixins .. change:: @@ -5582,7 +5577,7 @@ This document details individual issue-level changes made throughout .. seealso:: - :ref:`aiosqlite` + ref_aiosqlite .. change:: :tags: bug, regression, orm, declarative @@ -5632,7 +5627,7 @@ This document details individual issue-level changes made throughout .. seealso:: - :ref:`pysqlcipher` + ref_pysqlcipher .. change:: @@ -5746,7 +5741,7 @@ This document details individual issue-level changes made throughout .. seealso:: - :ref:`orm_declarative_dataclasses_mixin` + ref_orm_declarative_dataclasses_mixin .. change:: :tags: bug, sql, regression @@ -5845,7 +5840,7 @@ This document details individual issue-level changes made throughout .. seealso:: - :ref:`mssql_pyodbc_setinputsizes` + ref_mssql_pyodbc_setinputsizes .. change:: :tags: bug, orm, regression @@ -5947,7 +5942,7 @@ This document details individual issue-level changes made throughout .. seealso:: - :ref:`mypy_toplevel` + ref_mypy_toplevel .. change:: :tags: bug, sql @@ -6402,7 +6397,7 @@ This document details individual issue-level changes made throughout .. seealso:: - :ref:`orm_declarative_dataclasses_declarative_table` + ref_orm_declarative_dataclasses_declarative_table .. change:: :tags: bug, sql @@ -6510,7 +6505,7 @@ This document details individual issue-level changes made throughout .. seealso:: - :ref:`asyncpg_prepared_statement_cache` + ref_asyncpg_prepared_statement_cache .. change:: :tags: feature, mysql @@ -6521,7 +6516,7 @@ This document details individual issue-level changes made throughout .. seealso:: - :ref:`aiomysql` + ref_aiomysql .. change:: :tags: bug, reflection @@ -6605,7 +6600,7 @@ This document details individual issue-level changes made throughout .. seealso:: - :ref:`sqlite_on_conflict_insert` + ref_sqlite_on_conflict_insert .. change:: :tags: bug, asyncio @@ -6701,7 +6696,7 @@ This document details individual issue-level changes made throughout :ref:`mapper_automated_reflection_schemes` - in the ORM mapping documentation - :ref:`automap_intercepting_columns` - in the :ref:`automap_toplevel` documentation + ref_automap_intercepting_columns - in the ref_automap_toplevel documentation @@ -6792,7 +6787,7 @@ This document details individual issue-level changes made throughout .. seealso:: - :ref:`metadata_reflection_dbagnostic_types` - example usage + ref_metadata_reflection_dbagnostic_types - example usage .. change:: :tags: bug, sql @@ -7523,13 +7518,13 @@ This document details individual issue-level changes made throughout :tickets: 3414 SQLAlchemy now includes support for Python asyncio within both Core and - ORM, using the included :ref:`asyncio extension `. The + ORM, using the included ref_asyncio_toplevel. The extension makes use of the `greenlet `_ library in order to adapt SQLAlchemy's sync-oriented internals such that an asyncio interface that ultimately interacts with an asyncio database adapter is now feasible. The single driver supported at the moment is the - :ref:`dialect-postgresql-asyncpg` driver for PostgreSQL. + ref_dialect-postgresql-asyncpg driver for PostgreSQL. .. seealso:: @@ -7663,7 +7658,7 @@ This document details individual issue-level changes made throughout Microsoft SQL Server. This removes the deprecated feature of using :class:`.Sequence` objects to manipulate IDENTITY characteristics which should now be performed using ``mssql_identity_start`` and - ``mssql_identity_increment`` as documented at :ref:`mssql_identity`. The + ``mssql_identity_increment`` as documented at ref_mssql_identity. The change includes a new parameter :paramref:`.Sequence.data_type` to accommodate SQL Server's choice of datatype, which for that backend includes INTEGER, BIGINT, and DECIMAL(n, 0). The default starting value @@ -7814,7 +7809,7 @@ This document details individual issue-level changes made throughout .. seealso:: - :ref:`oracle_max_identifier_lengths` - in the Oracle dialect documentation + ref_oracle_max_identifier_lengths - in the Oracle dialect documentation .. change:: @@ -8284,7 +8279,7 @@ This document details individual issue-level changes made throughout .. seealso:: - :ref:`postgresql_readonly_deferrable` + ref_postgresql_readonly_deferrable .. change:: :tags: mysql, feature @@ -8549,7 +8544,7 @@ This document details individual issue-level changes made throughout Remove deprecated method ``Session.prune`` and parameter ``Session.weak_identity_map``. See the recipe at - :ref:`session_referencing_behavior` for an event-based approach to + ref_session_referencing_behavior for an event-based approach to maintaining strong identity references. This change also removes the class ``StrongInstanceDict``. diff --git a/doc/build/changelog/changelog_20.rst b/doc/build/changelog/changelog_20.rst index f6ec81c15c2..f254d45420a 100644 --- a/doc/build/changelog/changelog_20.rst +++ b/doc/build/changelog/changelog_20.rst @@ -109,7 +109,7 @@ .. seealso:: - :ref:`asyncpg_prepared_statement_name` + ref_asyncpg_prepared_statement_name .. change:: :tags: typing, bug @@ -347,7 +347,7 @@ :ref:`error_dcmx` - background on rationale - :ref:`orm_declarative_dc_mixins` + ref_orm_declarative_dc_mixins .. change:: :tags: bug, postgresql @@ -625,7 +625,7 @@ SQLAlchemy now computes rowcount for a RETURNING statement in this specific case by counting the rows returned, rather than relying upon ``cursor.rowcount``. In particular, the ORM versioned rows use case - (documented at :ref:`mapper_version_counter`) should now be fully + (documented at ref_mapper_version_counter) should now be fully supported with the SQL Server pyodbc dialect. @@ -829,7 +829,7 @@ :tags: usecase, typing :tickets: 9321 - Improved the typing support for the :ref:`hybrids_toplevel` + Improved the typing support for the ref_hybrids_toplevel extension, updated all documentation to use ORM Annotated Declarative mappings, and added a new modifier called :attr:`.hybrid_property.inplace`. This modifier provides a way to alter the state of a :class:`.hybrid_property` @@ -845,7 +845,7 @@ .. seealso:: - :ref:`hybrid_pep484_naming` + ref_hybrid_pep484_naming .. change:: :tags: bug, orm @@ -885,7 +885,7 @@ :tickets: 9295 Adjusted the behavior of the ``thick_mode`` parameter for the - :ref:`oracledb` dialect to correctly accept ``False`` as a value. + ref_oracledb dialect to correctly accept ``False`` as a value. Previously, only ``None`` would indicate that thick mode should be disabled. @@ -936,7 +936,7 @@ .. seealso:: - :ref:`dataclasses_pydantic` + ref_dataclasses_pydantic .. changelog:: @@ -1144,7 +1144,7 @@ :class:`_orm.Mapper` object is created within the class creation process, there was no documented means of running code at this point. The change is to immediately benefit custom mapping schemes such as that - of the :ref:`examples_versioned_history` example, which generate additional + of the ref_examples_versioned_history example, which generate additional mappers and tables in response to the creation of mapped classes. @@ -1174,7 +1174,7 @@ :tags: bug, examples :tickets: 9220 - Reworked the :ref:`examples_versioned_history` to work with + Reworked the ref_examples_versioned_history to work with version 2.0, while at the same time improving the overall working of this example to use newer APIs, including a newly added hook :meth:`_orm.MapperEvents.after_mapper_constructed`. @@ -1406,7 +1406,7 @@ .. seealso:: - :ref:`automap_by_module` - illustrates use of both techniques at once. + ref_automap_by_module - illustrates use of both techniques at once. .. change:: :tags: orm, bug @@ -1520,7 +1520,7 @@ .. seealso:: - :ref:`mssql_comment_support` + ref_mssql_comment_support .. changelog:: @@ -1937,7 +1937,7 @@ .. seealso:: - :ref:`orm_declarative_native_dataclasses_non_mapped_fields` + ref_orm_declarative_native_dataclasses_non_mapped_fields .. change:: @@ -2172,7 +2172,7 @@ :tags: bug, orm :tickets: 8880 - Fixed bug in :ref:`orm_declarative_native_dataclasses` feature where using + Fixed bug in ref_orm_declarative_native_dataclasses feature where using plain dataclass fields with the ``__allow_unmapped__`` directive in a mapping would not create a dataclass with the correct class-level state for those fields, copying the raw ``Field`` object to the class inappropriately @@ -2186,7 +2186,7 @@ Added support for the :func:`.association_proxy` extension function to take part within Python ``dataclasses`` configuration, when using the native dataclasses feature described at - :ref:`orm_declarative_native_dataclasses`. Included are attribute-level + ref_orm_declarative_native_dataclasses. Included are attribute-level arguments including :paramref:`.association_proxy.init` and :paramref:`.association_proxy.default_factory`. @@ -2237,7 +2237,7 @@ attribute constructs including :func:`_orm.mapped_column`, :func:`_orm.relationship` etc. to provide for the Python dataclasses ``compare`` parameter on ``field()``, when using the - :ref:`orm_declarative_native_dataclasses` feature. Pull request courtesy + ref_orm_declarative_native_dataclasses feature. Pull request courtesy Simon Schiele. .. change:: @@ -3005,7 +3005,7 @@ .. seealso:: - :ref:`sqlite_include_internal` + ref_sqlite_include_internal .. change:: :tags: feature, postgresql @@ -3039,7 +3039,7 @@ :ref:`ticket_8054` - :ref:`oracledb` + ref_oracledb .. change:: :tags: bug, engine @@ -3197,7 +3197,7 @@ :ref:`ticket_6842` - :ref:`postgresql_psycopg` + ref_postgresql_psycopg @@ -3231,7 +3231,7 @@ Additionally, classes mapped by :class:`_orm.composite` now support ordering comparison operations, e.g. ``<``, ``>=``, etc. - See the new documentation at :ref:`mapper_composite` for examples. + See the new documentation at ref_mapper_composite for examples. .. change:: :tags: engine, bug @@ -4005,7 +4005,7 @@ ``'value'``. For normal bound value handling, the :class:`_types.Unicode` datatype also may have implications for passing values to the DBAPI, again in the case of SQL Server, the pyodbc driver supports the use of - :ref:`setinputsizes mode ` which will handle + ref_mssql_pyodbc_setinputsizes which will handle :class:`_types.String` versus :class:`_types.Unicode` differently. diff --git a/doc/build/changelog/index.rst b/doc/build/changelog/index.rst index d6a0d26f65f..136f012b7a4 100644 --- a/doc/build/changelog/index.rst +++ b/doc/build/changelog/index.rst @@ -28,19 +28,6 @@ Change logs changelog_20 changelog_14 - changelog_13 - changelog_12 - changelog_11 - changelog_10 - changelog_09 - changelog_08 - changelog_07 - changelog_06 - changelog_05 - changelog_04 - changelog_03 - changelog_02 - changelog_01 Older Migration Guides @@ -51,12 +38,3 @@ Older Migration Guides migration_14 migration_13 - migration_12 - migration_11 - migration_10 - migration_09 - migration_08 - migration_07 - migration_06 - migration_05 - migration_04 diff --git a/doc/build/changelog/migration_04.rst b/doc/build/changelog/migration_04.rst deleted file mode 100644 index 2618c77e3a3..00000000000 --- a/doc/build/changelog/migration_04.rst +++ /dev/null @@ -1,921 +0,0 @@ -============================= -What's new in SQLAlchemy 0.4? -============================= - -.. admonition:: About this Document - - This document describes changes between SQLAlchemy version 0.3, - last released October 14, 2007, and SQLAlchemy version 0.4, - last released October 12, 2008. - - Document date: March 21, 2008 - -First Things First -================== - -If you're using any ORM features, make sure you import from -``sqlalchemy.orm``: - -:: - - from sqlalchemy import * - from sqlalchemy.orm import * - -Secondly, anywhere you used to say ``engine=``, -``connectable=``, ``bind_to=``, ``something.engine``, -``metadata.connect()``, use ``bind``: - -:: - - myengine = create_engine("sqlite://") - - meta = MetaData(myengine) - - meta2 = MetaData() - meta2.bind = myengine - - session = create_session(bind=myengine) - - statement = select([table], bind=myengine) - -Got those ? Good! You're now (95%) 0.4 compatible. If -you're using 0.3.10, you can make these changes immediately; -they'll work there too. - -Module Imports -============== - -In 0.3, "``from sqlalchemy import *``" would import all of -sqlalchemy's sub-modules into your namespace. Version 0.4 no -longer imports sub-modules into the namespace. This may mean -you need to add extra imports into your code. - -In 0.3, this code worked: - -:: - - from sqlalchemy import * - - - class UTCDateTime(types.TypeDecorator): - pass - -In 0.4, one must do: - -:: - - from sqlalchemy import * - from sqlalchemy import types - - - class UTCDateTime(types.TypeDecorator): - pass - -Object Relational Mapping -========================= - -Querying --------- - -New Query API -^^^^^^^^^^^^^ - -Query is standardized on the generative interface (old -interface is still there, just deprecated). While most of -the generative interface is available in 0.3, the 0.4 Query -has the inner guts to match the generative outside, and has -a lot more tricks. All result narrowing is via ``filter()`` -and ``filter_by()``, limiting/offset is either through array -slices or ``limit()``/``offset()``, joining is via -``join()`` and ``outerjoin()`` (or more manually, through -``select_from()`` as well as manually-formed criteria). - -To avoid deprecation warnings, you must make some changes to -your 03 code - -User.query.get_by( \**kwargs ) - -:: - - User.query.filter_by(**kwargs).first() - -User.query.select_by( \**kwargs ) - -:: - - User.query.filter_by(**kwargs).all() - -User.query.select() - -:: - - User.query.filter(xxx).all() - -New Property-Based Expression Constructs -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -By far the most palpable difference within the ORM is that -you can now construct your query criterion using class-based -attributes directly. The ".c." prefix is no longer needed -when working with mapped classes: - -:: - - session.query(User).filter(and_(User.name == "fred", User.id > 17)) - -While simple column-based comparisons are no big deal, the -class attributes have some new "higher level" constructs -available, including what was previously only available in -``filter_by()``: - -:: - - # comparison of scalar relations to an instance - filter(Address.user == user) - - # return all users who contain a particular address - filter(User.addresses.contains(address)) - - # return all users who *dont* contain the address - filter(~User.address.contains(address)) - - # return all users who contain a particular address with - # the email_address like '%foo%' - filter(User.addresses.any(Address.email_address.like("%foo%"))) - - # same, email address equals 'foo@bar.com'. can fall back to keyword - # args for simple comparisons - filter(User.addresses.any(email_address="foo@bar.com")) - - # return all Addresses whose user attribute has the username 'ed' - filter(Address.user.has(name="ed")) - - # return all Addresses whose user attribute has the username 'ed' - # and an id > 5 (mixing clauses with kwargs) - filter(Address.user.has(User.id > 5, name="ed")) - -The ``Column`` collection remains available on mapped -classes in the ``.c`` attribute. Note that property-based -expressions are only available with mapped properties of -mapped classes. ``.c`` is still used to access columns in -regular tables and selectable objects produced from SQL -Expressions. - -Automatic Join Aliasing -^^^^^^^^^^^^^^^^^^^^^^^ - -We've had join() and outerjoin() for a while now: - -:: - - session.query(Order).join("items") - -Now you can alias them: - -:: - - session.query(Order).join("items", aliased=True).filter(Item.name="item 1").join( - "items", aliased=True - ).filter(Item.name == "item 3") - -The above will create two joins from orders->items using -aliases. the ``filter()`` call subsequent to each will -adjust its table criterion to that of the alias. To get at -the ``Item`` objects, use ``add_entity()`` and target each -join with an ``id``: - -:: - - session.query(Order).join("items", id="j1", aliased=True).filter( - Item.name == "item 1" - ).join("items", aliased=True, id="j2").filter(Item.name == "item 3").add_entity( - Item, id="j1" - ).add_entity( - Item, id="j2" - ) - -Returns tuples in the form: ``(Order, Item, Item)``. - -Self-referential Queries -^^^^^^^^^^^^^^^^^^^^^^^^ - -So query.join() can make aliases now. What does that give -us ? Self-referential queries ! Joins can be done without -any ``Alias`` objects: - -:: - - # standard self-referential TreeNode mapper with backref - mapper( - TreeNode, - tree_nodes, - properties={ - "children": relation( - TreeNode, backref=backref("parent", remote_side=tree_nodes.id) - ) - }, - ) - - # query for node with child containing "bar" two levels deep - session.query(TreeNode).join(["children", "children"], aliased=True).filter_by( - name="bar" - ) - -To add criterion for each table along the way in an aliased -join, you can use ``from_joinpoint`` to keep joining against -the same line of aliases: - -:: - - # search for the treenode along the path "n1/n12/n122" - - # first find a Node with name="n122" - q = sess.query(Node).filter_by(name="n122") - - # then join to parent with "n12" - q = q.join("parent", aliased=True).filter_by(name="n12") - - # join again to the next parent with 'n1'. use 'from_joinpoint' - # so we join from the previous point, instead of joining off the - # root table - q = q.join("parent", aliased=True, from_joinpoint=True).filter_by(name="n1") - - node = q.first() - -``query.populate_existing()`` -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The eager version of ``query.load()`` (or -``session.refresh()``). Every instance loaded from the -query, including all eagerly loaded items, get refreshed -immediately if already present in the session: - -:: - - session.query(Blah).populate_existing().all() - -Relations ---------- - -SQL Clauses Embedded in Updates/Inserts -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -For inline execution of SQL clauses, embedded right in the -UPDATE or INSERT, during a ``flush()``: - -:: - - - myobject.foo = mytable.c.value + 1 - - user.pwhash = func.md5(password) - - order.hash = text("select hash from hashing_table") - -The column-attribute is set up with a deferred loader after -the operation, so that it issues the SQL to load the new -value when you next access. - -Self-referential and Cyclical Eager Loading -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Since our alias-fu has improved, ``relation()`` can join -along the same table \*any number of times*; you tell it how -deep you want to go. Lets show the self-referential -``TreeNode`` more clearly: - -:: - - nodes = Table( - "nodes", - metadata, - Column("id", Integer, primary_key=True), - Column("parent_id", Integer, ForeignKey("nodes.id")), - Column("name", String(30)), - ) - - - class TreeNode(object): - pass - - - mapper( - TreeNode, - nodes, - properties={"children": relation(TreeNode, lazy=False, join_depth=3)}, - ) - -So what happens when we say: - -:: - - create_session().query(TreeNode).all() - -? A join along aliases, three levels deep off the parent: - -.. sourcecode:: sql - - SELECT - nodes_3.id AS nodes_3_id, nodes_3.parent_id AS nodes_3_parent_id, nodes_3.name AS nodes_3_name, - nodes_2.id AS nodes_2_id, nodes_2.parent_id AS nodes_2_parent_id, nodes_2.name AS nodes_2_name, - nodes_1.id AS nodes_1_id, nodes_1.parent_id AS nodes_1_parent_id, nodes_1.name AS nodes_1_name, - nodes.id AS nodes_id, nodes.parent_id AS nodes_parent_id, nodes.name AS nodes_name - FROM nodes LEFT OUTER JOIN nodes AS nodes_1 ON nodes.id = nodes_1.parent_id - LEFT OUTER JOIN nodes AS nodes_2 ON nodes_1.id = nodes_2.parent_id - LEFT OUTER JOIN nodes AS nodes_3 ON nodes_2.id = nodes_3.parent_id - ORDER BY nodes.oid, nodes_1.oid, nodes_2.oid, nodes_3.oid - -Notice the nice clean alias names too. The joining doesn't -care if it's against the same immediate table or some other -object which then cycles back to the beginning. Any kind -of chain of eager loads can cycle back onto itself when -``join_depth`` is specified. When not present, eager -loading automatically stops when it hits a cycle. - -Composite Types -^^^^^^^^^^^^^^^ - -This is one from the Hibernate camp. Composite Types let -you define a custom datatype that is composed of more than -one column (or one column, if you wanted). Lets define a -new type, ``Point``. Stores an x/y coordinate: - -:: - - class Point(object): - def __init__(self, x, y): - self.x = x - self.y = y - - def __composite_values__(self): - return self.x, self.y - - def __eq__(self, other): - return other.x == self.x and other.y == self.y - - def __ne__(self, other): - return not self.__eq__(other) - -The way the ``Point`` object is defined is specific to a -custom type; constructor takes a list of arguments, and the -``__composite_values__()`` method produces a sequence of -those arguments. The order will match up to our mapper, as -we'll see in a moment. - -Let's create a table of vertices storing two points per row: - -:: - - vertices = Table( - "vertices", - metadata, - Column("id", Integer, primary_key=True), - Column("x1", Integer), - Column("y1", Integer), - Column("x2", Integer), - Column("y2", Integer), - ) - -Then, map it ! We'll create a ``Vertex`` object which -stores two ``Point`` objects: - -:: - - class Vertex(object): - def __init__(self, start, end): - self.start = start - self.end = end - - - mapper( - Vertex, - vertices, - properties={ - "start": composite(Point, vertices.c.x1, vertices.c.y1), - "end": composite(Point, vertices.c.x2, vertices.c.y2), - }, - ) - -Once you've set up your composite type, it's usable just -like any other type: - -:: - - - v = Vertex(Point(3, 4), Point(26, 15)) - session.save(v) - session.flush() - - # works in queries too - q = session.query(Vertex).filter(Vertex.start == Point(3, 4)) - -If you'd like to define the way the mapped attributes -generate SQL clauses when used in expressions, create your -own ``sqlalchemy.orm.PropComparator`` subclass, defining any -of the common operators (like ``__eq__()``, ``__le__()``, -etc.), and send it in to ``composite()``. Composite types -work as primary keys too, and are usable in ``query.get()``: - -:: - - # a Document class which uses a composite Version - # object as primary key - document = query.get(Version(1, "a")) - -``dynamic_loader()`` relations -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -A ``relation()`` that returns a live ``Query`` object for -all read operations. Write operations are limited to just -``append()`` and ``remove()``, changes to the collection are -not visible until the session is flushed. This feature is -particularly handy with an "autoflushing" session which will -flush before each query. - -:: - - mapper( - Foo, - foo_table, - properties={ - "bars": dynamic_loader( - Bar, - backref="foo", - # - ) - }, - ) - - session = create_session(autoflush=True) - foo = session.query(Foo).first() - - foo.bars.append(Bar(name="lala")) - - for bar in foo.bars.filter(Bar.name == "lala"): - print(bar) - - session.commit() - -New Options: ``undefer_group()``, ``eagerload_all()`` -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -A couple of query options which are handy. -``undefer_group()`` marks a whole group of "deferred" -columns as undeferred: - -:: - - mapper( - Class, - table, - properties={ - "foo": deferred(table.c.foo, group="group1"), - "bar": deferred(table.c.bar, group="group1"), - "bat": deferred(table.c.bat, group="group1"), - }, - ) - - session.query(Class).options(undefer_group("group1")).filter(...).all() - -and ``eagerload_all()`` sets a chain of attributes to be -eager in one pass: - -:: - - mapper(Foo, foo_table, properties={"bar": relation(Bar)}) - mapper(Bar, bar_table, properties={"bat": relation(Bat)}) - mapper(Bat, bat_table) - - # eager load bar and bat - session.query(Foo).options(eagerload_all("bar.bat")).filter(...).all() - -New Collection API -^^^^^^^^^^^^^^^^^^ - -Collections are no longer proxied by an -{{{InstrumentedList}}} proxy, and access to members, methods -and attributes is direct. Decorators now intercept objects -entering and leaving the collection, and it is now possible -to easily write a custom collection class that manages its -own membership. Flexible decorators also replace the named -method interface of custom collections in 0.3, allowing any -class to be easily adapted to use as a collection container. - -Dictionary-based collections are now much easier to use and -fully ``dict``-like. Changing ``__iter__`` is no longer -needed for ``dict``s, and new built-in ``dict`` types cover -many needs: - -:: - - # use a dictionary relation keyed by a column - relation(Item, collection_class=column_mapped_collection(items.c.keyword)) - # or named attribute - relation(Item, collection_class=attribute_mapped_collection("keyword")) - # or any function you like - relation(Item, collection_class=mapped_collection(lambda entity: entity.a + entity.b)) - -Existing 0.3 ``dict``-like and freeform object derived -collection classes will need to be updated for the new API. -In most cases this is simply a matter of adding a couple -decorators to the class definition. - -Mapped Relations from External Tables/Subqueries -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -This feature quietly appeared in 0.3 but has been improved -in 0.4 thanks to better ability to convert subqueries -against a table into subqueries against an alias of that -table; this is key for eager loading, aliased joins in -queries, etc. It reduces the need to create mappers against -select statements when you just need to add some extra -columns or subqueries: - -:: - - mapper( - User, - users, - properties={ - "fullname": column_property( - (users.c.firstname + users.c.lastname).label("fullname") - ), - "numposts": column_property( - select([func.count(1)], users.c.id == posts.c.user_id) - .correlate(users) - .label("posts") - ), - }, - ) - -a typical query looks like: - -.. sourcecode:: sql - - SELECT (SELECT count(1) FROM posts WHERE users.id = posts.user_id) AS count, - users.firstname || users.lastname AS fullname, - users.id AS users_id, users.firstname AS users_firstname, users.lastname AS users_lastname - FROM users ORDER BY users.oid - -Horizontal Scaling (Sharding) API ---------------------------------- - -[browser:/sqlalchemy/trunk/examples/sharding/attribute_shard -.py] - -Sessions --------- - -New Session Create Paradigm; SessionContext, assignmapper Deprecated -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -That's right, the whole shebang is being replaced with two -configurational functions. Using both will produce the most -0.1-ish feel we've had since 0.1 (i.e., the least amount of -typing). - -Configure your own ``Session`` class right where you define -your ``engine`` (or anywhere): - -:: - - from sqlalchemy import create_engine - from sqlalchemy.orm import sessionmaker - - engine = create_engine("myengine://") - Session = sessionmaker(bind=engine, autoflush=True, transactional=True) - - # use the new Session() freely - sess = Session() - sess.save(someobject) - sess.flush() - -If you need to post-configure your Session, say with an -engine, add it later with ``configure()``: - -:: - - Session.configure(bind=create_engine(...)) - -All the behaviors of ``SessionContext`` and the ``query`` -and ``__init__`` methods of ``assignmapper`` are moved into -the new ``scoped_session()`` function, which is compatible -with both ``sessionmaker`` as well as ``create_session()``: - -:: - - from sqlalchemy.orm import scoped_session, sessionmaker - - Session = scoped_session(sessionmaker(autoflush=True, transactional=True)) - Session.configure(bind=engine) - - u = User(name="wendy") - - sess = Session() - sess.save(u) - sess.commit() - - # Session constructor is thread-locally scoped. Everyone gets the same - # Session in the thread when scope="thread". - sess2 = Session() - assert sess is sess2 - -When using a thread-local ``Session``, the returned class -has all of ``Session's`` interface implemented as -classmethods, and "assignmapper"'s functionality is -available using the ``mapper`` classmethod. Just like the -old ``objectstore`` days.... - -:: - - - # "assignmapper"-like functionality available via ScopedSession.mapper - Session.mapper(User, users_table) - - u = User(name="wendy") - - Session.commit() - -Sessions are again Weak Referencing By Default -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The weak_identity_map flag is now set to ``True`` by default -on Session. Instances which are externally deferenced and -fall out of scope are removed from the session -automatically. However, items which have "dirty" changes -present will remain strongly referenced until those changes -are flushed at which case the object reverts to being weakly -referenced (this works for 'mutable' types, like picklable -attributes, as well). Setting weak_identity_map to -``False`` restores the old strong-referencing behavior for -those of you using the session like a cache. - -Auto-Transactional Sessions -^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -As you might have noticed above, we are calling ``commit()`` -on ``Session``. The flag ``transactional=True`` means the -``Session`` is always in a transaction, ``commit()`` -persists permanently. - -Auto-Flushing Sessions -^^^^^^^^^^^^^^^^^^^^^^ - -Also, ``autoflush=True`` means the ``Session`` will -``flush()`` before each ``query`` as well as when you call -``flush()`` or ``commit()``. So now this will work: - -:: - - Session = sessionmaker(bind=engine, autoflush=True, transactional=True) - - u = User(name="wendy") - - sess = Session() - sess.save(u) - - # wendy is flushed, comes right back from a query - wendy = sess.query(User).filter_by(name="wendy").one() - -Transactional methods moved onto sessions -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -``commit()`` and ``rollback()``, as well as ``begin()`` are -now directly on ``Session``. No more need to use -``SessionTransaction`` for anything (it remains in the -background). - -:: - - Session = sessionmaker(autoflush=True, transactional=False) - - sess = Session() - sess.begin() - - # use the session - - sess.commit() # commit transaction - -Sharing a ``Session`` with an enclosing engine-level (i.e. -non-ORM) transaction is easy: - -:: - - Session = sessionmaker(autoflush=True, transactional=False) - - conn = engine.connect() - trans = conn.begin() - sess = Session(bind=conn) - - # ... session is transactional - - # commit the outermost transaction - trans.commit() - -Nested Session Transactions with SAVEPOINT -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Available at the Engine and ORM level. ORM docs so far: - -https://www.sqlalchemy.org/docs/04/session.html#unitofwork_managing - -Two-Phase Commit Sessions -^^^^^^^^^^^^^^^^^^^^^^^^^ - -Available at the Engine and ORM level. ORM docs so far: - -https://www.sqlalchemy.org/docs/04/session.html#unitofwork_managing - -Inheritance ------------ - -Polymorphic Inheritance with No Joins or Unions -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -New docs for inheritance: https://www.sqlalchemy.org/docs/04 -/mappers.html#advdatamapping_mapper_inheritance_joined - -Better Polymorphic Behavior with ``get()`` -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -All classes within a joined-table inheritance hierarchy get -an ``_instance_key`` using the base class, i.e. -``(BaseClass, (1, ), None)``. That way when you call -``get()`` a ``Query`` against the base class, it can locate -subclass instances in the current identity map without -querying the database. - -Types ------ - -Custom Subclasses of ``sqlalchemy.types.TypeDecorator`` -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -There is a `New API `_ for subclassing a TypeDecorator. -Using the 0.3 API causes compilation errors in some cases. - -SQL Expressions -=============== - -All New, Deterministic Label/Alias Generation ---------------------------------------------- - -All the "anonymous" labels and aliases use a simple -_ format now. SQL is much easier to read and -is compatible with plan optimizer caches. Just check out -some of the examples in the tutorials: -https://www.sqlalchemy.org/docs/04/ormtutorial.html -https://www.sqlalchemy.org/docs/04/sqlexpression.html - -Generative select() Constructs ------------------------------- - -This is definitely the way to go with ``select()``. See htt -p://www.sqlalchemy.org/docs/04/sqlexpression.html#sql_transf -orm . - -New Operator System -------------------- - -SQL operators and more or less every SQL keyword there is -are now abstracted into the compiler layer. They now act -intelligently and are type/backend aware, see: -https://www.sqlalchemy.org/docs/04/sqlexpression.html#sql_operators - -All ``type`` Keyword Arguments Renamed to ``type_`` ---------------------------------------------------- - -Just like it says: - -:: - - b = bindparam("foo", type_=String) - -in\_ Function Changed to Accept Sequence or Selectable ------------------------------------------------------- - -The in\_ function now takes a sequence of values or a -selectable as its sole argument. The previous API of passing -in values as positional arguments still works, but is now -deprecated. This means that - -:: - - my_table.select(my_table.c.id.in_(1, 2, 3)) - my_table.select(my_table.c.id.in_(*listOfIds)) - -should be changed to - -:: - - my_table.select(my_table.c.id.in_([1, 2, 3])) - my_table.select(my_table.c.id.in_(listOfIds)) - -Schema and Reflection -===================== - -``MetaData``, ``BoundMetaData``, ``DynamicMetaData``... -------------------------------------------------------- - -In the 0.3.x series, ``BoundMetaData`` and -``DynamicMetaData`` were deprecated in favor of ``MetaData`` -and ``ThreadLocalMetaData``. The older names have been -removed in 0.4. Updating is simple: - -.. sourcecode:: text - - +-------------------------------------+-------------------------+ - |If You Had | Now Use | - +=====================================+=========================+ - | ``MetaData`` | ``MetaData`` | - +-------------------------------------+-------------------------+ - | ``BoundMetaData`` | ``MetaData`` | - +-------------------------------------+-------------------------+ - | ``DynamicMetaData`` (with one | ``MetaData`` | - | engine or threadlocal=False) | | - +-------------------------------------+-------------------------+ - | ``DynamicMetaData`` | ``ThreadLocalMetaData`` | - | (with different engines per thread) | | - +-------------------------------------+-------------------------+ - -The seldom-used ``name`` parameter to ``MetaData`` types has -been removed. The ``ThreadLocalMetaData`` constructor now -takes no arguments. Both types can now be bound to an -``Engine`` or a single ``Connection``. - -One Step Multi-Table Reflection -------------------------------- - -You can now load table definitions and automatically create -``Table`` objects from an entire database or schema in one -pass: - -:: - - >>> metadata = MetaData(myengine, reflect=True) - >>> metadata.tables.keys() - ['table_a', 'table_b', 'table_c', '...'] - -``MetaData`` also gains a ``.reflect()`` method enabling -finer control over the loading process, including -specification of a subset of available tables to load. - -SQL Execution -============= - -``engine``, ``connectable``, and ``bind_to`` are all now ``bind`` ------------------------------------------------------------------ - -``Transactions``, ``NestedTransactions`` and ``TwoPhaseTransactions`` ---------------------------------------------------------------------- - -Connection Pool Events ----------------------- - -The connection pool now fires events when new DB-API -connections are created, checked out and checked back into -the pool. You can use these to execute session-scoped SQL -setup statements on fresh connections, for example. - -Oracle Engine Fixed -------------------- - -In 0.3.11, there were bugs in the Oracle Engine on how -Primary Keys are handled. These bugs could cause programs -that worked fine with other engines, such as sqlite, to fail -when using the Oracle Engine. In 0.4, the Oracle Engine has -been reworked, fixing these Primary Key problems. - -Out Parameters for Oracle -------------------------- - -:: - - result = engine.execute( - text( - "begin foo(:x, :y, :z); end;", - bindparams=[ - bindparam("x", Numeric), - outparam("y", Numeric), - outparam("z", Numeric), - ], - ), - x=5, - ) - assert result.out_parameters == {"y": 10, "z": 75} - -Connection-bound ``MetaData``, ``Sessions`` -------------------------------------------- - -``MetaData`` and ``Session`` can be explicitly bound to a -connection: - -:: - - conn = engine.connect() - sess = create_session(bind=conn) - -Faster, More Foolproof ``ResultProxy`` Objects ----------------------------------------------- - diff --git a/doc/build/changelog/migration_05.rst b/doc/build/changelog/migration_05.rst deleted file mode 100644 index d26a22c0d00..00000000000 --- a/doc/build/changelog/migration_05.rst +++ /dev/null @@ -1,770 +0,0 @@ -============================= -What's new in SQLAlchemy 0.5? -============================= - -.. admonition:: About this Document - - This document describes changes between SQLAlchemy version 0.4, - last released October 12, 2008, and SQLAlchemy version 0.5, - last released January 16, 2010. - - Document date: August 4, 2009 - - -This guide documents API changes which affect users -migrating their applications from the 0.4 series of -SQLAlchemy to 0.5. It's also recommended for those working -from `Essential SQLAlchemy -`_, which only -covers 0.4 and seems to even have some old 0.3isms in it. -Note that SQLAlchemy 0.5 removes many behaviors which were -deprecated throughout the span of the 0.4 series, and also -deprecates more behaviors specific to 0.4. - -Major Documentation Changes -=========================== - -Some sections of the documentation have been completely -rewritten and can serve as an introduction to new ORM -features. The ``Query`` and ``Session`` objects in -particular have some distinct differences in API and -behavior which fundamentally change many of the basic ways -things are done, particularly with regards to constructing -highly customized ORM queries and dealing with stale session -state, commits and rollbacks. - -* `ORM Tutorial - `_ - -* `Session Documentation - `_ - -Deprecations Source -=================== - -Another source of information is documented within a series -of unit tests illustrating up to date usages of some common -``Query`` patterns; this file can be viewed at -[source:sqlalchemy/trunk/test/orm/test_deprecations.py]. - -Requirements Changes -==================== - -* Python 2.4 or higher is required. The SQLAlchemy 0.4 line - is the last version with Python 2.3 support. - -Object Relational Mapping -========================= - -* **Column level expressions within Query.** - as detailed - in the `tutorial - `_, - ``Query`` has the capability to create specific SELECT - statements, not just those against full rows: - - :: - - session.query(User.name, func.count(Address.id).label("numaddresses")).join( - Address - ).group_by(User.name) - - The tuples returned by any multi-column/entity query are - *named*' tuples: - - :: - - for row in ( - session.query(User.name, func.count(Address.id).label("numaddresses")) - .join(Address) - .group_by(User.name) - ): - print("name", row.name, "number", row.numaddresses) - - ``Query`` has a ``statement`` accessor, as well as a - ``subquery()`` method which allow ``Query`` to be used to - create more complex combinations: - - :: - - subq = ( - session.query(Keyword.id.label("keyword_id")) - .filter(Keyword.name.in_(["beans", "carrots"])) - .subquery() - ) - recipes = session.query(Recipe).filter( - exists() - .where(Recipe.id == recipe_keywords.c.recipe_id) - .where(recipe_keywords.c.keyword_id == subq.c.keyword_id) - ) - -* **Explicit ORM aliases are recommended for aliased joins** - - The ``aliased()`` function produces an "alias" of a - class, which allows fine-grained control of aliases in - conjunction with ORM queries. While a table-level alias - (i.e. ``table.alias()``) is still usable, an ORM level - alias retains the semantics of the ORM mapped object which - is significant for inheritance mappings, options, and - other scenarios. E.g.: - - :: - - Friend = aliased(Person) - session.query(Person, Friend).join((Friend, Person.friends)).all() - -* **query.join() greatly enhanced.** - You can now specify - the target and ON clause for a join in multiple ways. A - target class alone can be provided where SQLA will attempt - to form a join to it via foreign key in the same way as - ``table.join(someothertable)``. A target and an explicit - ON condition can be provided, where the ON condition can - be a ``relation()`` name, an actual class descriptor, or a - SQL expression. Or the old way of just a ``relation()`` - name or class descriptor works too. See the ORM tutorial - which has several examples. - -* **Declarative is recommended for applications which don't - require (and don't prefer) abstraction between tables and - mappers** - The [/docs/05/reference/ext/declarative.html - Declarative] module, which is used to combine the - expression of ``Table``, ``mapper()``, and user defined - class objects together, is highly recommended as it - simplifies application configuration, ensures the "one - mapper per class" pattern, and allows the full range of - configuration available to distinct ``mapper()`` calls. - Separate ``mapper()`` and ``Table`` usage is now referred - to as "classical SQLAlchemy usage" and of course is freely - mixable with declarative. - -* **The .c. attribute has been removed** from classes (i.e. - ``MyClass.c.somecolumn``). As is the case in 0.4, class- - level properties are usable as query elements, i.e. - ``Class.c.propname`` is now superseded by - ``Class.propname``, and the ``c`` attribute continues to - remain on ``Table`` objects where they indicate the - namespace of ``Column`` objects present on the table. - - To get at the Table for a mapped class (if you didn't keep - it around already): - - :: - - table = class_mapper(someclass).mapped_table - - Iterate through columns: - - :: - - for col in table.c: - print(col) - - Work with a specific column: - - :: - - table.c.somecolumn - - The class-bound descriptors support the full set of Column - operators as well as the documented relation-oriented - operators like ``has()``, ``any()``, ``contains()``, etc. - - The reason for the hard removal of ``.c.`` is that in 0.5, - class-bound descriptors carry potentially different - meaning, as well as information regarding class mappings, - versus plain ``Column`` objects - and there are use cases - where you'd specifically want to use one or the other. - Generally, using class-bound descriptors invokes a set of - mapping/polymorphic aware translations, and using table- - bound columns does not. In 0.4, these translations were - applied across the board to all expressions, but 0.5 - differentiates completely between columns and mapped - descriptors, only applying translations to the latter. So - in many cases, particularly when dealing with joined table - inheritance configurations as well as when using - ``query()``, ``Class.propname`` and - ``table.c.colname`` are not interchangeable. - - For example, ``session.query(users.c.id, users.c.name)`` - is different versus ``session.query(User.id, User.name)``; - in the latter case, the ``Query`` is aware of the mapper - in use and further mapper-specific operations like - ``query.join()``, ``query.with_parent()`` etc. - may be used, but in the former case cannot. Additionally, - in polymorphic inheritance scenarios, the class-bound - descriptors refer to the columns present in the - polymorphic selectable in use, not necessarily the table - column which directly corresponds to the descriptor. For - example, a set of classes related by joined-table - inheritance to the ``person`` table along the - ``person_id`` column of each table will all have their - ``Class.person_id`` attribute mapped to the ``person_id`` - column in ``person``, and not their subclass table. - Version 0.4 would map this behavior onto table-bound - ``Column`` objects automatically. In 0.5, this automatic - conversion has been removed, so that you in fact *can* use - table-bound columns as a means to override the - translations which occur with polymorphic querying; this - allows ``Query`` to be able to create optimized selects - among joined-table or concrete-table inheritance setups, - as well as portable subqueries, etc. - -* **Session Now Synchronizes Automatically with - Transactions.** Session now synchronizes against the - transaction automatically by default, including autoflush - and autoexpire. A transaction is present at all times - unless disabled using the ``autocommit`` option. When all - three flags are set to their default, the Session recovers - gracefully after rollbacks and it's very difficult to get - stale data into the session. See the new Session - documentation for details. - -* **Implicit Order By Is Removed**. This will impact ORM - users who rely upon SA's "implicit ordering" behavior, - which states that all Query objects which don't have an - ``order_by()`` will ORDER BY the "id" or "oid" column of - the primary mapped table, and all lazy/eagerly loaded - collections apply a similar ordering. In 0.5, automatic - ordering must be explicitly configured on ``mapper()`` and - ``relation()`` objects (if desired), or otherwise when - using ``Query``. - - To convert an 0.4 mapping to 0.5, such that its ordering - behavior will be extremely similar to 0.4 or previous, use - the ``order_by`` setting on ``mapper()`` and - ``relation()``: - - :: - - mapper( - User, - users, - properties={"addresses": relation(Address, order_by=addresses.c.id)}, - order_by=users.c.id, - ) - - To set ordering on a backref, use the ``backref()`` - function: - - :: - - "keywords": relation( - Keyword, - secondary=item_keywords, - order_by=keywords.c.name, - backref=backref("items", order_by=items.c.id), - ) - - Using declarative ? To help with the new ``order_by`` - requirement, ``order_by`` and friends can now be set using - strings which are evaluated in Python later on (this works - **only** with declarative, not plain mappers): - - :: - - class MyClass(MyDeclarativeBase): - ... - "addresses": relation("Address", order_by="Address.id") - - It's generally a good idea to set ``order_by`` on - ``relation()s`` which load list-based collections of - items, since that ordering cannot otherwise be affected. - Other than that, the best practice is to use - ``Query.order_by()`` to control ordering of the primary - entities being loaded. - -* **Session is now - autoflush=True/autoexpire=True/autocommit=False.** - To - set it up, just call ``sessionmaker()`` with no arguments. - The name ``transactional=True`` is now - ``autocommit=False``. Flushes occur upon each query - issued (disable with ``autoflush=False``), within each - ``commit()`` (as always), and before each - ``begin_nested()`` (so rolling back to the SAVEPOINT is - meaningful). All objects are expired after each - ``commit()`` and after each ``rollback()``. After - rollback, pending objects are expunged, deleted objects - move back to persistent. These defaults work together - very nicely and there's really no more need for old - techniques like ``clear()`` (which is renamed to - ``expunge_all()`` as well). - - P.S.: sessions are now reusable after a ``rollback()``. - Scalar and collection attribute changes, adds and deletes - are all rolled back. - -* **session.add() replaces session.save(), session.update(), - session.save_or_update().** - the - ``session.add(someitem)`` and ``session.add_all([list of - items])`` methods replace ``save()``, ``update()``, and - ``save_or_update()``. Those methods will remain - deprecated throughout 0.5. - -* **backref configuration made less verbose.** - The - ``backref()`` function now uses the ``primaryjoin`` and - ``secondaryjoin`` arguments of the forwards-facing - ``relation()`` when they are not explicitly stated. It's - no longer necessary to specify - ``primaryjoin``/``secondaryjoin`` in both directions - separately. - -* **Simplified polymorphic options.** - The ORM's - "polymorphic load" behavior has been simplified. In 0.4, - mapper() had an argument called ``polymorphic_fetch`` - which could be configured as ``select`` or ``deferred``. - This option is removed; the mapper will now just defer any - columns which were not present in the SELECT statement. - The actual SELECT statement used is controlled by the - ``with_polymorphic`` mapper argument (which is also in 0.4 - and replaces ``select_table``), as well as the - ``with_polymorphic()`` method on ``Query`` (also in 0.4). - - An improvement to the deferred loading of inheriting - classes is that the mapper now produces the "optimized" - version of the SELECT statement in all cases; that is, if - class B inherits from A, and several attributes only - present on class B have been expired, the refresh - operation will only include B's table in the SELECT - statement and will not JOIN to A. - -* The ``execute()`` method on ``Session`` converts plain - strings into ``text()`` constructs, so that bind - parameters may all be specified as ":bindname" without - needing to call ``text()`` explicitly. If "raw" SQL is - desired here, use ``session.connection().execute("raw - text")``. - -* ``session.Query().iterate_instances()`` has been renamed - to just ``instances()``. The old ``instances()`` method - returning a list instead of an iterator no longer exists. - If you were relying on that behavior, you should use - ``list(your_query.instances())``. - -Extending the ORM -================= - -In 0.5 we're moving forward with more ways to modify and -extend the ORM. Heres a summary: - -* **MapperExtension.** - This is the classic extension - class, which remains. Methods which should rarely be - needed are ``create_instance()`` and - ``populate_instance()``. To control the initialization of - an object when it's loaded from the database, use the - ``reconstruct_instance()`` method, or more easily the - ``@reconstructor`` decorator described in the - documentation. - -* **SessionExtension.** - This is an easy to use extension - class for session events. In particular, it provides - ``before_flush()``, ``after_flush()`` and - ``after_flush_postexec()`` methods. This usage is - recommended over ``MapperExtension.before_XXX`` in many - cases since within ``before_flush()`` you can modify the - flush plan of the session freely, something which cannot - be done from within ``MapperExtension``. - -* **AttributeExtension.** - This class is now part of the - public API, and allows the interception of userland events - on attributes, including attribute set and delete - operations, and collection appends and removes. It also - allows the value to be set or appended to be modified. - The ``@validates`` decorator, described in the - documentation, provides a quick way to mark any mapped - attributes as being "validated" by a particular class - method. - -* **Attribute Instrumentation Customization.** - An API is - provided for ambitious efforts to entirely replace - SQLAlchemy's attribute instrumentation, or just to augment - it in some cases. This API was produced for the purposes - of the Trellis toolkit, but is available as a public API. - Some examples are provided in the distribution in the - ``/examples/custom_attributes`` directory. - -Schema/Types -============ - -* **String with no length no longer generates TEXT, it - generates VARCHAR** - The ``String`` type no longer - magically converts into a ``Text`` type when specified - with no length. This only has an effect when CREATE TABLE - is issued, as it will issue ``VARCHAR`` with no length - parameter, which is not valid on many (but not all) - databases. To create a TEXT (or CLOB, i.e. unbounded - string) column, use the ``Text`` type. - -* **PickleType() with mutable=True requires an __eq__() - method** - The ``PickleType`` type needs to compare values - when mutable=True. The method of comparing - ``pickle.dumps()`` is inefficient and unreliable. If an - incoming object does not implement ``__eq__()`` and is - also not ``None``, the ``dumps()`` comparison is used but - a warning is raised. For types which implement - ``__eq__()`` which includes all dictionaries, lists, etc., - comparison will use ``==`` and is now reliable by default. - -* **convert_bind_param() and convert_result_value() methods - of TypeEngine/TypeDecorator are removed.** - The O'Reilly - book unfortunately documented these methods even though - they were deprecated post 0.3. For a user-defined type - which subclasses ``TypeEngine``, the ``bind_processor()`` - and ``result_processor()`` methods should be used for - bind/result processing. Any user defined type, whether - extending ``TypeEngine`` or ``TypeDecorator``, which uses - the old 0.3 style can be easily adapted to the new style - using the following adapter: - - :: - - class AdaptOldConvertMethods(object): - """A mixin which adapts 0.3-style convert_bind_param and - convert_result_value methods - - """ - - def bind_processor(self, dialect): - def convert(value): - return self.convert_bind_param(value, dialect) - - return convert - - def result_processor(self, dialect): - def convert(value): - return self.convert_result_value(value, dialect) - - return convert - - def convert_result_value(self, value, dialect): - return value - - def convert_bind_param(self, value, dialect): - return value - - To use the above mixin: - - :: - - class MyType(AdaptOldConvertMethods, TypeEngine): - ... - -* The ``quote`` flag on ``Column`` and ``Table`` as well as - the ``quote_schema`` flag on ``Table`` now control quoting - both positively and negatively. The default is ``None``, - meaning let regular quoting rules take effect. When - ``True``, quoting is forced on. When ``False``, quoting - is forced off. - -* Column ``DEFAULT`` value DDL can now be more conveniently - specified with ``Column(..., server_default='val')``, - deprecating ``Column(..., PassiveDefault('val'))``. - ``default=`` is now exclusively for Python-initiated - default values, and can coexist with server_default. A - new ``server_default=FetchedValue()`` replaces the - ``PassiveDefault('')`` idiom for marking columns as - subject to influence from external triggers and has no DDL - side effects. - -* SQLite's ``DateTime``, ``Time`` and ``Date`` types now - **only accept datetime objects, not strings** as bind - parameter input. If you'd like to create your own - "hybrid" type which accepts strings and returns results as - date objects (from whatever format you'd like), create a - ``TypeDecorator`` that builds on ``String``. If you only - want string-based dates, just use ``String``. - -* Additionally, the ``DateTime`` and ``Time`` types, when - used with SQLite, now represent the "microseconds" field - of the Python ``datetime.datetime`` object in the same - manner as ``str(datetime)`` - as fractional seconds, not a - count of microseconds. That is: - - :: - - dt = datetime.datetime(2008, 6, 27, 12, 0, 0, 125) # 125 usec - - # old way - "2008-06-27 12:00:00.125" - - # new way - "2008-06-27 12:00:00.000125" - - So if an existing SQLite file-based database intends to be - used across 0.4 and 0.5, you either have to upgrade the - datetime columns to store the new format (NOTE: please - test this, I'm pretty sure its correct): - - .. sourcecode:: sql - - UPDATE mytable SET somedatecol = - substr(somedatecol, 0, 19) || '.' || substr((substr(somedatecol, 21, -1) / 1000000), 3, -1); - - or, enable "legacy" mode as follows: - - :: - - from sqlalchemy.databases.sqlite import DateTimeMixin - - DateTimeMixin.__legacy_microseconds__ = True - -Connection Pool no longer threadlocal by default -================================================ - -0.4 has an unfortunate default setting of -"pool_threadlocal=True", leading to surprise behavior when, -for example, using multiple Sessions within a single thread. -This flag is now off in 0.5. To re-enable 0.4's behavior, -specify ``pool_threadlocal=True`` to ``create_engine()``, or -alternatively use the "threadlocal" strategy via -``strategy="threadlocal"``. - -\*args Accepted, \*args No Longer Accepted -========================================== - -The policy with ``method(\*args)`` vs. ``method([args])`` -is, if the method accepts a variable-length set of items -which represent a fixed structure, it takes ``\*args``. If -the method accepts a variable-length set of items that are -data-driven, it takes ``[args]``. - -* The various Query.options() functions ``eagerload()``, - ``eagerload_all()``, ``lazyload()``, ``contains_eager()``, - ``defer()``, ``undefer()`` all accept variable-length - ``\*keys`` as their argument now, which allows a path to - be formulated using descriptors, ie.: - - :: - - query.options(eagerload_all(User.orders, Order.items, Item.keywords)) - - A single array argument is still accepted for backwards - compatibility. - -* Similarly, the ``Query.join()`` and ``Query.outerjoin()`` - methods accept a variable length \*args, with a single - array accepted for backwards compatibility: - - :: - - query.join("orders", "items") - query.join(User.orders, Order.items) - -* the ``in_()`` method on columns and similar only accepts a - list argument now. It no longer accepts ``\*args``. - -Removed -======= - -* **entity_name** - This feature was always problematic and - rarely used. 0.5's more deeply fleshed out use cases - revealed further issues with ``entity_name`` which led to - its removal. If different mappings are required for a - single class, break the class into separate subclasses and - map them separately. An example of this is at - [wiki:UsageRecipes/EntityName]. More information - regarding rationale is described at https://groups.google.c - om/group/sqlalchemy/browse_thread/thread/9e23a0641a88b96d? - hl=en . - -* **get()/load() cleanup** - - - The ``load()`` method has been removed. Its - functionality was kind of arbitrary and basically copied - from Hibernate, where it's also not a particularly - meaningful method. - - To get equivalent functionality: - - :: - - x = session.query(SomeClass).populate_existing().get(7) - - ``Session.get(cls, id)`` and ``Session.load(cls, id)`` - have been removed. ``Session.get()`` is redundant vs. - ``session.query(cls).get(id)``. - - ``MapperExtension.get()`` is also removed (as is - ``MapperExtension.load()``). To override the - functionality of ``Query.get()``, use a subclass: - - :: - - class MyQuery(Query): - def get(self, ident): - ... - - - session = sessionmaker(query_cls=MyQuery)() - - ad1 = session.query(Address).get(1) - -* ``sqlalchemy.orm.relation()`` - - - The following deprecated keyword arguments have been - removed: - - foreignkey, association, private, attributeext, is_backref - - In particular, ``attributeext`` is replaced with - ``extension`` - the ``AttributeExtension`` class is now in - the public API. - -* ``session.Query()`` - - - The following deprecated functions have been removed: - - list, scalar, count_by, select_whereclause, get_by, - select_by, join_by, selectfirst, selectone, select, - execute, select_statement, select_text, join_to, join_via, - selectfirst_by, selectone_by, apply_max, apply_min, - apply_avg, apply_sum - - Additionally, the ``id`` keyword argument to ``join()``, - ``outerjoin()``, ``add_entity()`` and ``add_column()`` has - been removed. To target table aliases in ``Query`` to - result columns, use the ``aliased`` construct: - - :: - - from sqlalchemy.orm import aliased - - address_alias = aliased(Address) - print(session.query(User, address_alias).join((address_alias, User.addresses)).all()) - -* ``sqlalchemy.orm.Mapper`` - - - * instances() - - - * get_session() - this method was not very noticeable, but - had the effect of associating lazy loads with a - particular session even if the parent object was - entirely detached, when an extension such as - ``scoped_session()`` or the old ``SessionContextExt`` - was used. It's possible that some applications which - relied upon this behavior will no longer work as - expected; but the better programming practice here is - to always ensure objects are present within sessions if - database access from their attributes are required. - -* ``mapper(MyClass, mytable)`` - - - Mapped classes no are longer instrumented with a "c" class - attribute; e.g. ``MyClass.c`` - -* ``sqlalchemy.orm.collections`` - - - The _prepare_instrumentation alias for - prepare_instrumentation has been removed. - -* ``sqlalchemy.orm`` - - - Removed the ``EXT_PASS`` alias of ``EXT_CONTINUE``. - -* ``sqlalchemy.engine`` - - - The alias from ``DefaultDialect.preexecute_sequences`` to - ``.preexecute_pk_sequences`` has been removed. - - The deprecated engine_descriptors() function has been - removed. - -* ``sqlalchemy.ext.activemapper`` - - - Module removed. - -* ``sqlalchemy.ext.assignmapper`` - - - Module removed. - -* ``sqlalchemy.ext.associationproxy`` - - - Pass-through of keyword args on the proxy's - ``.append(item, \**kw)`` has been removed and is now - simply ``.append(item)`` - -* ``sqlalchemy.ext.selectresults``, - ``sqlalchemy.mods.selectresults`` - - Modules removed. - -* ``sqlalchemy.ext.declarative`` - - - ``declared_synonym()`` removed. - -* ``sqlalchemy.ext.sessioncontext`` - - - Module removed. - -* ``sqlalchemy.log`` - - - The ``SADeprecationWarning`` alias to - ``sqlalchemy.exc.SADeprecationWarning`` has been removed. - -* ``sqlalchemy.exc`` - - - ``exc.AssertionError`` has been removed and usage replaced - by the Python built-in of the same name. - -* ``sqlalchemy.databases.mysql`` - - - The deprecated ``get_version_info`` dialect method has - been removed. - -Renamed or Moved -================ - -* ``sqlalchemy.exceptions`` is now ``sqlalchemy.exc`` - - - The module may still be imported under the old name until - 0.6. - -* ``FlushError``, ``ConcurrentModificationError``, - ``UnmappedColumnError`` -> sqlalchemy.orm.exc - - These exceptions moved to the orm package. Importing - 'sqlalchemy.orm' will install aliases in sqlalchemy.exc - for compatibility until 0.6. - -* ``sqlalchemy.logging`` -> ``sqlalchemy.log`` - - - This internal module was renamed. No longer needs to be - special cased when packaging SA with py2app and similar - tools that scan imports. - -* ``session.Query().iterate_instances()`` -> - ``session.Query().instances()``. - -Deprecated -========== - -* ``Session.save()``, ``Session.update()``, - ``Session.save_or_update()`` - - All three replaced by ``Session.add()`` - -* ``sqlalchemy.PassiveDefault`` - - - Use ``Column(server_default=...)`` Translates to - sqlalchemy.DefaultClause() under the hood. - -* ``session.Query().iterate_instances()``. It has been - renamed to ``instances()``. - diff --git a/doc/build/changelog/migration_06.rst b/doc/build/changelog/migration_06.rst deleted file mode 100644 index 0330ac5d4a4..00000000000 --- a/doc/build/changelog/migration_06.rst +++ /dev/null @@ -1,1241 +0,0 @@ -============================= -What's New in SQLAlchemy 0.6? -============================= - -.. admonition:: About this Document - - This document describes changes between SQLAlchemy version 0.5, - last released January 16, 2010, and SQLAlchemy version 0.6, - last released May 5, 2012. - - Document date: June 6, 2010 - -This guide documents API changes which affect users -migrating their applications from the 0.5 series of -SQLAlchemy to 0.6. Note that SQLAlchemy 0.6 removes some -behaviors which were deprecated throughout the span of the -0.5 series, and also deprecates more behaviors specific to -0.5. - -Platform Support -================ - -* cPython versions 2.4 and upwards throughout the 2.xx - series - -* Jython 2.5.1 - using the zxJDBC DBAPI included with - Jython. - -* cPython 3.x - see [source:sqlalchemy/trunk/README.py3k] - for information on how to build for python3. - -New Dialect System -================== - -Dialect modules are now broken up into distinct -subcomponents, within the scope of a single database -backend. Dialect implementations are now in the -``sqlalchemy.dialects`` package. The -``sqlalchemy.databases`` package still exists as a -placeholder to provide some level of backwards compatibility -for simple imports. - -For each supported database, a sub-package exists within -``sqlalchemy.dialects`` where several files are contained. -Each package contains a module called ``base.py`` which -defines the specific SQL dialect used by that database. It -also contains one or more "driver" modules, each one -corresponding to a specific DBAPI - these files are named -corresponding to the DBAPI itself, such as ``pysqlite``, -``cx_oracle``, or ``pyodbc``. The classes used by -SQLAlchemy dialects are first declared in the ``base.py`` -module, defining all behavioral characteristics defined by -the database. These include capability mappings, such as -"supports sequences", "supports returning", etc., type -definitions, and SQL compilation rules. Each "driver" -module in turn provides subclasses of those classes as -needed which override the default behavior to accommodate -the additional features, behaviors, and quirks of that -DBAPI. For DBAPIs that support multiple backends (pyodbc, -zxJDBC, mxODBC), the dialect module will use mixins from the -``sqlalchemy.connectors`` package, which provide -functionality common to that DBAPI across all backends, most -typically dealing with connect arguments. This means that -connecting using pyodbc, zxJDBC or mxODBC (when implemented) -is extremely consistent across supported backends. - -The URL format used by ``create_engine()`` has been enhanced -to handle any number of DBAPIs for a particular backend, -using a scheme that is inspired by that of JDBC. The -previous format still works, and will select a "default" -DBAPI implementation, such as the PostgreSQL URL below that -will use psycopg2: - -:: - - create_engine("postgresql://scott:tiger@localhost/test") - -However to specify a specific DBAPI backend such as pg8000, -add it to the "protocol" section of the URL using a plus -sign "+": - -:: - - create_engine("postgresql+pg8000://scott:tiger@localhost/test") - -Important Dialect Links: - -* Documentation on connect arguments: - https://www.sqlalchemy.org/docs/06/dbengine.html#create- - engine-url-arguments. - -* Reference documentation for individual dialects: https://ww - w.sqlalchemy.org/docs/06/reference/dialects/index.html - -* The tips and tricks at DatabaseNotes. - - -Other notes regarding dialects: - -* the type system has been changed dramatically in - SQLAlchemy 0.6. This has an impact on all dialects - regarding naming conventions, behaviors, and - implementations. See the section on "Types" below. - -* the ``ResultProxy`` object now offers a 2x speed - improvement in some cases thanks to some refactorings. - -* the ``RowProxy``, i.e. individual result row object, is - now directly pickleable. - -* the setuptools entrypoint used to locate external dialects - is now called ``sqlalchemy.dialects``. An external - dialect written against 0.4 or 0.5 will need to be - modified to work with 0.6 in any case so this change does - not add any additional difficulties. - -* dialects now receive an initialize() event on initial - connection to determine connection properties. - -* Functions and operators generated by the compiler now use - (almost) regular dispatch functions of the form - "visit_" and "visit__fn" to provide - customed processing. This replaces the need to copy the - "functions" and "operators" dictionaries in compiler - subclasses with straightforward visitor methods, and also - allows compiler subclasses complete control over - rendering, as the full _Function or _BinaryExpression - object is passed in. - -Dialect Imports ---------------- - -The import structure of dialects has changed. Each dialect -now exports its base "dialect" class as well as the full set -of SQL types supported on that dialect via -``sqlalchemy.dialects.``. For example, to import a -set of PG types: - -:: - - from sqlalchemy.dialects.postgresql import ( - INTEGER, - BIGINT, - SMALLINT, - VARCHAR, - MACADDR, - DATE, - BYTEA, - ) - -Above, ``INTEGER`` is actually the plain ``INTEGER`` type -from ``sqlalchemy.types``, but the PG dialect makes it -available in the same way as those types which are specific -to PG, such as ``BYTEA`` and ``MACADDR``. - -Expression Language Changes -=========================== - -An Important Expression Language Gotcha ---------------------------------------- - -There's one quite significant behavioral change to the -expression language which may affect some applications. -The boolean value of Python boolean expressions, i.e. -``==``, ``!=``, and similar, now evaluates accurately with -regards to the two clause objects being compared. - -As we know, comparing a ``ClauseElement`` to any other -object returns another ``ClauseElement``: - -:: - - >>> from sqlalchemy.sql import column - >>> column("foo") == 5 - - -This so that Python expressions produce SQL expressions when -converted to strings: - -:: - - >>> str(column("foo") == 5) - 'foo = :foo_1' - -But what happens if we say this? - -:: - - >>> if column("foo") == 5: - ... print("yes") - -In previous versions of SQLAlchemy, the returned -``_BinaryExpression`` was a plain Python object which -evaluated to ``True``. Now it evaluates to whether or not -the actual ``ClauseElement`` should have the same hash value -as to that being compared. Meaning: - -:: - - >>> bool(column("foo") == 5) - False - >>> bool(column("foo") == column("foo")) - False - >>> c = column("foo") - >>> bool(c == c) - True - >>> - -That means code such as the following: - -:: - - if expression: - print("the expression is:", expression) - -Would not evaluate if ``expression`` was a binary clause. -Since the above pattern should never be used, the base -``ClauseElement`` now raises an exception if called in a -boolean context: - -:: - - >>> bool(c) - Traceback (most recent call last): - File "", line 1, in - ... - raise TypeError("Boolean value of this clause is not defined") - TypeError: Boolean value of this clause is not defined - -Code that wants to check for the presence of a -``ClauseElement`` expression should instead say: - -:: - - if expression is not None: - print("the expression is:", expression) - -Keep in mind, **this applies to Table and Column objects -too**. - -The rationale for the change is twofold: - -* Comparisons of the form ``if c1 == c2: `` - can actually be written now - -* Support for correct hashing of ``ClauseElement`` objects - now works on alternate platforms, namely Jython. Up until - this point SQLAlchemy relied heavily on the specific - behavior of cPython in this regard (and still had - occasional problems with it). - -Stricter "executemany" Behavior -------------------------------- - -An "executemany" in SQLAlchemy corresponds to a call to -``execute()``, passing along a collection of bind parameter -sets: - -:: - - connection.execute(table.insert(), {"data": "row1"}, {"data": "row2"}, {"data": "row3"}) - -When the ``Connection`` object sends off the given -``insert()`` construct for compilation, it passes to the -compiler the keynames present in the first set of binds -passed along to determine the construction of the -statement's VALUES clause. Users familiar with this -construct will know that additional keys present in the -remaining dictionaries don't have any impact. What's -different now is that all subsequent dictionaries need to -include at least *every* key that is present in the first -dictionary. This means that a call like this no longer -works: - -:: - - connection.execute( - table.insert(), - {"timestamp": today, "data": "row1"}, - {"timestamp": today, "data": "row2"}, - {"data": "row3"}, - ) - -Because the third row does not specify the 'timestamp' -column. Previous versions of SQLAlchemy would simply insert -NULL for these missing columns. However, if the -``timestamp`` column in the above example contained a -Python-side default value or function, it would *not* be -used. This because the "executemany" operation is optimized -for maximum performance across huge numbers of parameter -sets, and does not attempt to evaluate Python-side defaults -for those missing keys. Because defaults are often -implemented either as SQL expressions which are embedded -inline with the INSERT statement, or are server side -expressions which again are triggered based on the structure -of the INSERT string, which by definition cannot fire off -conditionally based on each parameter set, it would be -inconsistent for Python side defaults to behave differently -vs. SQL/server side defaults. (SQL expression based -defaults are embedded inline as of the 0.5 series, again to -minimize the impact of huge numbers of parameter sets). - -SQLAlchemy 0.6 therefore establishes predictable consistency -by forbidding any subsequent parameter sets from leaving any -fields blank. That way, there's no more silent failure of -Python side default values and functions, which additionally -are allowed to remain consistent in their behavior versus -SQL and server side defaults. - -UNION and other "compound" constructs parenthesize consistently ---------------------------------------------------------------- - -A rule that was designed to help SQLite has been removed, -that of the first compound element within another compound -(such as, a ``union()`` inside of an ``except_()``) wouldn't -be parenthesized. This is inconsistent and produces the -wrong results on PostgreSQL, which has precedence rules -regarding INTERSECTION, and its generally a surprise. When -using complex composites with SQLite, you now need to turn -the first element into a subquery (which is also compatible -on PG). A new example is in the SQL expression tutorial at -the end of -[https://www.sqlalchemy.org/docs/06/sqlexpression.html -#unions-and-other-set-operations]. See :ticket:`1665` and -r6690 for more background. - -C Extensions for Result Fetching -================================ - -The ``ResultProxy`` and related elements, including most -common "row processing" functions such as unicode -conversion, numerical/boolean conversions and date parsing, -have been re-implemented as optional C extensions for the -purposes of performance. This represents the beginning of -SQLAlchemy's path to the "dark side" where we hope to -continue improving performance by reimplementing critical -sections in C. The extensions can be built by specifying -``--with-cextensions``, i.e. ``python setup.py --with- -cextensions install``. - -The extensions have the most dramatic impact on result -fetching using direct ``ResultProxy`` access, i.e. that -which is returned by ``engine.execute()``, -``connection.execute()``, or ``session.execute()``. Within -results returned by an ORM ``Query`` object, result fetching -is not as high a percentage of overhead, so ORM performance -improves more modestly, and mostly in the realm of fetching -large result sets. The performance improvements highly -depend on the dbapi in use and on the syntax used to access -the columns of each row (eg ``row['name']`` is much faster -than ``row.name``). The current extensions have no impact -on the speed of inserts/updates/deletes, nor do they improve -the latency of SQL execution, that is, an application that -spends most of its time executing many statements with very -small result sets will not see much improvement. - -Performance has been improved in 0.6 versus 0.5 regardless -of the extensions. A quick overview of what connecting and -fetching 50,000 rows looks like with SQLite, using mostly -direct SQLite access, a ``ResultProxy``, and a simple mapped -ORM object: - -.. sourcecode:: text - - sqlite select/native: 0.260s - - 0.6 / C extension - - sqlalchemy.sql select: 0.360s - sqlalchemy.orm fetch: 2.500s - - 0.6 / Pure Python - - sqlalchemy.sql select: 0.600s - sqlalchemy.orm fetch: 3.000s - - 0.5 / Pure Python - - sqlalchemy.sql select: 0.790s - sqlalchemy.orm fetch: 4.030s - -Above, the ORM fetches the rows 33% faster than 0.5 due to -in-python performance enhancements. With the C extensions -we get another 20%. However, ``ResultProxy`` fetches -improve by 67% with the C extension versus not. Other -tests report as much as a 200% speed improvement for some -scenarios, such as those where lots of string conversions -are occurring. - -New Schema Capabilities -======================= - -The ``sqlalchemy.schema`` package has received some long- -needed attention. The most visible change is the newly -expanded DDL system. In SQLAlchemy, it was possible since -version 0.5 to create custom DDL strings and associate them -with tables or metadata objects: - -:: - - from sqlalchemy.schema import DDL - - DDL("CREATE TRIGGER users_trigger ...").execute_at("after-create", metadata) - -Now the full suite of DDL constructs are available under the -same system, including those for CREATE TABLE, ADD -CONSTRAINT, etc.: - -:: - - from sqlalchemy.schema import Constraint, AddConstraint - - AddContraint(CheckConstraint("value > 5")).execute_at("after-create", mytable) - -Additionally, all the DDL objects are now regular -``ClauseElement`` objects just like any other SQLAlchemy -expression object: - -:: - - from sqlalchemy.schema import CreateTable - - create = CreateTable(mytable) - - # dumps the CREATE TABLE as a string - print(create) - - # executes the CREATE TABLE statement - engine.execute(create) - -and using the ``sqlalchemy.ext.compiler`` extension you can -make your own: - -:: - - from sqlalchemy.schema import DDLElement - from sqlalchemy.ext.compiler import compiles - - - class AlterColumn(DDLElement): - def __init__(self, column, cmd): - self.column = column - self.cmd = cmd - - - @compiles(AlterColumn) - def visit_alter_column(element, compiler, **kw): - return "ALTER TABLE %s ALTER COLUMN %s %s ..." % ( - element.column.table.name, - element.column.name, - element.cmd, - ) - - - engine.execute(AlterColumn(table.c.mycolumn, "SET DEFAULT 'test'")) - -Deprecated/Removed Schema Elements ----------------------------------- - -The schema package has also been greatly streamlined. Many -options and methods which were deprecated throughout 0.5 -have been removed. Other little known accessors and methods -have also been removed. - -* the "owner" keyword argument is removed from ``Table``. - Use "schema" to represent any namespaces to be prepended - to the table name. - -* deprecated ``MetaData.connect()`` and - ``ThreadLocalMetaData.connect()`` have been removed - send - the "bind" attribute to bind a metadata. - -* deprecated metadata.table_iterator() method removed (use - sorted_tables) - -* the "metadata" argument is removed from - ``DefaultGenerator`` and subclasses, but remains locally - present on ``Sequence``, which is a standalone construct - in DDL. - -* deprecated ``PassiveDefault`` - use ``DefaultClause``. - - -* Removed public mutability from ``Index`` and - ``Constraint`` objects: - - * ``ForeignKeyConstraint.append_element()`` - - - * ``Index.append_column()`` - - - * ``UniqueConstraint.append_column()`` - - - * ``PrimaryKeyConstraint.add()`` - - - * ``PrimaryKeyConstraint.remove()`` - - -These should be constructed declaratively (i.e. in one -construction). - -* Other removed things: - - - * ``Table.key`` (no idea what this was for) - - - * ``Column.bind`` (get via column.table.bind) - - - * ``Column.metadata`` (get via column.table.metadata) - - - * ``Column.sequence`` (use column.default) - - -Other Behavioral Changes ------------------------- - -* ``UniqueConstraint``, ``Index``, ``PrimaryKeyConstraint`` - all accept lists of column names or column objects as - arguments. - -* The ``use_alter`` flag on ``ForeignKey`` is now a shortcut - option for operations that can be hand-constructed using - the ``DDL()`` event system. A side effect of this refactor - is that ``ForeignKeyConstraint`` objects with - ``use_alter=True`` will *not* be emitted on SQLite, which - does not support ALTER for foreign keys. This has no - effect on SQLite's behavior since SQLite does not actually - honor FOREIGN KEY constraints. - -* ``Table.primary_key`` is not assignable - use - ``table.append_constraint(PrimaryKeyConstraint(...))`` - -* A ``Column`` definition with a ``ForeignKey`` and no type, - e.g. ``Column(name, ForeignKey(sometable.c.somecol))`` - used to get the type of the referenced column. Now support - for that automatic type inference is partial and may not - work in all cases. - -Logging opened up -================= - -At the expense of a few extra method calls here and there, -you can set log levels for INFO and DEBUG after an engine, -pool, or mapper has been created, and logging will commence. -The ``isEnabledFor(INFO)`` method is now called -per-``Connection`` and ``isEnabledFor(DEBUG)`` -per-``ResultProxy`` if already enabled on the parent -connection. Pool logging sends to ``log.info()`` and -``log.debug()`` with no check - note that pool -checkout/checkin is typically once per transaction. - -Reflection/Inspector API -======================== - -The reflection system, which allows reflection of table -columns via ``Table('sometable', metadata, autoload=True)`` -has been opened up into its own fine-grained API, which -allows direct inspection of database elements such as -tables, columns, constraints, indexes, and more. This API -expresses return values as simple lists of strings, -dictionaries, and ``TypeEngine`` objects. The internals of -``autoload=True`` now build upon this system such that the -translation of raw database information into -``sqlalchemy.schema`` constructs is centralized and the -contract of individual dialects greatly simplified, vastly -reducing bugs and inconsistencies across different backends. - -To use an inspector: - -:: - - from sqlalchemy.engine.reflection import Inspector - - insp = Inspector.from_engine(my_engine) - - print(insp.get_schema_names()) - -the ``from_engine()`` method will in some cases provide a -backend-specific inspector with additional capabilities, -such as that of PostgreSQL which provides a -``get_table_oid()`` method: - -:: - - - my_engine = create_engine("postgresql://...") - pg_insp = Inspector.from_engine(my_engine) - - print(pg_insp.get_table_oid("my_table")) - -RETURNING Support -================= - -The ``insert()``, ``update()`` and ``delete()`` constructs -now support a ``returning()`` method, which corresponds to -the SQL RETURNING clause as supported by PostgreSQL, Oracle, -MS-SQL, and Firebird. It is not supported for any other -backend at this time. - -Given a list of column expressions in the same manner as -that of a ``select()`` construct, the values of these -columns will be returned as a regular result set: - -:: - - - result = connection.execute( - table.insert().values(data="some data").returning(table.c.id, table.c.timestamp) - ) - row = result.first() - print("ID:", row["id"], "Timestamp:", row["timestamp"]) - -The implementation of RETURNING across the four supported -backends varies wildly, in the case of Oracle requiring an -intricate usage of OUT parameters which are re-routed into a -"mock" result set, and in the case of MS-SQL using an -awkward SQL syntax. The usage of RETURNING is subject to -limitations: - -* it does not work for any "executemany()" style of - execution. This is a limitation of all supported DBAPIs. - -* Some backends, such as Oracle, only support RETURNING that - returns a single row - this includes UPDATE and DELETE - statements, meaning the update() or delete() construct - must match only a single row, or an error is raised (by - Oracle, not SQLAlchemy). - -RETURNING is also used automatically by SQLAlchemy, when -available and when not otherwise specified by an explicit -``returning()`` call, to fetch the value of newly generated -primary key values for single-row INSERT statements. This -means there's no more "SELECT nextval(sequence)" pre- -execution for insert statements where the primary key value -is required. Truth be told, implicit RETURNING feature -does incur more method overhead than the old "select -nextval()" system, which used a quick and dirty -cursor.execute() to get at the sequence value, and in the -case of Oracle requires additional binding of out -parameters. So if method/protocol overhead is proving to be -more expensive than additional database round trips, the -feature can be disabled by specifying -``implicit_returning=False`` to ``create_engine()``. - -Type System Changes -=================== - -New Architecture ----------------- - -The type system has been completely reworked behind the -scenes to provide two goals: - -* Separate the handling of bind parameters and result row - values, typically a DBAPI requirement, from the SQL - specification of the type itself, which is a database - requirement. This is consistent with the overall dialect - refactor that separates database SQL behavior from DBAPI. - -* Establish a clear and consistent contract for generating - DDL from a ``TypeEngine`` object and for constructing - ``TypeEngine`` objects based on column reflection. - -Highlights of these changes include: - -* The construction of types within dialects has been totally - overhauled. Dialects now define publicly available types - as UPPERCASE names exclusively, and internal - implementation types using underscore identifiers (i.e. - are private). The system by which types are expressed in - SQL and DDL has been moved to the compiler system. This - has the effect that there are much fewer type objects - within most dialects. A detailed document on this - architecture for dialect authors is in [source:/lib/sqlalc - hemy/dialects/type_migration_guidelines.txt]. - -* Reflection of types now returns the exact UPPERCASE type - within types.py, or the UPPERCASE type within the dialect - itself if the type is not a standard SQL type. This means - reflection now returns more accurate information about - reflected types. - -* User defined types that subclass ``TypeEngine`` and wish - to provide ``get_col_spec()`` should now subclass - ``UserDefinedType``. - -* The ``result_processor()`` method on all type classes now - accepts an additional argument ``coltype``. This is the - DBAPI type object attached to cursor.description, and - should be used when applicable to make better decisions on - what kind of result-processing callable should be - returned. Ideally result processor functions would never - need to use ``isinstance()``, which is an expensive call - at this level. - -Native Unicode Mode -------------------- - -As more DBAPIs support returning Python unicode objects -directly, the base dialect now performs a check upon the -first connection which establishes whether or not the DBAPI -returns a Python unicode object for a basic select of a -VARCHAR value. If so, the ``String`` type and all -subclasses (i.e. ``Text``, ``Unicode``, etc.) will skip the -"unicode" check/conversion step when result rows are -received. This offers a dramatic performance increase for -large result sets. The "unicode mode" currently is known to -work with: - -* sqlite3 / pysqlite - - -* psycopg2 - SQLA 0.6 now uses the "UNICODE" type extension - by default on each psycopg2 connection object - -* pg8000 - - -* cx_oracle (we use an output processor - nice feature !) - - -Other types may choose to disable unicode processing as -needed, such as the ``NVARCHAR`` type when used with MS-SQL. - -In particular, if porting an application based on a DBAPI -that formerly returned non-unicode strings, the "native -unicode" mode has a plainly different default behavior - -columns that are declared as ``String`` or ``VARCHAR`` now -return unicode by default whereas they would return strings -before. This can break code which expects non-unicode -strings. The psycopg2 "native unicode" mode can be -disabled by passing ``use_native_unicode=False`` to -``create_engine()``. - -A more general solution for string columns that explicitly -do not want a unicode object is to use a ``TypeDecorator`` -that converts unicode back to utf-8, or whatever is desired: - -:: - - class UTF8Encoded(TypeDecorator): - """Unicode type which coerces to utf-8.""" - - impl = sa.VARCHAR - - def process_result_value(self, value, dialect): - if isinstance(value, unicode): - value = value.encode("utf-8") - return value - -Note that the ``assert_unicode`` flag is now deprecated. -SQLAlchemy allows the DBAPI and backend database in use to -handle Unicode parameters when available, and does not add -operational overhead by checking the incoming type; modern -systems like sqlite and PostgreSQL will raise an encoding -error on their end if invalid data is passed. In those -cases where SQLAlchemy does need to coerce a bind parameter -from Python Unicode to an encoded string, or when the -Unicode type is used explicitly, a warning is raised if the -object is a bytestring. This warning can be suppressed or -converted to an exception using the Python warnings filter -documented at: https://docs.python.org/library/warnings.html - -Generic Enum Type ------------------ - -We now have an ``Enum`` in the ``types`` module. This is a -string type that is given a collection of "labels" which -constrain the possible values given to those labels. By -default, this type generates a ``VARCHAR`` using the size of -the largest label, and applies a CHECK constraint to the -table within the CREATE TABLE statement. When using MySQL, -the type by default uses MySQL's ENUM type, and when using -PostgreSQL the type will generate a user defined type using -``CREATE TYPE AS ENUM``. In order to create the -type using PostgreSQL, the ``name`` parameter must be -specified to the constructor. The type also accepts a -``native_enum=False`` option which will issue the -VARCHAR/CHECK strategy for all databases. Note that -PostgreSQL ENUM types currently don't work with pg8000 or -zxjdbc. - -Reflection Returns Dialect-Specific Types ------------------------------------------ - -Reflection now returns the most specific type possible from -the database. That is, if you create a table using -``String``, then reflect it back, the reflected column will -likely be ``VARCHAR``. For dialects that support a more -specific form of the type, that's what you'll get. So a -``Text`` type would come back as ``oracle.CLOB`` on Oracle, -a ``LargeBinary`` might be an ``mysql.MEDIUMBLOB`` etc. The -obvious advantage here is that reflection preserves as much -information possible from what the database had to say. - -Some applications that deal heavily in table metadata may -wish to compare types across reflected tables and/or non- -reflected tables. There's a semi-private accessor available -on ``TypeEngine`` called ``_type_affinity`` and an -associated comparison helper ``_compare_type_affinity``. -This accessor returns the "generic" ``types`` class which -the type corresponds to: - -:: - - >>> String(50)._compare_type_affinity(postgresql.VARCHAR(50)) - True - >>> Integer()._compare_type_affinity(mysql.REAL) - False - -Miscellaneous API Changes -------------------------- - -The usual "generic" types are still the general system in -use, i.e. ``String``, ``Float``, ``DateTime``. There's a -few changes there: - -* Types no longer make any guesses as to default parameters. - In particular, ``Numeric``, ``Float``, as well as - subclasses NUMERIC, FLOAT, DECIMAL don't generate any - length or scale unless specified. This also continues to - include the controversial ``String`` and ``VARCHAR`` types - (although MySQL dialect will pre-emptively raise when - asked to render VARCHAR with no length). No defaults are - assumed, and if they are used in a CREATE TABLE statement, - an error will be raised if the underlying database does - not allow non-lengthed versions of these types. - -* the ``Binary`` type has been renamed to ``LargeBinary``, - for BLOB/BYTEA/similar types. For ``BINARY`` and - ``VARBINARY``, those are present directly as - ``types.BINARY``, ``types.VARBINARY``, as well as in the - MySQL and MS-SQL dialects. - -* ``PickleType`` now uses == for comparison of values when - mutable=True, unless the "comparator" argument with a - comparison function is specified to the type. If you are - pickling a custom object you should implement an - ``__eq__()`` method so that value-based comparisons are - accurate. - -* The default "precision" and "scale" arguments of Numeric - and Float have been removed and now default to None. - NUMERIC and FLOAT will be rendered with no numeric - arguments by default unless these values are provided. - -* DATE, TIME and DATETIME types on SQLite can now take - optional "storage_format" and "regexp" argument. - "storage_format" can be used to store those types using a - custom string format. "regexp" allows to use a custom - regular expression to match string values from the - database. - -* ``__legacy_microseconds__`` on SQLite ``Time`` and - ``DateTime`` types is not supported anymore. You should - use the new "storage_format" argument instead. - -* ``DateTime`` types on SQLite now use by a default a - stricter regular expression to match strings from the - database. Use the new "regexp" argument if you are using - data stored in a legacy format. - -ORM Changes -=========== - -Upgrading an ORM application from 0.5 to 0.6 should require -little to no changes, as the ORM's behavior remains almost -identical. There are some default argument and name -changes, and some loading behaviors have been improved. - -New Unit of Work ----------------- - -The internals for the unit of work, primarily -``topological.py`` and ``unitofwork.py``, have been -completely rewritten and are vastly simplified. This -should have no impact on usage, as all existing behavior -during flush has been maintained exactly (or at least, as -far as it is exercised by our testsuite and the handful of -production environments which have tested it heavily). The -performance of flush() now uses 20-30% fewer method calls -and should also use less memory. The intent and flow of the -source code should now be reasonably easy to follow, and the -architecture of the flush is fairly open-ended at this -point, creating room for potential new areas of -sophistication. The flush process no longer has any -reliance on recursion so flush plans of arbitrary size and -complexity can be flushed. Additionally, the mapper's -"save" process, which issues INSERT and UPDATE statements, -now caches the "compiled" form of the two statements so that -callcounts are further dramatically reduced with very large -flushes. - -Any changes in behavior observed with flush versus earlier -versions of 0.6 or 0.5 should be reported to us ASAP - we'll -make sure no functionality is lost. - -Changes to ``query.update()`` and ``query.delete()`` ----------------------------------------------------- - -* the 'expire' option on query.update() has been renamed to - 'fetch', thus matching that of query.delete() - -* ``query.update()`` and ``query.delete()`` both default to - 'evaluate' for the synchronize strategy. - -* the 'synchronize' strategy for update() and delete() - raises an error on failure. There is no implicit fallback - onto "fetch". Failure of evaluation is based on the - structure of criteria, so success/failure is deterministic - based on code structure. - -``relation()`` is officially named ``relationship()`` ------------------------------------------------------ - -This to solve the long running issue that "relation" means a -"table or derived table" in relational algebra terms. The -``relation()`` name, which is less typing, will hang around -for the foreseeable future so this change should be entirely -painless. - -Subquery eager loading ----------------------- - -A new kind of eager loading is added called "subquery" -loading. This is a load that emits a second SQL query -immediately after the first which loads full collections for -all the parents in the first query, joining upwards to the -parent using INNER JOIN. Subquery loading is used similarly -to the current joined-eager loading, using the -```subqueryload()```` and ````subqueryload_all()```` options -as well as the ````lazy='subquery'```` setting on -````relationship()```. The subquery load is usually much -more efficient for loading many larger collections as it -uses INNER JOIN unconditionally and also doesn't re-load -parent rows. - -```eagerload()````, ````eagerload_all()```` is now ````joinedload()````, ````joinedload_all()``` ------------------------------------------------------------------------------------------------- - -To make room for the new subquery load feature, the existing -```eagerload()````/````eagerload_all()```` options are now -superseded by ````joinedload()```` and -````joinedload_all()````. The old names will hang around -for the foreseeable future just like ````relation()```. - -```lazy=False|None|True|'dynamic'```` now accepts ````lazy='noload'|'joined'|'subquery'|'select'|'dynamic'``` -------------------------------------------------------------------------------------------------------------- - -Continuing on the theme of loader strategies opened up, the -standard keywords for the ```lazy```` option on -````relationship()```` are now ````select```` for lazy -loading (via a SELECT issued on attribute access), -````joined```` for joined-eager loading, ````subquery```` -for subquery-eager loading, ````noload```` for no loading -should occur, and ````dynamic```` for a "dynamic" -relationship. The old ````True````, ````False````, -````None``` arguments are still accepted with the identical -behavior as before. - -innerjoin=True on relation, joinedload --------------------------------------- - -Joined-eagerly loaded scalars and collections can now be -instructed to use INNER JOIN instead of OUTER JOIN. On -PostgreSQL this is observed to provide a 300-600% speedup on -some queries. Set this flag for any many-to-one which is -on a NOT NULLable foreign key, and similarly for any -collection where related items are guaranteed to exist. - -At mapper level: - -:: - - mapper(Child, child) - mapper( - Parent, - parent, - properties={"child": relationship(Child, lazy="joined", innerjoin=True)}, - ) - -At query time level: - -:: - - session.query(Parent).options(joinedload(Parent.child, innerjoin=True)).all() - -The ``innerjoin=True`` flag at the ``relationship()`` level -will also take effect for any ``joinedload()`` option which -does not override the value. - -Many-to-one Enhancements ------------------------- - -* many-to-one relations now fire off a lazyload in fewer - cases, including in most cases will not fetch the "old" - value when a new one is replaced. - -* many-to-one relation to a joined-table subclass now uses - get() for a simple load (known as the "use_get" - condition), i.e. ``Related``->``Sub(Base)``, without the - need to redefine the primaryjoin condition in terms of the - base table. [ticket:1186] - -* specifying a foreign key with a declarative column, i.e. - ``ForeignKey(MyRelatedClass.id)`` doesn't break the - "use_get" condition from taking place [ticket:1492] - -* relationship(), joinedload(), and joinedload_all() now - feature an option called "innerjoin". Specify ``True`` or - ``False`` to control whether an eager join is constructed - as an INNER or OUTER join. Default is ``False`` as always. - The mapper options will override whichever setting is - specified on relationship(). Should generally be set for - many-to-one, not nullable foreign key relations to allow - improved join performance. [ticket:1544] - -* the behavior of joined eager loading such that the main - query is wrapped in a subquery when LIMIT/OFFSET are - present now makes an exception for the case when all eager - loads are many-to-one joins. In those cases, the eager - joins are against the parent table directly along with the - limit/offset without the extra overhead of a subquery, - since a many-to-one join does not add rows to the result. - - For example, in 0.5 this query: - - :: - - session.query(Address).options(eagerload(Address.user)).limit(10) - - would produce SQL like: - - .. sourcecode:: sql - - SELECT * FROM - (SELECT * FROM addresses LIMIT 10) AS anon_1 - LEFT OUTER JOIN users AS users_1 ON users_1.id = anon_1.addresses_user_id - - This because the presence of any eager loaders suggests - that some or all of them may relate to multi-row - collections, which would necessitate wrapping any kind of - rowcount-sensitive modifiers like LIMIT inside of a - subquery. - - In 0.6, that logic is more sensitive and can detect if all - eager loaders represent many-to-ones, in which case the - eager joins don't affect the rowcount: - - .. sourcecode:: sql - - SELECT * FROM addresses LEFT OUTER JOIN users AS users_1 ON users_1.id = addresses.user_id LIMIT 10 - -Mutable Primary Keys with Joined Table Inheritance --------------------------------------------------- - -A joined table inheritance config where the child table has -a PK that foreign keys to the parent PK can now be updated -on a CASCADE-capable database like PostgreSQL. -``mapper()`` now has an option ``passive_updates=True`` -which indicates this foreign key is updated automatically. -If on a non-cascading database like SQLite or MySQL/MyISAM, -set this flag to ``False``. A future feature enhancement -will try to get this flag to be auto-configuring based on -dialect/table style in use. - -Beaker Caching --------------- - -A promising new example of Beaker integration is in -``examples/beaker_caching``. This is a straightforward -recipe which applies a Beaker cache within the result- -generation engine of ``Query``. Cache parameters are -provided via ``query.options()``, and allows full control -over the contents of the cache. SQLAlchemy 0.6 includes -improvements to the ``Session.merge()`` method to support -this and similar recipes, as well as to provide -significantly improved performance in most scenarios. - -Other Changes -------------- - -* the "row tuple" object returned by ``Query`` when multiple - column/entities are selected is now picklable as well as - higher performing. - -* ``query.join()`` has been reworked to provide more - consistent behavior and more flexibility (includes - [ticket:1537]) - -* ``query.select_from()`` accepts multiple clauses to - produce multiple comma separated entries within the FROM - clause. Useful when selecting from multiple-homed join() - clauses. - -* the "dont_load=True" flag on ``Session.merge()`` is - deprecated and is now "load=False". - -* added "make_transient()" helper function which transforms - a persistent/ detached instance into a transient one (i.e. - deletes the instance_key and removes from any session.) - [ticket:1052] - -* the allow_null_pks flag on mapper() is deprecated and has - been renamed to allow_partial_pks. It is turned "on" by - default. This means that a row which has a non-null value - for any of its primary key columns will be considered an - identity. The need for this scenario typically only occurs - when mapping to an outer join. When set to False, a PK - that has NULLs in it will not be considered a primary key - - in particular this means a result row will come back as - None (or not be filled into a collection), and new in 0.6 - also indicates that session.merge() won't issue a round - trip to the database for such a PK value. [ticket:1680] - -* the mechanics of "backref" have been fully merged into the - finer grained "back_populates" system, and take place - entirely within the ``_generate_backref()`` method of - ``RelationProperty``. This makes the initialization - procedure of ``RelationProperty`` simpler and allows - easier propagation of settings (such as from subclasses of - ``RelationProperty``) into the reverse reference. The - internal ``BackRef()`` is gone and ``backref()`` returns a - plain tuple that is understood by ``RelationProperty``. - -* the keys attribute of ``ResultProxy`` is now a method, so - references to it (``result.keys``) must be changed to - method invocations (``result.keys()``) - -* ``ResultProxy.last_inserted_ids`` is now deprecated, use - ``ResultProxy.inserted_primary_key`` instead. - -Deprecated/Removed ORM Elements -------------------------------- - -Most elements that were deprecated throughout 0.5 and raised -deprecation warnings have been removed (with a few -exceptions). All elements that were marked "pending -deprecation" are now deprecated and will raise a warning -upon use. - -* 'transactional' flag on sessionmaker() and others is - removed. Use 'autocommit=True' to indicate - 'transactional=False'. - -* 'polymorphic_fetch' argument on mapper() is removed. - Loading can be controlled using the 'with_polymorphic' - option. - -* 'select_table' argument on mapper() is removed. Use - 'with_polymorphic=("*", )' for this - functionality. - -* 'proxy' argument on synonym() is removed. This flag did - nothing throughout 0.5, as the "proxy generation" - behavior is now automatic. - -* Passing a single list of elements to joinedload(), - joinedload_all(), contains_eager(), lazyload(), defer(), - and undefer() instead of multiple positional \*args is - deprecated. - -* Passing a single list of elements to query.order_by(), - query.group_by(), query.join(), or query.outerjoin() - instead of multiple positional \*args is deprecated. - -* ``query.iterate_instances()`` is removed. Use - ``query.instances()``. - -* ``Query.query_from_parent()`` is removed. Use the - sqlalchemy.orm.with_parent() function to produce a - "parent" clause, or alternatively ``query.with_parent()``. - -* ``query._from_self()`` is removed, use - ``query.from_self()`` instead. - -* the "comparator" argument to composite() is removed. Use - "comparator_factory". - -* ``RelationProperty._get_join()`` is removed. - - -* the 'echo_uow' flag on Session is removed. Use logging - on the "sqlalchemy.orm.unitofwork" name. - -* ``session.clear()`` is removed. use - ``session.expunge_all()``. - -* ``session.save()``, ``session.update()``, - ``session.save_or_update()`` are removed. Use - ``session.add()`` and ``session.add_all()``. - -* the "objects" flag on session.flush() remains deprecated. - - -* the "dont_load=True" flag on session.merge() is deprecated - in favor of "load=False". - -* ``ScopedSession.mapper`` remains deprecated. See the - usage recipe at https://www.sqlalchemy.org/trac/wiki/Usag - eRecipes/SessionAwareMapper - -* passing an ``InstanceState`` (internal SQLAlchemy state - object) to ``attributes.init_collection()`` or - ``attributes.get_history()`` is deprecated. These - functions are public API and normally expect a regular - mapped object instance. - -* the 'engine' parameter to ``declarative_base()`` is - removed. Use the 'bind' keyword argument. - -Extensions -========== - -SQLSoup -------- - -SQLSoup has been modernized and updated to reflect common -0.5/0.6 capabilities, including well defined session -integration. Please read the new docs at [https://www.sqlalc -hemy.org/docs/06/reference/ext/sqlsoup.html]. - -Declarative ------------ - -The ``DeclarativeMeta`` (default metaclass for -``declarative_base``) previously allowed subclasses to -modify ``dict_`` to add class attributes (e.g. columns). -This no longer works, the ``DeclarativeMeta`` constructor -now ignores ``dict_``. Instead, the class attributes should -be assigned directly, e.g. ``cls.id=Column(...)``, or the -`MixIn class `_ approach should be used -instead of the metaclass approach. - diff --git a/doc/build/changelog/migration_07.rst b/doc/build/changelog/migration_07.rst deleted file mode 100644 index 19716ad3c4c..00000000000 --- a/doc/build/changelog/migration_07.rst +++ /dev/null @@ -1,1369 +0,0 @@ -============================= -What's New in SQLAlchemy 0.7? -============================= - -.. admonition:: About this Document - - This document describes changes between SQLAlchemy version 0.6, - last released May 5, 2012, and SQLAlchemy version 0.7, - undergoing maintenance releases as of October, 2012. - - Document date: July 27, 2011 - -Introduction -============ - -This guide introduces what's new in SQLAlchemy version 0.7, -and also documents changes which affect users migrating -their applications from the 0.6 series of SQLAlchemy to 0.7. - -To as great a degree as possible, changes are made in such a -way as to not break compatibility with applications built -for 0.6. The changes that are necessarily not backwards -compatible are very few, and all but one, the change to -mutable attribute defaults, should affect an exceedingly -small portion of applications - many of the changes regard -non-public APIs and undocumented hacks some users may have -been attempting to use. - -A second, even smaller class of non-backwards-compatible -changes is also documented. This class of change regards -those features and behaviors that have been deprecated at -least since version 0.5 and have been raising warnings since -their deprecation. These changes would only affect -applications that are still using 0.4- or early 0.5-style -APIs. As the project matures, we have fewer and fewer of -these kinds of changes with 0.x level releases, which is a -product of our API having ever fewer features that are less -than ideal for the use cases they were meant to solve. - -An array of existing functionalities have been superseded in -SQLAlchemy 0.7. There's not much difference between the -terms "superseded" and "deprecated", except that the former -has a much weaker suggestion of the old feature would ever -be removed. In 0.7, features like ``synonym`` and -``comparable_property``, as well as all the ``Extension`` -and other event classes, have been superseded. But these -"superseded" features have been re-implemented such that -their implementations live mostly outside of core ORM code, -so their continued "hanging around" doesn't impact -SQLAlchemy's ability to further streamline and refine its -internals, and we expect them to remain within the API for -the foreseeable future. - -New Features -============ - -New Event System ----------------- - -SQLAlchemy started early with the ``MapperExtension`` class, -which provided hooks into the persistence cycle of mappers. -As SQLAlchemy quickly became more componentized, pushing -mappers into a more focused configurational role, many more -"extension", "listener", and "proxy" classes popped up to -solve various activity-interception use cases in an ad-hoc -fashion. Part of this was driven by the divergence of -activities; ``ConnectionProxy`` objects wanted to provide a -system of rewriting statements and parameters; -``AttributeExtension`` provided a system of replacing -incoming values, and ``DDL`` objects had events that could -be switched off of dialect-sensitive callables. - -0.7 re-implements virtually all of these plugin points with -a new, unified approach, which retains all the -functionalities of the different systems, provides more -flexibility and less boilerplate, performs better, and -eliminates the need to learn radically different APIs for -each event subsystem. The pre-existing classes -``MapperExtension``, ``SessionExtension``, -``AttributeExtension``, ``ConnectionProxy``, -``PoolListener`` as well as the ``DDLElement.execute_at`` -method are deprecated and now implemented in terms of the -new system - these APIs remain fully functional and are -expected to remain in place for the foreseeable future. - -The new approach uses named events and user-defined -callables to associate activities with events. The API's -look and feel was driven by such diverse sources as JQuery, -Blinker, and Hibernate, and was also modified further on -several occasions during conferences with dozens of users on -Twitter, which appears to have a much higher response rate -than the mailing list for such questions. - -It also features an open-ended system of target -specification that allows events to be associated with API -classes, such as for all ``Session`` or ``Engine`` objects, -with specific instances of API classes, such as for a -specific ``Pool`` or ``Mapper``, as well as for related -objects like a user- defined class that's mapped, or -something as specific as a certain attribute on instances of -a particular subclass of a mapped parent class. Individual -listener subsystems can apply wrappers to incoming user- -defined listener functions which modify how they are called -- an mapper event can receive either the instance of the -object being operated upon, or its underlying -``InstanceState`` object. An attribute event can opt whether -or not to have the responsibility of returning a new value. - -Several systems now build upon the new event API, including -the new "mutable attributes" API as well as composite -attributes. The greater emphasis on events has also led to -the introduction of a handful of new events, including -attribute expiration and refresh operations, pickle -loads/dumps operations, completed mapper construction -operations. - -.. seealso:: - - :ref:`event_toplevel` - -:ticket:`1902` - -Hybrid Attributes, implements/supersedes synonym(), comparable_property() -------------------------------------------------------------------------- - -The "derived attributes" example has now been turned into an -official extension. The typical use case for ``synonym()`` -is to provide descriptor access to a mapped column; the use -case for ``comparable_property()`` is to be able to return a -``PropComparator`` from any descriptor. In practice, the -approach of "derived" is easier to use, more extensible, is -implemented in a few dozen lines of pure Python with almost -no imports, and doesn't require the ORM core to even be -aware of it. The feature is now known as the "Hybrid -Attributes" extension. - -``synonym()`` and ``comparable_property()`` are still part -of the ORM, though their implementations have been moved -outwards, building on an approach that is similar to that of -the hybrid extension, so that the core ORM -mapper/query/property modules aren't really aware of them -otherwise. - -.. seealso:: - - :ref:`hybrids_toplevel` - -:ticket:`1903` - -Speed Enhancements ------------------- - -As is customary with all major SQLA releases, a wide pass -through the internals to reduce overhead and callcounts has -been made which further reduces the work needed in common -scenarios. Highlights of this release include: - -* The flush process will now bundle INSERT statements into - batches fed to ``cursor.executemany()``, for rows where - the primary key is already present. In particular this - usually applies to the "child" table on a joined table - inheritance configuration, meaning the number of calls to - ``cursor.execute`` for a large bulk insert of joined- - table objects can be cut in half, allowing native DBAPI - optimizations to take place for those statements passed - to ``cursor.executemany()`` (such as re-using a prepared - statement). - -* The codepath invoked when accessing a many-to-one - reference to a related object that's already loaded has - been greatly simplified. The identity map is checked - directly without the need to generate a new ``Query`` - object first, which is expensive in the context of - thousands of in-memory many-to-ones being accessed. The - usage of constructed-per-call "loader" objects is also no - longer used for the majority of lazy attribute loads. - -* The rewrite of composites allows a shorter codepath when - mapper internals access mapped attributes within a - flush. - -* New inlined attribute access functions replace the - previous usage of "history" when the "save-update" and - other cascade operations need to cascade among the full - scope of datamembers associated with an attribute. This - reduces the overhead of generating a new ``History`` - object for this speed-critical operation. - -* The internals of the ``ExecutionContext``, the object - corresponding to a statement execution, have been - inlined and simplified. - -* The ``bind_processor()`` and ``result_processor()`` - callables generated by types for each statement - execution are now cached (carefully, so as to avoid memory - leaks for ad-hoc types and dialects) for the lifespan of - that type, further reducing per-statement call overhead. - -* The collection of "bind processors" for a particular - ``Compiled`` instance of a statement is also cached on - the ``Compiled`` object, taking further advantage of the - "compiled cache" used by the flush process to re-use the - same compiled form of INSERT, UPDATE, DELETE statements. - -A demonstration of callcount reduction including a sample -benchmark script is at -https://techspot.zzzeek.org/2010/12/12/a-tale-of-three- -profiles/ - -Composites Rewritten --------------------- - -The "composite" feature has been rewritten, like -``synonym()`` and ``comparable_property()``, to use a -lighter weight implementation based on descriptors and -events, rather than building into the ORM internals. This -allowed the removal of some latency from the mapper/unit of -work internals, and simplifies the workings of composite. -The composite attribute now no longer conceals the -underlying columns it builds upon, which now remain as -regular attributes. Composites can also act as a proxy for -``relationship()`` as well as ``Column()`` attributes. - -The major backwards-incompatible change of composites is -that they no longer use the ``mutable=True`` system to -detect in-place mutations. Please use the `Mutation -Tracking `_ extension to establish in-place change events -to existing composite usage. - -.. seealso:: - - :ref:`mapper_composite` - - :ref:`mutable_toplevel` - -:ticket:`2008` :ticket:`2024` - -More succinct form of query.join(target, onclause) --------------------------------------------------- - -The default method of issuing ``query.join()`` to a target -with an explicit onclause is now: - -:: - - query.join(SomeClass, SomeClass.id == ParentClass.some_id) - -In 0.6, this usage was considered to be an error, because -``join()`` accepts multiple arguments corresponding to -multiple JOIN clauses - the two-argument form needed to be -in a tuple to disambiguate between single-argument and two- -argument join targets. In the middle of 0.6 we added -detection and an error message for this specific calling -style, since it was so common. In 0.7, since we are -detecting the exact pattern anyway, and since having to type -out a tuple for no reason is extremely annoying, the non- -tuple method now becomes the "normal" way to do it. The -"multiple JOIN" use case is exceedingly rare compared to the -single join case, and multiple joins these days are more -clearly represented by multiple calls to ``join()``. - -The tuple form will remain for backwards compatibility. - -Note that all the other forms of ``query.join()`` remain -unchanged: - -:: - - query.join(MyClass.somerelation) - query.join("somerelation") - query.join(MyTarget) - # ... etc - -`Querying with Joins -`_ - -:ticket:`1923` - -.. _07_migration_mutation_extension: - -Mutation event extension, supersedes "mutable=True" ---------------------------------------------------- - -A new extension, :ref:`mutable_toplevel`, provides a -mechanism by which user-defined datatypes can provide change -events back to the owning parent or parents. The extension -includes an approach for scalar database values, such as -those managed by :class:`.PickleType`, ``postgresql.ARRAY``, or -other custom ``MutableType`` classes, as well as an approach -for ORM "composites", those configured using :func:`~.sqlalchemy.orm.composite`. - -.. seealso:: - - :ref:`mutable_toplevel` - -NULLS FIRST / NULLS LAST operators ----------------------------------- - -These are implemented as an extension to the ``asc()`` and -``desc()`` operators, called ``nullsfirst()`` and -``nullslast()``. - -.. seealso:: - - :func:`.nullsfirst` - - :func:`.nullslast` - -:ticket:`723` - -select.distinct(), query.distinct() accepts \*args for PostgreSQL DISTINCT ON ------------------------------------------------------------------------------ - -This was already available by passing a list of expressions -to the ``distinct`` keyword argument of ``select()``, the -``distinct()`` method of ``select()`` and ``Query`` now -accept positional arguments which are rendered as DISTINCT -ON when a PostgreSQL backend is used. - -`distinct() `_ - -`Query.distinct() `_ - -:ticket:`1069` - -``Index()`` can be placed inline inside of ``Table``, ``__table_args__`` ------------------------------------------------------------------------- - -The Index() construct can be created inline with a Table -definition, using strings as column names, as an alternative -to the creation of the index outside of the Table. That is: - -:: - - Table( - "mytable", - metadata, - Column("id", Integer, primary_key=True), - Column("name", String(50), nullable=False), - Index("idx_name", "name"), - ) - -The primary rationale here is for the benefit of declarative -``__table_args__``, particularly when used with mixins: - -:: - - class HasNameMixin(object): - name = Column("name", String(50), nullable=False) - - @declared_attr - def __table_args__(cls): - return (Index("name"), {}) - - - class User(HasNameMixin, Base): - __tablename__ = "user" - id = Column("id", Integer, primary_key=True) - -`Indexes `_ - -Window Function SQL Construct ------------------------------ - -A "window function" provides to a statement information -about the result set as it's produced. This allows criteria -against various things like "row number", "rank" and so -forth. They are known to be supported at least by -PostgreSQL, SQL Server and Oracle, possibly others. - -The best introduction to window functions is on PostgreSQL's -site, where window functions have been supported since -version 8.4: - -https://www.postgresql.org/docs/current/static/tutorial-window.html - -SQLAlchemy provides a simple construct typically invoked via -an existing function clause, using the ``over()`` method, -which accepts ``order_by`` and ``partition_by`` keyword -arguments. Below we replicate the first example in PG's -tutorial: - -:: - - from sqlalchemy.sql import table, column, select, func - - empsalary = table("empsalary", column("depname"), column("empno"), column("salary")) - - s = select( - [ - empsalary, - func.avg(empsalary.c.salary) - .over(partition_by=empsalary.c.depname) - .label("avg"), - ] - ) - - print(s) - -SQL: - -.. sourcecode:: sql - - SELECT empsalary.depname, empsalary.empno, empsalary.salary, - avg(empsalary.salary) OVER (PARTITION BY empsalary.depname) AS avg - FROM empsalary - -`sqlalchemy.sql.expression.over `_ - -:ticket:`1844` - -execution_options() on Connection accepts "isolation_level" argument --------------------------------------------------------------------- - -This sets the transaction isolation level for a single -``Connection``, until that ``Connection`` is closed and its -underlying DBAPI resource returned to the connection pool, -upon which the isolation level is reset back to the default. -The default isolation level is set using the -``isolation_level`` argument to ``create_engine()``. - -Transaction isolation support is currently only supported by -the PostgreSQL and SQLite backends. - -`execution_options() `_ - -:ticket:`2001` - -``TypeDecorator`` works with integer primary key columns --------------------------------------------------------- - -A ``TypeDecorator`` which extends the behavior of -``Integer`` can be used with a primary key column. The -"autoincrement" feature of ``Column`` will now recognize -that the underlying database column is still an integer so -that lastrowid mechanisms continue to function. The -``TypeDecorator`` itself will have its result value -processor applied to newly generated primary keys, including -those received by the DBAPI ``cursor.lastrowid`` accessor. - -:ticket:`2005` :ticket:`2006` - -``TypeDecorator`` is present in the "sqlalchemy" import space -------------------------------------------------------------- - -No longer need to import this from ``sqlalchemy.types``, -it's now mirrored in ``sqlalchemy``. - -New Dialects ------------- - -Dialects have been added: - -* a MySQLdb driver for the Drizzle database: - - - `Drizzle `_ - -* support for the pymysql DBAPI: - - - `pymsql Notes - `_ - -* psycopg2 now works with Python 3 - - -Behavioral Changes (Backwards Compatible) -========================================= - -C Extensions Build by Default ------------------------------ - -This is as of 0.7b4. The exts will build if cPython 2.xx -is detected. If the build fails, such as on a windows -install, that condition is caught and the non-C install -proceeds. The C exts won't build if Python 3 or PyPy is -used. - -Query.count() simplified, should work virtually always ------------------------------------------------------- - -The very old guesswork which occurred within -``Query.count()`` has been modernized to use -``.from_self()``. That is, ``query.count()`` is now -equivalent to: - -:: - - query.from_self(func.count(literal_column("1"))).scalar() - -Previously, internal logic attempted to rewrite the columns -clause of the query itself, and upon detection of a -"subquery" condition, such as a column-based query that -might have aggregates in it, or a query with DISTINCT, would -go through a convoluted process of rewriting the columns -clause. This logic failed in complex conditions, -particularly those involving joined table inheritance, and -was long obsolete by the more comprehensive ``.from_self()`` -call. - -The SQL emitted by ``query.count()`` is now always of the -form: - -.. sourcecode:: sql - - SELECT count(1) AS count_1 FROM ( - SELECT user.id AS user_id, user.name AS user_name from user - ) AS anon_1 - -that is, the original query is preserved entirely inside of -a subquery, with no more guessing as to how count should be -applied. - -:ticket:`2093` - -To emit a non-subquery form of count() -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -MySQL users have already reported that the MyISAM engine not -surprisingly falls over completely with this simple change. -Note that for a simple ``count()`` that optimizes for DBs -that can't handle simple subqueries, ``func.count()`` should -be used: - -:: - - from sqlalchemy import func - - session.query(func.count(MyClass.id)).scalar() - -or for ``count(*)``: - -:: - - from sqlalchemy import func, literal_column - - session.query(func.count(literal_column("*"))).select_from(MyClass).scalar() - -LIMIT/OFFSET clauses now use bind parameters --------------------------------------------- - -The LIMIT and OFFSET clauses, or their backend equivalents -(i.e. TOP, ROW NUMBER OVER, etc.), use bind parameters for -the actual values, for all backends which support it (most -except for Sybase). This allows better query optimizer -performance as the textual string for multiple statements -with differing LIMIT/OFFSET are now identical. - -:ticket:`805` - -Logging enhancements --------------------- - -Vinay Sajip has provided a patch to our logging system such -that the "hex string" embedded in logging statements for -engines and pools is no longer needed to allow the ``echo`` -flag to work correctly. A new system that uses filtered -logging objects allows us to maintain our current behavior -of ``echo`` being local to individual engines without the -need for additional identifying strings local to those -engines. - -:ticket:`1926` - -Simplified polymorphic_on assignment ------------------------------------- - -The population of the ``polymorphic_on`` column-mapped -attribute, when used in an inheritance scenario, now occurs -when the object is constructed, i.e. its ``__init__`` method -is called, using the init event. The attribute then behaves -the same as any other column-mapped attribute. Previously, -special logic would fire off during flush to populate this -column, which prevented any user code from modifying its -behavior. The new approach improves upon this in three -ways: 1. the polymorphic identity is now present on the -object as soon as its constructed; 2. the polymorphic -identity can be changed by user code without any difference -in behavior from any other column-mapped attribute; 3. the -internals of the mapper during flush are simplified and no -longer need to make special checks for this column. - -:ticket:`1895` - -contains_eager() chains across multiple paths (i.e. "all()") ------------------------------------------------------------- - -The ```contains_eager()```` modifier now will chain itself -for a longer path without the need to emit individual -````contains_eager()``` calls. Instead of: - -:: - - session.query(A).options(contains_eager(A.b), contains_eager(A.b, B.c)) - -you can say: - -:: - - session.query(A).options(contains_eager(A.b, B.c)) - -:ticket:`2032` - -Flushing of orphans that have no parent is allowed --------------------------------------------------- - -We've had a long standing behavior that checks for a so- -called "orphan" during flush, that is, an object which is -associated with a ``relationship()`` that specifies "delete- -orphan" cascade, has been newly added to the session for an -INSERT, and no parent relationship has been established. -This check was added years ago to accommodate some test -cases which tested the orphan behavior for consistency. In -modern SQLA, this check is no longer needed on the Python -side. The equivalent behavior of the "orphan check" is -accomplished by making the foreign key reference to the -object's parent row NOT NULL, where the database does its -job of establishing data consistency in the same way SQLA -allows most other operations to do. If the object's parent -foreign key is nullable, then the row can be inserted. The -"orphan" behavior runs when the object was persisted with a -particular parent, and is then disassociated with that -parent, leading to a DELETE statement emitted for it. - -:ticket:`1912` - -Warnings generated when collection members, scalar referents not part of the flush ----------------------------------------------------------------------------------- - -Warnings are now emitted when related objects referenced via -a loaded ``relationship()`` on a parent object marked as -"dirty" are not present in the current ``Session``. - -The ``save-update`` cascade takes effect when objects are -added to the ``Session``, or when objects are first -associated with a parent, so that an object and everything -related to it are usually all present in the same -``Session``. However, if ``save-update`` cascade is -disabled for a particular ``relationship()``, then this -behavior does not occur, and the flush process does not try -to correct for it, instead staying consistent to the -configured cascade behavior. Previously, when such objects -were detected during the flush, they were silently skipped. -The new behavior is that a warning is emitted, for the -purposes of alerting to a situation that more often than not -is the source of unexpected behavior. - -:ticket:`1973` - -Setup no longer installs a Nose plugin --------------------------------------- - -Since we moved to nose we've used a plugin that installs via -setuptools, so that the ``nosetests`` script would -automatically run SQLA's plugin code, necessary for our -tests to have a full environment. In the middle of 0.6, we -realized that the import pattern here meant that Nose's -"coverage" plugin would break, since "coverage" requires -that it be started before any modules to be covered are -imported; so in the middle of 0.6 we made the situation -worse by adding a separate ``sqlalchemy-nose`` package to -the build to overcome this. - -In 0.7 we've done away with trying to get ``nosetests`` to -work automatically, since the SQLAlchemy module would -produce a large number of nose configuration options for all -usages of ``nosetests``, not just the SQLAlchemy unit tests -themselves, and the additional ``sqlalchemy-nose`` install -was an even worse idea, producing an extra package in Python -environments. The ``sqla_nose.py`` script in 0.7 is now -the only way to run the tests with nose. - -:ticket:`1949` - -Non-``Table``-derived constructs can be mapped ----------------------------------------------- - -A construct that isn't against any ``Table`` at all, like a -function, can be mapped. - -:: - - from sqlalchemy import select, func - from sqlalchemy.orm import mapper - - - class Subset(object): - pass - - - selectable = select(["x", "y", "z"]).select_from(func.some_db_function()).alias() - mapper(Subset, selectable, primary_key=[selectable.c.x]) - -:ticket:`1876` - -aliased() accepts ``FromClause`` elements ------------------------------------------ - -This is a convenience helper such that in the case a plain -``FromClause``, such as a ``select``, ``Table`` or ``join`` -is passed to the ``orm.aliased()`` construct, it passes -through to the ``.alias()`` method of that from construct -rather than constructing an ORM level ``AliasedClass``. - -:ticket:`2018` - -Session.connection(), Session.execute() accept 'bind' ------------------------------------------------------ - -This is to allow execute/connection operations to -participate in the open transaction of an engine explicitly. -It also allows custom subclasses of ``Session`` that -implement their own ``get_bind()`` method and arguments to -use those custom arguments with both the ``execute()`` and -``connection()`` methods equally. - -`Session.connection `_ -`Session.execute `_ - -:ticket:`1996` - -Standalone bind parameters in columns clause auto-labeled. ----------------------------------------------------------- - -Bind parameters present in the "columns clause" of a select -are now auto-labeled like other "anonymous" clauses, which -among other things allows their "type" to be meaningful when -the row is fetched, as in result row processors. - -SQLite - relative file paths are normalized through os.path.abspath() ---------------------------------------------------------------------- - -This so that a script that changes the current directory -will continue to target the same location as subsequent -SQLite connections are established. - -:ticket:`2036` - -MS-SQL - ``String``/``Unicode``/``VARCHAR``/``NVARCHAR``/``VARBINARY`` emit "max" for no length ------------------------------------------------------------------------------------------------ - -On the MS-SQL backend, the String/Unicode types, and their -counterparts VARCHAR/ NVARCHAR, as well as VARBINARY -(:ticket:`1833`) emit "max" as the length when no length is -specified. This makes it more compatible with PostgreSQL's -VARCHAR type which is similarly unbounded when no length -specified. SQL Server defaults the length on these types -to '1' when no length is specified. - -Behavioral Changes (Backwards Incompatible) -=========================================== - -Note again, aside from the default mutability change, most -of these changes are \*extremely minor* and will not affect -most users. - -``PickleType`` and ARRAY mutability turned off by default ---------------------------------------------------------- - -This change refers to the default behavior of the ORM when -mapping columns that have either the ``PickleType`` or -``postgresql.ARRAY`` datatypes. The ``mutable`` flag is now -set to ``False`` by default. If an existing application uses -these types and depends upon detection of in-place -mutations, the type object must be constructed with -``mutable=True`` to restore the 0.6 behavior: - -:: - - Table( - "mytable", - metadata, - # .... - Column("pickled_data", PickleType(mutable=True)), - ) - -The ``mutable=True`` flag is being phased out, in favor of -the new `Mutation Tracking `_ extension. This extension -provides a mechanism by which user-defined datatypes can -provide change events back to the owning parent or parents. - -The previous approach of using ``mutable=True`` does not -provide for change events - instead, the ORM must scan -through all mutable values present in a session and compare -them against their original value for changes every time -``flush()`` is called, which is a very time consuming event. -This is a holdover from the very early days of SQLAlchemy -when ``flush()`` was not automatic and the history tracking -system was not nearly as sophisticated as it is now. - -Existing applications which use ``PickleType``, -``postgresql.ARRAY`` or other ``MutableType`` subclasses, -and require in-place mutation detection, should migrate to -the new mutation tracking system, as ``mutable=True`` is -likely to be deprecated in the future. - -:ticket:`1980` - -Mutability detection of ``composite()`` requires the Mutation Tracking Extension --------------------------------------------------------------------------------- - -So-called "composite" mapped attributes, those configured -using the technique described at `Composite Column Types -`_, have been re-implemented such -that the ORM internals are no longer aware of them (leading -to shorter and more efficient codepaths in critical -sections). While composite types are generally intended to -be treated as immutable value objects, this was never -enforced. For applications that use composites with -mutability, the `Mutation Tracking `_ extension offers a -base class which establishes a mechanism for user-defined -composite types to send change event messages back to the -owning parent or parents of each object. - -Applications which use composite types and rely upon in- -place mutation detection of these objects should either -migrate to the "mutation tracking" extension, or change the -usage of the composite types such that in-place changes are -no longer needed (i.e., treat them as immutable value -objects). - -SQLite - the SQLite dialect now uses ``NullPool`` for file-based databases --------------------------------------------------------------------------- - -This change is **99.999% backwards compatible**, unless you -are using temporary tables across connection pool -connections. - -A file-based SQLite connection is blazingly fast, and using -``NullPool`` means that each call to ``Engine.connect`` -creates a new pysqlite connection. - -Previously, the ``SingletonThreadPool`` was used, which -meant that all connections to a certain engine in a thread -would be the same connection. It's intended that the new -approach is more intuitive, particularly when multiple -connections are used. - -``SingletonThreadPool`` is still the default engine when a -``:memory:`` database is used. - -Note that this change **breaks temporary tables used across -Session commits**, due to the way SQLite handles temp -tables. See the note at -https://www.sqlalchemy.org/docs/dialects/sqlite.html#using- -temporary-tables-with-sqlite if temporary tables beyond the -scope of one pool connection are desired. - -:ticket:`1921` - -``Session.merge()`` checks version ids for versioned mappers ------------------------------------------------------------- - -Session.merge() will check the version id of the incoming -state against that of the database, assuming the mapping -uses version ids and incoming state has a version_id -assigned, and raise StaleDataError if they don't match. -This is the correct behavior, in that if incoming state -contains a stale version id, it should be assumed the state -is stale. - -If merging data into a versioned state, the version id -attribute can be left undefined, and no version check will -take place. - -This check was confirmed by examining what Hibernate does - -both the ``merge()`` and the versioning features were -originally adapted from Hibernate. - -:ticket:`2027` - -Tuple label names in Query Improved ------------------------------------ - -This improvement is potentially slightly backwards -incompatible for an application that relied upon the old -behavior. - -Given two mapped classes ``Foo`` and ``Bar`` each with a -column ``spam``: - -:: - - - qa = session.query(Foo.spam) - qb = session.query(Bar.spam) - - qu = qa.union(qb) - -The name given to the single column yielded by ``qu`` will -be ``spam``. Previously it would be something like -``foo_spam`` due to the way the ``union`` would combine -things, which is inconsistent with the name ``spam`` in the -case of a non-unioned query. - -:ticket:`1942` - -Mapped column attributes reference the most specific column first ------------------------------------------------------------------ - -This is a change to the behavior involved when a mapped -column attribute references multiple columns, specifically -when dealing with an attribute on a joined-table subclass -that has the same name as that of an attribute on the -superclass. - -Using declarative, the scenario is this: - -:: - - class Parent(Base): - __tablename__ = "parent" - id = Column(Integer, primary_key=True) - - - class Child(Parent): - __tablename__ = "child" - id = Column(Integer, ForeignKey("parent.id"), primary_key=True) - -Above, the attribute ``Child.id`` refers to both the -``child.id`` column as well as ``parent.id`` - this due to -the name of the attribute. If it were named differently on -the class, such as ``Child.child_id``, it then maps -distinctly to ``child.id``, with ``Child.id`` being the same -attribute as ``Parent.id``. - -When the ``id`` attribute is made to reference both -``parent.id`` and ``child.id``, it stores them in an ordered -list. An expression such as ``Child.id`` then refers to -just *one* of those columns when rendered. Up until 0.6, -this column would be ``parent.id``. In 0.7, it is the less -surprising ``child.id``. - -The legacy of this behavior deals with behaviors and -restrictions of the ORM that don't really apply anymore; all -that was needed was to reverse the order. - -A primary advantage of this approach is that it's now easier -to construct ``primaryjoin`` expressions that refer to the -local column: - -:: - - class Child(Parent): - __tablename__ = "child" - id = Column(Integer, ForeignKey("parent.id"), primary_key=True) - some_related = relationship( - "SomeRelated", primaryjoin="Child.id==SomeRelated.child_id" - ) - - - class SomeRelated(Base): - __tablename__ = "some_related" - id = Column(Integer, primary_key=True) - child_id = Column(Integer, ForeignKey("child.id")) - -Prior to 0.7 the ``Child.id`` expression would reference -``Parent.id``, and it would be necessary to map ``child.id`` -to a distinct attribute. - -It also means that a query like this one changes its -behavior: - -:: - - session.query(Parent).filter(Child.id > 7) - -In 0.6, this would render: - -.. sourcecode:: sql - - SELECT parent.id AS parent_id - FROM parent - WHERE parent.id > :id_1 - -in 0.7, you get: - -.. sourcecode:: sql - - SELECT parent.id AS parent_id - FROM parent, child - WHERE child.id > :id_1 - -which you'll note is a cartesian product - this behavior is -now equivalent to that of any other attribute that is local -to ``Child``. The ``with_polymorphic()`` method, or a -similar strategy of explicitly joining the underlying -``Table`` objects, is used to render a query against all -``Parent`` objects with criteria against ``Child``, in the -same manner as that of 0.5 and 0.6: - -:: - - print(s.query(Parent).with_polymorphic([Child]).filter(Child.id > 7)) - -Which on both 0.6 and 0.7 renders: - -.. sourcecode:: sql - - SELECT parent.id AS parent_id, child.id AS child_id - FROM parent LEFT OUTER JOIN child ON parent.id = child.id - WHERE child.id > :id_1 - -Another effect of this change is that a joined-inheritance -load across two tables will populate from the child table's -value, not that of the parent table. An unusual case is that -a query against "Parent" using ``with_polymorphic="*"`` -issues a query against "parent", with a LEFT OUTER JOIN to -"child". The row is located in "Parent", sees the -polymorphic identity corresponds to "Child", but suppose the -actual row in "child" has been *deleted*. Due to this -corruption, the row comes in with all the columns -corresponding to "child" set to NULL - this is now the value -that gets populated, not the one in the parent table. - -:ticket:`1892` - -Mapping to joins with two or more same-named columns requires explicit declaration ----------------------------------------------------------------------------------- - -This is somewhat related to the previous change in -:ticket:`1892`. When mapping to a join, same-named columns -must be explicitly linked to mapped attributes, i.e. as -described in `Mapping a Class Against Multiple Tables `_. - -Given two tables ``foo`` and ``bar``, each with a primary -key column ``id``, the following now produces an error: - -:: - - - foobar = foo.join(bar, foo.c.id == bar.c.foo_id) - mapper(FooBar, foobar) - -This because the ``mapper()`` refuses to guess what column -is the primary representation of ``FooBar.id`` - is it -``foo.c.id`` or is it ``bar.c.id`` ? The attribute must be -explicit: - -:: - - - foobar = foo.join(bar, foo.c.id == bar.c.foo_id) - mapper(FooBar, foobar, properties={"id": [foo.c.id, bar.c.id]}) - -:ticket:`1896` - -Mapper requires that polymorphic_on column be present in the mapped selectable ------------------------------------------------------------------------------- - -This is a warning in 0.6, now an error in 0.7. The column -given for ``polymorphic_on`` must be in the mapped -selectable. This to prevent some occasional user errors -such as: - -:: - - mapper(SomeClass, sometable, polymorphic_on=some_lookup_table.c.id) - -where above the polymorphic_on needs to be on a -``sometable`` column, in this case perhaps -``sometable.c.some_lookup_id``. There are also some -"polymorphic union" scenarios where similar mistakes -sometimes occur. - -Such a configuration error has always been "wrong", and the -above mapping doesn't work as specified - the column would -be ignored. It is however potentially backwards -incompatible in the rare case that an application has been -unknowingly relying upon this behavior. - -:ticket:`1875` - -``DDL()`` constructs now escape percent signs ---------------------------------------------- - -Previously, percent signs in ``DDL()`` strings would have to -be escaped, i.e. ``%%`` depending on DBAPI, for those DBAPIs -that accept ``pyformat`` or ``format`` binds (i.e. psycopg2, -mysql-python), which was inconsistent versus ``text()`` -constructs which did this automatically. The same escaping -now occurs for ``DDL()`` as for ``text()``. - -:ticket:`1897` - -``Table.c`` / ``MetaData.tables`` refined a bit, don't allow direct mutation ----------------------------------------------------------------------------- - -Another area where some users were tinkering around in such -a way that doesn't actually work as expected, but still left -an exceedingly small chance that some application was -relying upon this behavior, the construct returned by the -``.c`` attribute on ``Table`` and the ``.tables`` attribute -on ``MetaData`` is explicitly non-mutable. The "mutable" -version of the construct is now private. Adding columns to -``.c`` involves using the ``append_column()`` method of -``Table``, which ensures things are associated with the -parent ``Table`` in the appropriate way; similarly, -``MetaData.tables`` has a contract with the ``Table`` -objects stored in this dictionary, as well as a little bit -of new bookkeeping in that a ``set()`` of all schema names -is tracked, which is satisfied only by using the public -``Table`` constructor as well as ``Table.tometadata()``. - -It is of course possible that the ``ColumnCollection`` and -``dict`` collections consulted by these attributes could -someday implement events on all of their mutational methods -such that the appropriate bookkeeping occurred upon direct -mutation of the collections, but until someone has the -motivation to implement all that along with dozens of new -unit tests, narrowing the paths to mutation of these -collections will ensure no application is attempting to rely -upon usages that are currently not supported. - -:ticket:`1893` :ticket:`1917` - -server_default consistently returns None for all inserted_primary_key values ----------------------------------------------------------------------------- - -Established consistency when server_default is present on an -Integer PK column. SQLA doesn't pre-fetch these, nor do they -come back in cursor.lastrowid (DBAPI). Ensured all backends -consistently return None in result.inserted_primary_key for -these - some backends may have returned a value previously. -Using a server_default on a primary key column is extremely -unusual. If a special function or SQL expression is used -to generate primary key defaults, this should be established -as a Python-side "default" instead of server_default. - -Regarding reflection for this case, reflection of an int PK -col with a server_default sets the "autoincrement" flag to -False, except in the case of a PG SERIAL col where we -detected a sequence default. - -:ticket:`2020` :ticket:`2021` - -The ``sqlalchemy.exceptions`` alias in sys.modules is removed -------------------------------------------------------------- - -For a few years we've added the string -``sqlalchemy.exceptions`` to ``sys.modules``, so that a -statement like "``import sqlalchemy.exceptions``" would -work. The name of the core exceptions module has been -``exc`` for a long time now, so the recommended import for -this module is: - -:: - - from sqlalchemy import exc - -The ``exceptions`` name is still present in "``sqlalchemy``" -for applications which might have said ``from sqlalchemy -import exceptions``, but they should also start using the -``exc`` name. - -Query Timing Recipe Changes ---------------------------- - -While not part of SQLAlchemy itself, it's worth mentioning -that the rework of the ``ConnectionProxy`` into the new -event system means it is no longer appropriate for the -"Timing all Queries" recipe. Please adjust query-timers to -use the ``before_cursor_execute()`` and -``after_cursor_execute()`` events, demonstrated in the -updated recipe UsageRecipes/Profiling. - -Deprecated API -============== - -Default constructor on types will not accept arguments ------------------------------------------------------- - -Simple types like ``Integer``, ``Date`` etc. in the core -types module don't accept arguments. The default -constructor that accepts/ignores a catchall ``\*args, -\**kwargs`` is restored as of 0.7b4/0.7.0, but emits a -deprecation warning. - -If arguments are being used with a core type like -``Integer``, it may be that you intended to use a dialect -specific type, such as ``sqlalchemy.dialects.mysql.INTEGER`` -which does accept a "display_width" argument for example. - -compile_mappers() renamed configure_mappers(), simplified configuration internals ---------------------------------------------------------------------------------- - -This system slowly morphed from something small, implemented -local to an individual mapper, and poorly named into -something that's more of a global "registry-" level function -and poorly named, so we've fixed both by moving the -implementation out of ``Mapper`` altogether and renaming it -to ``configure_mappers()``. It is of course normally not -needed for an application to call ``configure_mappers()`` as -this process occurs on an as-needed basis, as soon as the -mappings are needed via attribute or query access. - -:ticket:`1966` - -Core listener/proxy superseded by event listeners -------------------------------------------------- - -``PoolListener``, ``ConnectionProxy``, -``DDLElement.execute_at`` are superseded by -``event.listen()``, using the ``PoolEvents``, -``EngineEvents``, ``DDLEvents`` dispatch targets, -respectively. - -ORM extensions superseded by event listeners --------------------------------------------- - -``MapperExtension``, ``AttributeExtension``, -``SessionExtension`` are superseded by ``event.listen()``, -using the ``MapperEvents``/``InstanceEvents``, -``AttributeEvents``, ``SessionEvents``, dispatch targets, -respectively. - -Sending a string to 'distinct' in select() for MySQL should be done via prefixes --------------------------------------------------------------------------------- - -This obscure feature allows this pattern with the MySQL -backend: - -:: - - select([mytable], distinct="ALL", prefixes=["HIGH_PRIORITY"]) - -The ``prefixes`` keyword or ``prefix_with()`` method should -be used for non-standard or unusual prefixes: - -:: - - select([mytable]).prefix_with("HIGH_PRIORITY", "ALL") - -``useexisting`` superseded by ``extend_existing`` and ``keep_existing`` ------------------------------------------------------------------------ - -The ``useexisting`` flag on Table has been superseded by a -new pair of flags ``keep_existing`` and ``extend_existing``. -``extend_existing`` is equivalent to ``useexisting`` - the -existing Table is returned, and additional constructor -elements are added. With ``keep_existing``, the existing -Table is returned, but additional constructor elements are -not added - these elements are only applied when the Table -is newly created. - -Backwards Incompatible API Changes -================================== - -Callables passed to ``bindparam()`` don't get evaluated - affects the Beaker example ------------------------------------------------------------------------------------- - -:ticket:`1950` - -Note this affects the Beaker caching example, where the -workings of the ``_params_from_query()`` function needed a -slight adjustment. If you're using code from the Beaker -example, this change should be applied. - -types.type_map is now private, types._type_map ----------------------------------------------- - -We noticed some users tapping into this dictionary inside of -``sqlalchemy.types`` as a shortcut to associating Python -types with SQL types. We can't guarantee the contents or -format of this dictionary, and additionally the business of -associating Python types in a one-to-one fashion has some -grey areas that should are best decided by individual -applications, so we've underscored this attribute. - -:ticket:`1870` - -Renamed the ``alias`` keyword arg of standalone ``alias()`` function to ``name`` --------------------------------------------------------------------------------- - -This so that the keyword argument ``name`` matches that of -the ``alias()`` methods on all ``FromClause`` objects as -well as the ``name`` argument on ``Query.subquery()``. - -Only code that uses the standalone ``alias()`` function, and -not the method bound functions, and passes the alias name -using the explicit keyword name ``alias``, and not -positionally, would need modification here. - -Non-public ``Pool`` methods underscored ---------------------------------------- - -All methods of ``Pool`` and subclasses which are not -intended for public use have been renamed with underscores. -That they were not named this way previously was a bug. - -Pooling methods now underscored or removed: - -``Pool.create_connection()`` -> -``Pool._create_connection()`` - -``Pool.do_get()`` -> ``Pool._do_get()`` - -``Pool.do_return_conn()`` -> ``Pool._do_return_conn()`` - -``Pool.do_return_invalid()`` -> removed, was not used - -``Pool.return_conn()`` -> ``Pool._return_conn()`` - -``Pool.get()`` -> ``Pool._get()``, public API is -``Pool.connect()`` - -``SingletonThreadPool.cleanup()`` -> ``_cleanup()`` - -``SingletonThreadPool.dispose_local()`` -> removed, use -``conn.invalidate()`` - -:ticket:`1982` - -Previously Deprecated, Now Removed -================================== - -Query.join(), Query.outerjoin(), eagerload(), eagerload_all(), others no longer allow lists of attributes as arguments ----------------------------------------------------------------------------------------------------------------------- - -Passing a list of attributes or attribute names to -``Query.join``, ``eagerload()``, and similar has been -deprecated since 0.5: - -:: - - # old way, deprecated since 0.5 - session.query(Houses).join([Houses.rooms, Room.closets]) - session.query(Houses).options(eagerload_all([Houses.rooms, Room.closets])) - -These methods all accept \*args as of the 0.5 series: - -:: - - # current way, in place since 0.5 - session.query(Houses).join(Houses.rooms, Room.closets) - session.query(Houses).options(eagerload_all(Houses.rooms, Room.closets)) - -``ScopedSession.mapper`` is removed ------------------------------------ - -This feature provided a mapper extension which linked class- -based functionality with a particular ``ScopedSession``, in -particular providing the behavior such that new object -instances would be automatically associated with that -session. The feature was overused by tutorials and -frameworks which led to great user confusion due to its -implicit behavior, and was deprecated in 0.5.5. Techniques -for replicating its functionality are at -[wiki:UsageRecipes/SessionAwareMapper] - diff --git a/doc/build/changelog/migration_08.rst b/doc/build/changelog/migration_08.rst deleted file mode 100644 index 0f661cca790..00000000000 --- a/doc/build/changelog/migration_08.rst +++ /dev/null @@ -1,1539 +0,0 @@ -============================= -What's New in SQLAlchemy 0.8? -============================= - -.. admonition:: About this Document - - This document describes changes between SQLAlchemy version 0.7, - undergoing maintenance releases as of October, 2012, - and SQLAlchemy version 0.8, which is expected for release - in early 2013. - - Document date: October 25, 2012 - Updated: March 9, 2013 - -Introduction -============ - -This guide introduces what's new in SQLAlchemy version 0.8, -and also documents changes which affect users migrating -their applications from the 0.7 series of SQLAlchemy to 0.8. - -SQLAlchemy releases are closing in on 1.0, and each new -version since 0.5 features fewer major usage changes. Most -applications that are settled into modern 0.7 patterns -should be movable to 0.8 with no changes. Applications that -use 0.6 and even 0.5 patterns should be directly migratable -to 0.8 as well, though larger applications may want to test -with each interim version. - -Platform Support -================ - -Targeting Python 2.5 and Up Now -------------------------------- - -SQLAlchemy 0.8 will target Python 2.5 and forward; -compatibility for Python 2.4 is being dropped. - -The internals will be able to make usage of Python ternaries -(that is, ``x if y else z``) which will improve things -versus the usage of ``y and x or z``, which naturally has -been the source of some bugs, as well as context managers -(that is, ``with:``) and perhaps in some cases -``try:/except:/else:`` blocks which will help with code -readability. - -SQLAlchemy will eventually drop 2.5 support as well - when -2.6 is reached as the baseline, SQLAlchemy will move to use -2.6/3.3 in-place compatibility, removing the usage of the -``2to3`` tool and maintaining a source base that works with -Python 2 and 3 at the same time. - -New ORM Features -================ - -.. _feature_relationship_08: - -Rewritten :func:`_orm.relationship` mechanics ----------------------------------------------- -0.8 features a much improved and capable system regarding -how :func:`_orm.relationship` determines how to join between two -entities. The new system includes these features: - -* The ``primaryjoin`` argument is **no longer needed** when - constructing a :func:`_orm.relationship` against a class that - has multiple foreign key paths to the target. Only the - ``foreign_keys`` argument is needed to specify those - columns which should be included: - - :: - - - class Parent(Base): - __tablename__ = "parent" - id = Column(Integer, primary_key=True) - child_id_one = Column(Integer, ForeignKey("child.id")) - child_id_two = Column(Integer, ForeignKey("child.id")) - - child_one = relationship("Child", foreign_keys=child_id_one) - child_two = relationship("Child", foreign_keys=child_id_two) - - - class Child(Base): - __tablename__ = "child" - id = Column(Integer, primary_key=True) - -* relationships against self-referential, composite foreign - keys where **a column points to itself** are now - supported. The canonical case is as follows: - - :: - - class Folder(Base): - __tablename__ = "folder" - __table_args__ = ( - ForeignKeyConstraint( - ["account_id", "parent_id"], ["folder.account_id", "folder.folder_id"] - ), - ) - - account_id = Column(Integer, primary_key=True) - folder_id = Column(Integer, primary_key=True) - parent_id = Column(Integer) - name = Column(String) - - parent_folder = relationship( - "Folder", backref="child_folders", remote_side=[account_id, folder_id] - ) - - Above, the ``Folder`` refers to its parent ``Folder`` - joining from ``account_id`` to itself, and ``parent_id`` - to ``folder_id``. When SQLAlchemy constructs an auto- - join, no longer can it assume all columns on the "remote" - side are aliased, and all columns on the "local" side are - not - the ``account_id`` column is **on both sides**. So - the internal relationship mechanics were totally rewritten - to support an entirely different system whereby two copies - of ``account_id`` are generated, each containing different - *annotations* to determine their role within the - statement. Note the join condition within a basic eager - load: - - .. sourcecode:: sql - - SELECT - folder.account_id AS folder_account_id, - folder.folder_id AS folder_folder_id, - folder.parent_id AS folder_parent_id, - folder.name AS folder_name, - folder_1.account_id AS folder_1_account_id, - folder_1.folder_id AS folder_1_folder_id, - folder_1.parent_id AS folder_1_parent_id, - folder_1.name AS folder_1_name - FROM folder - LEFT OUTER JOIN folder AS folder_1 - ON - folder_1.account_id = folder.account_id - AND folder.folder_id = folder_1.parent_id - - WHERE folder.folder_id = ? AND folder.account_id = ? - -* Previously difficult custom join conditions, like those involving - functions and/or CASTing of types, will now function as - expected in most cases:: - - class HostEntry(Base): - __tablename__ = "host_entry" - - id = Column(Integer, primary_key=True) - ip_address = Column(INET) - content = Column(String(50)) - - # relationship() using explicit foreign_keys, remote_side - parent_host = relationship( - "HostEntry", - primaryjoin=ip_address == cast(content, INET), - foreign_keys=content, - remote_side=ip_address, - ) - - The new :func:`_orm.relationship` mechanics make use of a - SQLAlchemy concept known as :term:`annotations`. These annotations - are also available to application code explicitly via - the :func:`.foreign` and :func:`.remote` functions, either - as a means to improve readability for advanced configurations - or to directly inject an exact configuration, bypassing - the usual join-inspection heuristics:: - - from sqlalchemy.orm import foreign, remote - - - class HostEntry(Base): - __tablename__ = "host_entry" - - id = Column(Integer, primary_key=True) - ip_address = Column(INET) - content = Column(String(50)) - - # relationship() using explicit foreign() and remote() annotations - # in lieu of separate arguments - parent_host = relationship( - "HostEntry", - primaryjoin=remote(ip_address) == cast(foreign(content), INET), - ) - -.. seealso:: - - :ref:`relationship_configure_joins` - a newly revised section on :func:`_orm.relationship` - detailing the latest techniques for customizing related attributes and collection - access. - -:ticket:`1401` :ticket:`610` - -.. _feature_orminspection_08: - -New Class/Object Inspection System ----------------------------------- - -Lots of SQLAlchemy users are writing systems that require -the ability to inspect the attributes of a mapped class, -including being able to get at the primary key columns, -object relationships, plain attributes, and so forth, -typically for the purpose of building data-marshalling -systems, like JSON/XML conversion schemes and of course form -libraries galore. - -Originally, the :class:`_schema.Table` and :class:`_schema.Column` model were the -original inspection points, which have a well-documented -system. While SQLAlchemy ORM models are also fully -introspectable, this has never been a fully stable and -supported feature, and users tended to not have a clear idea -how to get at this information. - -0.8 now provides a consistent, stable and fully -documented API for this purpose, including an inspection -system which works on mapped classes, instances, attributes, -and other Core and ORM constructs. The entrypoint to this -system is the core-level :func:`_sa.inspect` function. -In most cases, the object being inspected -is one already part of SQLAlchemy's system, -such as :class:`_orm.Mapper`, :class:`.InstanceState`, -:class:`_reflection.Inspector`. In some cases, new objects have been -added with the job of providing the inspection API in -certain contexts, such as :class:`.AliasedInsp` and -:class:`.AttributeState`. - -A walkthrough of some key capabilities follows: - -.. sourcecode:: pycon+sql - - >>> class User(Base): - ... __tablename__ = "user" - ... id = Column(Integer, primary_key=True) - ... name = Column(String) - ... name_syn = synonym(name) - ... addresses = relationship("Address") - - >>> # universal entry point is inspect() - >>> b = inspect(User) - - >>> # b in this case is the Mapper - >>> b - - - >>> # Column namespace - >>> b.columns.id - Column('id', Integer(), table=, primary_key=True, nullable=False) - - >>> # mapper's perspective of the primary key - >>> b.primary_key - (Column('id', Integer(), table=, primary_key=True, nullable=False),) - - >>> # MapperProperties available from .attrs - >>> b.attrs.keys() - ['name_syn', 'addresses', 'id', 'name'] - - >>> # .column_attrs, .relationships, etc. filter this collection - >>> b.column_attrs.keys() - ['id', 'name'] - - >>> list(b.relationships) - [] - - >>> # they are also namespaces - >>> b.column_attrs.id - - - >>> b.relationships.addresses - - - >>> # point inspect() at a mapped, class level attribute, - >>> # returns the attribute itself - >>> b = inspect(User.addresses) - >>> b - - - >>> # From here we can get the mapper: - >>> b.mapper - - - >>> # the parent inspector, in this case a mapper - >>> b.parent - - - >>> # an expression - >>> print(b.expression) - {printsql}"user".id = address.user_id{stop} - - >>> # inspect works on instances - >>> u1 = User(id=3, name="x") - >>> b = inspect(u1) - - >>> # it returns the InstanceState - >>> b - - - >>> # similar attrs accessor refers to the - >>> b.attrs.keys() - ['id', 'name_syn', 'addresses', 'name'] - - >>> # attribute interface - from attrs, you get a state object - >>> b.attrs.id - - - >>> # this object can give you, current value... - >>> b.attrs.id.value - 3 - - >>> # ... current history - >>> b.attrs.id.history - History(added=[3], unchanged=(), deleted=()) - - >>> # InstanceState can also provide session state information - >>> # lets assume the object is persistent - >>> s = Session() - >>> s.add(u1) - >>> s.commit() - - >>> # now we can get primary key identity, always - >>> # works in query.get() - >>> b.identity - (3,) - - >>> # the mapper level key - >>> b.identity_key - (, (3,)) - - >>> # state within the session - >>> b.persistent, b.transient, b.deleted, b.detached - (True, False, False, False) - - >>> # owning session - >>> b.session - - -.. seealso:: - - :ref:`core_inspection_toplevel` - -:ticket:`2208` - -New with_polymorphic() feature, can be used anywhere ----------------------------------------------------- - -The :meth:`_query.Query.with_polymorphic` method allows the user to -specify which tables should be present when querying against -a joined-table entity. Unfortunately the method is awkward -and only applies to the first entity in the list, and -otherwise has awkward behaviors both in usage as well as -within the internals. A new enhancement to the -:func:`.aliased` construct has been added called -:func:`.with_polymorphic` which allows any entity to be -"aliased" into a "polymorphic" version of itself, freely -usable anywhere: - -:: - - from sqlalchemy.orm import with_polymorphic - - palias = with_polymorphic(Person, [Engineer, Manager]) - session.query(Company).join(palias, Company.employees).filter( - or_(Engineer.language == "java", Manager.hair == "pointy") - ) - -.. seealso:: - - :ref:`with_polymorphic` - newly updated documentation for polymorphic - loading control. - -:ticket:`2333` - -of_type() works with alias(), with_polymorphic(), any(), has(), joinedload(), subqueryload(), contains_eager() --------------------------------------------------------------------------------------------------------------- - -The :meth:`.PropComparator.of_type` method is used to specify -a specific subtype to use when constructing SQL expressions along -a :func:`_orm.relationship` that has a :term:`polymorphic` mapping as its target. -This method can now be used to target *any number* of target subtypes, -by combining it with the new :func:`.with_polymorphic` function:: - - # use eager loading in conjunction with with_polymorphic targets - Job_P = with_polymorphic(Job, [SubJob, ExtraJob], aliased=True) - q = ( - s.query(DataContainer) - .join(DataContainer.jobs.of_type(Job_P)) - .options(contains_eager(DataContainer.jobs.of_type(Job_P))) - ) - -The method now works equally well in most places a regular relationship -attribute is accepted, including with loader functions like -:func:`_orm.joinedload`, :func:`.subqueryload`, :func:`.contains_eager`, -and comparison methods like :meth:`.PropComparator.any` -and :meth:`.PropComparator.has`:: - - # use eager loading in conjunction with with_polymorphic targets - Job_P = with_polymorphic(Job, [SubJob, ExtraJob], aliased=True) - q = ( - s.query(DataContainer) - .join(DataContainer.jobs.of_type(Job_P)) - .options(contains_eager(DataContainer.jobs.of_type(Job_P))) - ) - - # pass subclasses to eager loads (implicitly applies with_polymorphic) - q = s.query(ParentThing).options( - joinedload_all(ParentThing.container, DataContainer.jobs.of_type(SubJob)) - ) - - # control self-referential aliasing with any()/has() - Job_A = aliased(Job) - q = ( - s.query(Job) - .join(DataContainer.jobs) - .filter( - DataContainer.jobs.of_type(Job_A).any( - and_(Job_A.id < Job.id, Job_A.type == "fred") - ) - ) - ) - -.. seealso:: - - :ref:`inheritance_of_type` - -:ticket:`2438` :ticket:`1106` - -Events Can Be Applied to Unmapped Superclasses ----------------------------------------------- - -Mapper and instance events can now be associated with an unmapped -superclass, where those events will be propagated to subclasses -as those subclasses are mapped. The ``propagate=True`` flag -should be used. This feature allows events to be associated -with a declarative base class:: - - from sqlalchemy.ext.declarative import declarative_base - - Base = declarative_base() - - - @event.listens_for("load", Base, propagate=True) - def on_load(target, context): - print("New instance loaded:", target) - - - # on_load() will be applied to SomeClass - class SomeClass(Base): - __tablename__ = "sometable" - - # ... - -:ticket:`2585` - -Declarative Distinguishes Between Modules/Packages --------------------------------------------------- - -A key feature of Declarative is the ability to refer -to other mapped classes using their string name. The -registry of class names is now sensitive to the owning -module and package of a given class. The classes -can be referred to via dotted name in expressions:: - - class Snack(Base): - # ... - - peanuts = relationship( - "nuts.Peanut", primaryjoin="nuts.Peanut.snack_id == Snack.id" - ) - -The resolution allows that any full or partial -disambiguating package name can be used. If the -path to a particular class is still ambiguous, -an error is raised. - -:ticket:`2338` - - -New DeferredReflection Feature in Declarative ---------------------------------------------- - -The "deferred reflection" example has been moved to a -supported feature within Declarative. This feature allows -the construction of declarative mapped classes with only -placeholder ``Table`` metadata, until a ``prepare()`` step -is called, given an ``Engine`` with which to reflect fully -all tables and establish actual mappings. The system -supports overriding of columns, single and joined -inheritance, as well as distinct bases-per-engine. A full -declarative configuration can now be created against an -existing table that is assembled upon engine creation time -in one step: - -:: - - class ReflectedOne(DeferredReflection, Base): - __abstract__ = True - - - class ReflectedTwo(DeferredReflection, Base): - __abstract__ = True - - - class MyClass(ReflectedOne): - __tablename__ = "mytable" - - - class MyOtherClass(ReflectedOne): - __tablename__ = "myothertable" - - - class YetAnotherClass(ReflectedTwo): - __tablename__ = "yetanothertable" - - - ReflectedOne.prepare(engine_one) - ReflectedTwo.prepare(engine_two) - -.. seealso:: - - :class:`.DeferredReflection` - -:ticket:`2485` - -ORM Classes Now Accepted by Core Constructs -------------------------------------------- - -While the SQL expressions used with :meth:`_query.Query.filter`, -such as ``User.id == 5``, have always been compatible for -use with core constructs such as :func:`_expression.select`, the mapped -class itself would not be recognized when passed to :func:`_expression.select`, -:meth:`_expression.Select.select_from`, or :meth:`_expression.Select.correlate`. -A new SQL registration system allows a mapped class to be -accepted as a FROM clause within the core:: - - from sqlalchemy import select - - stmt = select([User]).where(User.id == 5) - -Above, the mapped ``User`` class will expand into -the :class:`_schema.Table` to which ``User`` is mapped. - -:ticket:`2245` - -.. _change_orm_2365: - -Query.update() supports UPDATE..FROM ------------------------------------- - -The new UPDATE..FROM mechanics work in query.update(). -Below, we emit an UPDATE against ``SomeEntity``, adding -a FROM clause (or equivalent, depending on backend) -against ``SomeOtherEntity``:: - - query(SomeEntity).filter(SomeEntity.id == SomeOtherEntity.id).filter( - SomeOtherEntity.foo == "bar" - ).update({"data": "x"}) - -In particular, updates to joined-inheritance -entities are supported, provided the target of the UPDATE is local to the -table being filtered on, or if the parent and child tables -are mixed, they are joined explicitly in the query. Below, -given ``Engineer`` as a joined subclass of ``Person``: - -:: - - query(Engineer).filter(Person.id == Engineer.id).filter( - Person.name == "dilbert" - ).update({"engineer_data": "java"}) - -would produce: - -.. sourcecode:: sql - - UPDATE engineer SET engineer_data='java' FROM person - WHERE person.id=engineer.id AND person.name='dilbert' - -:ticket:`2365` - -rollback() will only roll back "dirty" objects from a begin_nested() --------------------------------------------------------------------- - -A behavioral change that should improve efficiency for those -users using SAVEPOINT via ``Session.begin_nested()`` - upon -``rollback()``, only those objects that were made dirty -since the last flush will be expired, the rest of the -``Session`` remains intact. This because a ROLLBACK to a -SAVEPOINT does not terminate the containing transaction's -isolation, so no expiry is needed except for those changes -that were not flushed in the current transaction. - -:ticket:`2452` - -Caching Example now uses dogpile.cache --------------------------------------- - -The caching example now uses `dogpile.cache `_. -Dogpile.cache is a rewrite of the caching portion -of Beaker, featuring vastly simpler and faster operation, -as well as support for distributed locking. - -Note that the SQLAlchemy APIs used by the Dogpile example as well -as the previous Beaker example have changed slightly, in particular -this change is needed as illustrated in the Beaker example: - -.. sourcecode:: diff - - --- examples/beaker_caching/caching_query.py - +++ examples/beaker_caching/caching_query.py - @@ -222,7 +222,8 @@ - - """ - if query._current_path: - - mapper, key = query._current_path[-2:] - + mapper, prop = query._current_path[-2:] - + key = prop.key - - for cls in mapper.class_.__mro__: - if (cls, key) in self._relationship_options: - -.. seealso:: - - :ref:`examples_caching` - -:ticket:`2589` - -New Core Features -================= - -Fully extensible, type-level operator support in Core ------------------------------------------------------ - -The Core has to date never had any system of adding support -for new SQL operators to Column and other expression -constructs, other than the :meth:`.ColumnOperators.op` method -which is "just enough" to make things work. There has also -never been any system in place for Core which allows the -behavior of existing operators to be overridden. Up until -now, the only way operators could be flexibly redefined was -in the ORM layer, using :func:`.column_property` given a -``comparator_factory`` argument. Third party libraries -like GeoAlchemy therefore were forced to be ORM-centric and -rely upon an array of hacks to apply new operations as well -as to get them to propagate correctly. - -The new operator system in Core adds the one hook that's -been missing all along, which is to associate new and -overridden operators with *types*. Since after all, it's -not really a column, CAST operator, or SQL function that -really drives what kinds of operations are present, it's the -*type* of the expression. The implementation details are -minimal - only a few extra methods are added to the core -:class:`_expression.ColumnElement` type so that it consults its -:class:`.TypeEngine` object for an optional set of operators. -New or revised operations can be associated with any type, -either via subclassing of an existing type, by using -:class:`.TypeDecorator`, or "globally across-the-board" by -attaching a new :class:`.TypeEngine.Comparator` object to an existing type -class. - -For example, to add logarithm support to :class:`.Numeric` types: - -:: - - - from sqlalchemy.types import Numeric - from sqlalchemy.sql import func - - - class CustomNumeric(Numeric): - class comparator_factory(Numeric.Comparator): - def log(self, other): - return func.log(self.expr, other) - -The new type is usable like any other type: - -:: - - - data = Table( - "data", - metadata, - Column("id", Integer, primary_key=True), - Column("x", CustomNumeric(10, 5)), - Column("y", CustomNumeric(10, 5)), - ) - - stmt = select([data.c.x.log(data.c.y)]).where(data.c.x.log(2) < value) - print(conn.execute(stmt).fetchall()) - -New features which have come from this immediately include -support for PostgreSQL's HSTORE type, as well as new -operations associated with PostgreSQL's ARRAY -type. It also paves the way for existing types to acquire -lots more operators that are specific to those types, such -as more string, integer and date operators. - -.. seealso:: - - :ref:`types_operators` - - :class:`.HSTORE` - -:ticket:`2547` - -.. _feature_2623: - -Multiple-VALUES support for Insert ----------------------------------- - -The :meth:`_expression.Insert.values` method now supports a list of dictionaries, -which will render a multi-VALUES statement such as -``VALUES (), (), ...``. This is only relevant to backends which -support this syntax, including PostgreSQL, SQLite, and MySQL. It is -not the same thing as the usual ``executemany()`` style of INSERT which -remains unchanged:: - - users.insert().values( - [ - {"name": "some name"}, - {"name": "some other name"}, - {"name": "yet another name"}, - ] - ) - -.. seealso:: - - :meth:`_expression.Insert.values` - -:ticket:`2623` - -Type Expressions ----------------- - -SQL expressions can now be associated with types. Historically, -:class:`.TypeEngine` has always allowed Python-side functions which -receive both bound parameters as well as result row values, passing -them through a Python side conversion function on the way to/back from -the database. The new feature allows similar -functionality, except on the database side:: - - from sqlalchemy.types import String - from sqlalchemy import func, Table, Column, MetaData - - - class LowerString(String): - def bind_expression(self, bindvalue): - return func.lower(bindvalue) - - def column_expression(self, col): - return func.lower(col) - - - metadata = MetaData() - test_table = Table("test_table", metadata, Column("data", LowerString)) - -Above, the ``LowerString`` type defines a SQL expression that will be emitted -whenever the ``test_table.c.data`` column is rendered in the columns -clause of a SELECT statement: - -.. sourcecode:: pycon+sql - - >>> print(select([test_table]).where(test_table.c.data == "HI")) - {printsql}SELECT lower(test_table.data) AS data - FROM test_table - WHERE test_table.data = lower(:data_1) - -This feature is also used heavily by the new release of GeoAlchemy, -to embed PostGIS expressions inline in SQL based on type rules. - -.. seealso:: - - :ref:`types_sql_value_processing` - -:ticket:`1534` - -Core Inspection System ----------------------- - -The :func:`_sa.inspect` function introduced in :ref:`feature_orminspection_08` -also applies to the core. Applied to an :class:`_engine.Engine` it produces -an :class:`_reflection.Inspector` object:: - - from sqlalchemy import inspect - from sqlalchemy import create_engine - - engine = create_engine("postgresql://scott:tiger@localhost/test") - insp = inspect(engine) - print(insp.get_table_names()) - -It can also be applied to any :class:`_expression.ClauseElement`, which returns -the :class:`_expression.ClauseElement` itself, such as :class:`_schema.Table`, :class:`_schema.Column`, -:class:`_expression.Select`, etc. This allows it to work fluently between Core -and ORM constructs. - - -New Method :meth:`_expression.Select.correlate_except` -------------------------------------------------------- -:func:`_expression.select` now has a method :meth:`_expression.Select.correlate_except` -which specifies "correlate on all FROM clauses except those -specified". It can be used for mapping scenarios where -a related subquery should correlate normally, except -against a particular target selectable:: - - class SnortEvent(Base): - __tablename__ = "event" - - id = Column(Integer, primary_key=True) - signature = Column(Integer, ForeignKey("signature.id")) - - signatures = relationship("Signature", lazy=False) - - - class Signature(Base): - __tablename__ = "signature" - - id = Column(Integer, primary_key=True) - - sig_count = column_property( - select([func.count("*")]) - .where(SnortEvent.signature == id) - .correlate_except(SnortEvent) - ) - -.. seealso:: - - :meth:`_expression.Select.correlate_except` - -PostgreSQL HSTORE type ----------------------- - -Support for PostgreSQL's ``HSTORE`` type is now available as -:class:`_postgresql.HSTORE`. This type makes great usage -of the new operator system to provide a full range of operators -for HSTORE types, including index access, concatenation, -and containment methods such as -:meth:`~.HSTORE.comparator_factory.has_key`, -:meth:`~.HSTORE.comparator_factory.has_any`, and -:meth:`~.HSTORE.comparator_factory.matrix`:: - - from sqlalchemy.dialects.postgresql import HSTORE - - data = Table( - "data_table", - metadata, - Column("id", Integer, primary_key=True), - Column("hstore_data", HSTORE), - ) - - engine.execute(select([data.c.hstore_data["some_key"]])).scalar() - - engine.execute(select([data.c.hstore_data.matrix()])).scalar() - -.. seealso:: - - :class:`_postgresql.HSTORE` - - :class:`_postgresql.hstore` - -:ticket:`2606` - -Enhanced PostgreSQL ARRAY type ------------------------------- - -The :class:`_postgresql.ARRAY` type will accept an optional -"dimension" argument, pinning it to a fixed number of -dimensions and greatly improving efficiency when retrieving -results: - -:: - - # old way, still works since PG supports N-dimensions per row: - Column("my_array", postgresql.ARRAY(Integer)) - - # new way, will render ARRAY with correct number of [] in DDL, - # will process binds and results more efficiently as we don't need - # to guess how many levels deep to go - Column("my_array", postgresql.ARRAY(Integer, dimensions=2)) - -The type also introduces new operators, using the new type-specific -operator framework. New operations include indexed access:: - - result = conn.execute(select([mytable.c.arraycol[2]])) - -slice access in SELECT:: - - result = conn.execute(select([mytable.c.arraycol[2:4]])) - -slice updates in UPDATE:: - - conn.execute(mytable.update().values({mytable.c.arraycol[2:3]: [7, 8]})) - -freestanding array literals:: - - >>> from sqlalchemy.dialects import postgresql - >>> conn.scalar(select([postgresql.array([1, 2]) + postgresql.array([3, 4, 5])])) - [1, 2, 3, 4, 5] - -array concatenation, where below, the right side ``[4, 5, 6]`` is coerced into an array literal:: - - select([mytable.c.arraycol + [4, 5, 6]]) - -.. seealso:: - - :class:`_postgresql.ARRAY` - - :class:`_postgresql.array` - -:ticket:`2441` - -New, configurable DATE, TIME types for SQLite ---------------------------------------------- - -SQLite has no built-in DATE, TIME, or DATETIME types, and -instead provides some support for storage of date and time -values either as strings or integers. The date and time -types for SQLite are enhanced in 0.8 to be much more -configurable as to the specific format, including that the -"microseconds" portion is optional, as well as pretty much -everything else. - -:: - - Column("sometimestamp", sqlite.DATETIME(truncate_microseconds=True)) - Column( - "sometimestamp", - sqlite.DATETIME( - storage_format=( - "%(year)04d%(month)02d%(day)02d" - "%(hour)02d%(minute)02d%(second)02d%(microsecond)06d" - ), - regexp="(\d{4})(\d{2})(\d{2})(\d{2})(\d{2})(\d{2})(\d{6})", - ), - ) - Column( - "somedate", - sqlite.DATE( - storage_format="%(month)02d/%(day)02d/%(year)04d", - regexp="(?P\d+)/(?P\d+)/(?P\d+)", - ), - ) - -Huge thanks to Nate Dub for the sprinting on this at Pycon 2012. - -.. seealso:: - - :class:`_sqlite.DATETIME` - - :class:`_sqlite.DATE` - - :class:`_sqlite.TIME` - -:ticket:`2363` - -"COLLATE" supported across all dialects; in particular MySQL, PostgreSQL, SQLite --------------------------------------------------------------------------------- - -The "collate" keyword, long accepted by the MySQL dialect, is now established -on all :class:`.String` types and will render on any backend, including -when features such as :meth:`_schema.MetaData.create_all` and :func:`.cast` is used: - -.. sourcecode:: pycon+sql - - >>> stmt = select([cast(sometable.c.somechar, String(20, collation="utf8"))]) - >>> print(stmt) - {printsql}SELECT CAST(sometable.somechar AS VARCHAR(20) COLLATE "utf8") AS anon_1 - FROM sometable - -.. seealso:: - - :class:`.String` - -:ticket:`2276` - -"Prefixes" now supported for :func:`_expression.update`, :func:`_expression.delete` ------------------------------------------------------------------------------------- -Geared towards MySQL, a "prefix" can be rendered within any of -these constructs. E.g.:: - - stmt = table.delete().prefix_with("LOW_PRIORITY", dialect="mysql") - - - stmt = table.update().prefix_with("LOW_PRIORITY", dialect="mysql") - -The method is new in addition to those which already existed -on :func:`_expression.insert`, :func:`_expression.select` and :class:`_query.Query`. - -.. seealso:: - - :meth:`_expression.Update.prefix_with` - - :meth:`_expression.Delete.prefix_with` - - :meth:`_expression.Insert.prefix_with` - - :meth:`_expression.Select.prefix_with` - - :meth:`_query.Query.prefix_with` - -:ticket:`2431` - - -Behavioral Changes -================== - -.. _legacy_is_orphan_addition: - -The consideration of a "pending" object as an "orphan" has been made more aggressive ------------------------------------------------------------------------------------- - -This is a late add to the 0.8 series, however it is hoped that the new behavior -is generally more consistent and intuitive in a wider variety of -situations. The ORM has since at least version 0.4 included behavior -such that an object that's "pending", meaning that it's -associated with a :class:`.Session` but hasn't been inserted into the database -yet, is automatically expunged from the :class:`.Session` when it becomes an "orphan", -which means it has been de-associated with a parent object that refers to it -with ``delete-orphan`` cascade on the configured :func:`_orm.relationship`. This -behavior is intended to approximately mirror the behavior of a persistent -(that is, already inserted) object, where the ORM will emit a DELETE for such -objects that become orphans based on the interception of detachment events. - -The behavioral change comes into play for objects that -are referred to by multiple kinds of parents that each specify ``delete-orphan``; the -typical example is an :ref:`association object ` that bridges two other kinds of objects -in a many-to-many pattern. Previously, the behavior was such that the -pending object would be expunged only when de-associated with *all* of its parents. -With the behavioral change, the pending object -is expunged as soon as it is de-associated from *any* of the parents that it was -previously associated with. This behavior is intended to more closely -match that of persistent objects, which are deleted as soon -as they are de-associated from any parent. - -The rationale for the older behavior dates back -at least to version 0.4, and was basically a defensive decision to try to alleviate -confusion when an object was still being constructed for INSERT. But the reality -is that the object is re-associated with the :class:`.Session` as soon as it is -attached to any new parent in any case. - -It's still possible to flush an object -that is not associated with all of its required parents, if the object was either -not associated with those parents in the first place, or if it was expunged, but then -re-associated with a :class:`.Session` via a subsequent attachment event but still -not fully associated. In this situation, it is expected that the database -would emit an integrity error, as there are likely NOT NULL foreign key columns -that are unpopulated. The ORM makes the decision to let these INSERT attempts -occur, based on the judgment that an object that is only partially associated with -its required parents but has been actively associated with some of them, -is more often than not a user error, rather than an intentional -omission which should be silently skipped - silently skipping the INSERT here would -make user errors of this nature very hard to debug. - -The old behavior, for applications that might have been relying upon it, can be re-enabled for -any :class:`_orm.Mapper` by specifying the flag ``legacy_is_orphan`` as a mapper -option. - -The new behavior allows the following test case to work:: - - from sqlalchemy import Column, Integer, String, ForeignKey - from sqlalchemy.orm import relationship, backref - from sqlalchemy.ext.declarative import declarative_base - - Base = declarative_base() - - - class User(Base): - __tablename__ = "user" - id = Column(Integer, primary_key=True) - name = Column(String(64)) - - - class UserKeyword(Base): - __tablename__ = "user_keyword" - user_id = Column(Integer, ForeignKey("user.id"), primary_key=True) - keyword_id = Column(Integer, ForeignKey("keyword.id"), primary_key=True) - - user = relationship( - User, backref=backref("user_keywords", cascade="all, delete-orphan") - ) - - keyword = relationship( - "Keyword", backref=backref("user_keywords", cascade="all, delete-orphan") - ) - - # uncomment this to enable the old behavior - # __mapper_args__ = {"legacy_is_orphan": True} - - - class Keyword(Base): - __tablename__ = "keyword" - id = Column(Integer, primary_key=True) - keyword = Column("keyword", String(64)) - - - from sqlalchemy import create_engine - from sqlalchemy.orm import Session - - # note we're using PostgreSQL to ensure that referential integrity - # is enforced, for demonstration purposes. - e = create_engine("postgresql://scott:tiger@localhost/test", echo=True) - - Base.metadata.drop_all(e) - Base.metadata.create_all(e) - - session = Session(e) - - u1 = User(name="u1") - k1 = Keyword(keyword="k1") - - session.add_all([u1, k1]) - - uk1 = UserKeyword(keyword=k1, user=u1) - - # previously, if session.flush() were called here, - # this operation would succeed, but if session.flush() - # were not called here, the operation fails with an - # integrity error. - # session.flush() - del u1.user_keywords[0] - - session.commit() - -:ticket:`2655` - -The after_attach event fires after the item is associated with the Session instead of before; before_attach added ------------------------------------------------------------------------------------------------------------------ - -Event handlers which use after_attach can now assume the -given instance is associated with the given session: - -:: - - @event.listens_for(Session, "after_attach") - def after_attach(session, instance): - assert instance in session - -Some use cases require that it work this way. However, -other use cases require that the item is *not* yet part of -the session, such as when a query, intended to load some -state required for an instance, emits autoflush first and -would otherwise prematurely flush the target object. Those -use cases should use the new "before_attach" event: - -:: - - @event.listens_for(Session, "before_attach") - def before_attach(session, instance): - instance.some_necessary_attribute = ( - session.query(Widget).filter_by(instance.widget_name).first() - ) - -:ticket:`2464` - - - -Query now auto-correlates like a select() does ----------------------------------------------- - -Previously it was necessary to call :meth:`_query.Query.correlate` in -order to have a column- or WHERE-subquery correlate to the -parent: - -:: - - subq = ( - session.query(Entity.value) - .filter(Entity.id == Parent.entity_id) - .correlate(Parent) - .as_scalar() - ) - session.query(Parent).filter(subq == "some value") - -This was the opposite behavior of a plain ``select()`` -construct which would assume auto-correlation by default. -The above statement in 0.8 will correlate automatically: - -:: - - subq = session.query(Entity.value).filter(Entity.id == Parent.entity_id).as_scalar() - session.query(Parent).filter(subq == "some value") - -like in ``select()``, correlation can be disabled by calling -``query.correlate(None)`` or manually set by passing an -entity, ``query.correlate(someentity)``. - -:ticket:`2179` - -.. _correlation_context_specific: - -Correlation is now always context-specific ------------------------------------------- - -To allow a wider variety of correlation scenarios, the behavior of -:meth:`_expression.Select.correlate` and :meth:`_query.Query.correlate` has changed slightly -such that the SELECT statement will omit the "correlated" target from the -FROM clause only if the statement is actually used in that context. Additionally, -it's no longer possible for a SELECT statement that's placed as a FROM -in an enclosing SELECT statement to "correlate" (i.e. omit) a FROM clause. - -This change only makes things better as far as rendering SQL, in that it's no -longer possible to render illegal SQL where there are insufficient FROM -objects relative to what's being selected:: - - from sqlalchemy.sql import table, column, select - - t1 = table("t1", column("x")) - t2 = table("t2", column("y")) - s = select([t1, t2]).correlate(t1) - - print(s) - -Prior to this change, the above would return: - -.. sourcecode:: sql - - SELECT t1.x, t2.y FROM t2 - -which is invalid SQL as "t1" is not referred to in any FROM clause. - -Now, in the absence of an enclosing SELECT, it returns: - -.. sourcecode:: sql - - SELECT t1.x, t2.y FROM t1, t2 - -Within a SELECT, the correlation takes effect as expected: - -.. sourcecode:: python - - s2 = select([t1, t2]).where(t1.c.x == t2.c.y).where(t1.c.x == s) - print(s2) - -.. sourcecode:: sql - - SELECT t1.x, t2.y FROM t1, t2 - WHERE t1.x = t2.y AND t1.x = - (SELECT t1.x, t2.y FROM t2) - -This change is not expected to impact any existing applications, as -the correlation behavior remains identical for properly constructed -expressions. Only an application that relies, most likely within a -testing scenario, on the invalid string output of a correlated -SELECT used in a non-correlating context would see any change. - -:ticket:`2668` - - -.. _metadata_create_drop_tables: - -create_all() and drop_all() will now honor an empty list as such ----------------------------------------------------------------- - -The methods :meth:`_schema.MetaData.create_all` and :meth:`_schema.MetaData.drop_all` -will now accept a list of :class:`_schema.Table` objects that is empty, -and will not emit any CREATE or DROP statements. Previously, -an empty list was interpreted the same as passing ``None`` -for a collection, and CREATE/DROP would be emitted for all -items unconditionally. - -This is a bug fix but some applications may have been relying upon -the previous behavior. - -:ticket:`2664` - -Repaired the Event Targeting of :class:`.InstrumentationEvents` ---------------------------------------------------------------- - -The :class:`.InstrumentationEvents` series of event targets have -documented that the events will only be fired off according to -the actual class passed as a target. Through 0.7, this wasn't the -case, and any event listener applied to :class:`.InstrumentationEvents` -would be invoked for all classes mapped. In 0.8, additional -logic has been added so that the events will only invoke for those -classes sent in. The ``propagate`` flag here is set to ``True`` -by default as class instrumentation events are typically used to -intercept classes that aren't yet created. - -:ticket:`2590` - -No more magic coercion of "=" to IN when comparing to subquery in MS-SQL ------------------------------------------------------------------------- - -We found a very old behavior in the MSSQL dialect which -would attempt to rescue users from themselves when -doing something like this: - -:: - - scalar_subq = select([someothertable.c.id]).where(someothertable.c.data == "foo") - select([sometable]).where(sometable.c.id == scalar_subq) - -SQL Server doesn't allow an equality comparison to a scalar -SELECT, that is, "x = (SELECT something)". The MSSQL dialect -would convert this to an IN. The same thing would happen -however upon a comparison like "(SELECT something) = x", and -overall this level of guessing is outside of SQLAlchemy's -usual scope so the behavior is removed. - -:ticket:`2277` - -Fixed the behavior of :meth:`.Session.is_modified` --------------------------------------------------- - -The :meth:`.Session.is_modified` method accepts an argument -``passive`` which basically should not be necessary, the -argument in all cases should be the value ``True`` - when -left at its default of ``False`` it would have the effect of -hitting the database, and often triggering autoflush which -would itself change the results. In 0.8 the ``passive`` -argument will have no effect, and unloaded attributes will -never be checked for history since by definition there can -be no pending state change on an unloaded attribute. - -.. seealso:: - - :meth:`.Session.is_modified` - -:ticket:`2320` - -:attr:`_schema.Column.key` is honored in the :attr:`_expression.Select.c` attribute of :func:`_expression.select` with :meth:`_expression.Select.apply_labels` ---------------------------------------------------------------------------------------------------------------------------------------------------------------- -Users of the expression system know that :meth:`_expression.Select.apply_labels` -prepends the table name to each column name, affecting the -names that are available from :attr:`_expression.Select.c`: - -:: - - s = select([table1]).apply_labels() - s.c.table1_col1 - s.c.table1_col2 - -Before 0.8, if the :class:`_schema.Column` had a different :attr:`_schema.Column.key`, this -key would be ignored, inconsistently versus when -:meth:`_expression.Select.apply_labels` were not used: - -:: - - # before 0.8 - table1 = Table("t1", metadata, Column("col1", Integer, key="column_one")) - s = select([table1]) - s.c.column_one # would be accessible like this - s.c.col1 # would raise AttributeError - - s = select([table1]).apply_labels() - s.c.table1_column_one # would raise AttributeError - s.c.table1_col1 # would be accessible like this - -In 0.8, :attr:`_schema.Column.key` is honored in both cases: - -:: - - # with 0.8 - table1 = Table("t1", metadata, Column("col1", Integer, key="column_one")) - s = select([table1]) - s.c.column_one # works - s.c.col1 # AttributeError - - s = select([table1]).apply_labels() - s.c.table1_column_one # works - s.c.table1_col1 # AttributeError - -All other behavior regarding "name" and "key" are the same, -including that the rendered SQL will still use the form -``_`` - the emphasis here was on -preventing the :attr:`_schema.Column.key` contents from being rendered into the -``SELECT`` statement so that there are no issues with -special/ non-ascii characters used in the :attr:`_schema.Column.key`. - -:ticket:`2397` - -single_parent warning is now an error -------------------------------------- - -A :func:`_orm.relationship` that is many-to-one or many-to-many and -specifies "cascade='all, delete-orphan'", which is an -awkward but nonetheless supported use case (with -restrictions) will now raise an error if the relationship -does not specify the ``single_parent=True`` option. -Previously it would only emit a warning, but a failure would -follow almost immediately within the attribute system in any -case. - -:ticket:`2405` - -Adding the ``inspector`` argument to the ``column_reflect`` event ------------------------------------------------------------------ - -0.7 added a new event called ``column_reflect``, provided so -that the reflection of columns could be augmented as each -one were reflected. We got this event slightly wrong in -that the event gave no way to get at the current -``Inspector`` and ``Connection`` being used for the -reflection, in the case that additional information from the -database is needed. As this is a new event not widely used -yet, we'll be adding the ``inspector`` argument into it -directly:: - - @event.listens_for(Table, "column_reflect") - def listen_for_col(inspector, table, column_info): - ... - -:ticket:`2418` - -Disabling auto-detect of collations, casing for MySQL ------------------------------------------------------ - -The MySQL dialect does two calls, one very expensive, to -load all possible collations from the database as well as -information on casing, the first time an ``Engine`` -connects. Neither of these collections are used for any -SQLAlchemy functions, so these calls will be changed to no -longer be emitted automatically. Applications that might -have relied on these collections being present on -``engine.dialect`` will need to call upon -``_detect_collations()`` and ``_detect_casing()`` directly. - -:ticket:`2404` - -"Unconsumed column names" warning becomes an exception ------------------------------------------------------- - -Referring to a non-existent column in an ``insert()`` or -``update()`` construct will raise an error instead of a -warning: - -:: - - t1 = table("t1", column("x")) - t1.insert().values(x=5, z=5) # raises "Unconsumed column names: z" - -:ticket:`2415` - -Inspector.get_primary_keys() is deprecated, use Inspector.get_pk_constraint ---------------------------------------------------------------------------- - -These two methods on ``Inspector`` were redundant, where -``get_primary_keys()`` would return the same information as -``get_pk_constraint()`` minus the name of the constraint: - -:: - - >>> insp.get_primary_keys() - ["a", "b"] - - >>> insp.get_pk_constraint() - {"name":"pk_constraint", "constrained_columns":["a", "b"]} - -:ticket:`2422` - -Case-insensitive result row names will be disabled in most cases ----------------------------------------------------------------- - -A very old behavior, the column names in ``RowProxy`` were -always compared case-insensitively: - -:: - - >>> row = result.fetchone() - >>> row["foo"] == row["FOO"] == row["Foo"] - True - -This was for the benefit of a few dialects which in the -early days needed this, like Oracle and Firebird, but in -modern usage we have more accurate ways of dealing with the -case-insensitive behavior of these two platforms. - -Going forward, this behavior will be available only -optionally, by passing the flag ```case_sensitive=False``` -to ```create_engine()```, but otherwise column names -requested from the row must match as far as casing. - -:ticket:`2423` - -``InstrumentationManager`` and alternate class instrumentation is now an extension ----------------------------------------------------------------------------------- - -The ``sqlalchemy.orm.interfaces.InstrumentationManager`` -class is moved to -``sqlalchemy.ext.instrumentation.InstrumentationManager``. -The "alternate instrumentation" system was built for the -benefit of a very small number of installations that needed -to work with existing or unusual class instrumentation -systems, and generally is very seldom used. The complexity -of this system has been exported to an ``ext.`` module. It -remains unused until once imported, typically when a third -party library imports ``InstrumentationManager``, at which -point it is injected back into ``sqlalchemy.orm`` by -replacing the default ``InstrumentationFactory`` with -``ExtendedInstrumentationRegistry``. - -Removed -======= - -SQLSoup -------- - -SQLSoup is a handy package that presents an alternative -interface on top of the SQLAlchemy ORM. SQLSoup is now -moved into its own project and documented/released -separately; see https://bitbucket.org/zzzeek/sqlsoup. - -SQLSoup is a very simple tool that could also benefit from -contributors who are interested in its style of usage. - -:ticket:`2262` - -MutableType ------------ - -The older "mutable" system within the SQLAlchemy ORM has -been removed. This refers to the ``MutableType`` interface -which was applied to types such as ``PickleType`` and -conditionally to ``TypeDecorator``, and since very early -SQLAlchemy versions has provided a way for the ORM to detect -changes in so-called "mutable" data structures such as JSON -structures and pickled objects. However, the -implementation was never reasonable and forced a very -inefficient mode of usage on the unit-of-work which caused -an expensive scan of all objects to take place during flush. -In 0.7, the `sqlalchemy.ext.mutable `_ extension was -introduced so that user-defined datatypes can appropriately -send events to the unit of work as changes occur. - -Today, usage of ``MutableType`` is expected to be low, as -warnings have been in place for some years now regarding its -inefficiency. - -:ticket:`2442` - -sqlalchemy.exceptions (has been sqlalchemy.exc for years) ---------------------------------------------------------- - -We had left in an alias ``sqlalchemy.exceptions`` to attempt -to make it slightly easier for some very old libraries that -hadn't yet been upgraded to use ``sqlalchemy.exc``. Some -users are still being confused by it however so in 0.8 we're -taking it out entirely to eliminate any of that confusion. - -:ticket:`2433` - diff --git a/doc/build/changelog/migration_09.rst b/doc/build/changelog/migration_09.rst deleted file mode 100644 index 287fc2c933a..00000000000 --- a/doc/build/changelog/migration_09.rst +++ /dev/null @@ -1,1922 +0,0 @@ -============================== -What's New in SQLAlchemy 0.9? -============================== - -.. admonition:: About this Document - - This document describes changes between SQLAlchemy version 0.8, - undergoing maintenance releases as of May, 2013, - and SQLAlchemy version 0.9, which had its first production - release on December 30, 2013. - - Document last updated: June 10, 2015 - -Introduction -============ - -This guide introduces what's new in SQLAlchemy version 0.9, -and also documents changes which affect users migrating -their applications from the 0.8 series of SQLAlchemy to 0.9. - -Please carefully review -:ref:`behavioral_changes_orm_09` and :ref:`behavioral_changes_core_09` for -potentially backwards-incompatible changes. - -Platform Support -================ - -Targeting Python 2.6 and Up Now, Python 3 without 2to3 -------------------------------------------------------- - -The first achievement of the 0.9 release is to remove the dependency -on the 2to3 tool for Python 3 compatibility. To make this -more straightforward, the lowest Python release targeted now -is 2.6, which features a wide degree of cross-compatibility with -Python 3. All SQLAlchemy modules and unit tests are now interpreted -equally well with any Python interpreter from 2.6 forward, including -the 3.1 and 3.2 interpreters. - -:ticket:`2671` - -C Extensions Supported on Python 3 ------------------------------------ - -The C extensions have been ported to support Python 3 and now build -in both Python 2 and Python 3 environments. - -:ticket:`2161` - -.. _behavioral_changes_orm_09: - -Behavioral Changes - ORM -======================== - -.. _migration_2824: - -Composite attributes are now returned as their object form when queried on a per-attribute basis ------------------------------------------------------------------------------------------------- - -Using a :class:`_query.Query` in conjunction with a composite attribute now returns the object -type maintained by that composite, rather than being broken out into individual -columns. Using the mapping setup at :ref:`mapper_composite`:: - - >>> session.query(Vertex.start, Vertex.end).filter(Vertex.start == Point(3, 4)).all() - [(Point(x=3, y=4), Point(x=5, y=6))] - -This change is backwards-incompatible with code that expects the individual attribute -to be expanded into individual columns. To get that behavior, use the ``.clauses`` -accessor:: - - - >>> session.query(Vertex.start.clauses, Vertex.end.clauses).filter( - ... Vertex.start == Point(3, 4) - ... ).all() - [(3, 4, 5, 6)] - -.. seealso:: - - :ref:`change_2824` - -:ticket:`2824` - - -.. _migration_2736: - -:meth:`_query.Query.select_from` no longer applies the clause to corresponding entities ----------------------------------------------------------------------------------------- -The :meth:`_query.Query.select_from` method has been popularized in recent versions -as a means of controlling the first thing that a :class:`_query.Query` object -"selects from", typically for the purposes of controlling how a JOIN will -render. - -Consider the following example against the usual ``User`` mapping:: - - select_stmt = select([User]).where(User.id == 7).alias() - - q = ( - session.query(User) - .join(select_stmt, User.id == select_stmt.c.id) - .filter(User.name == "ed") - ) - -The above statement predictably renders SQL like the following: - -.. sourcecode:: sql - - SELECT "user".id AS user_id, "user".name AS user_name - FROM "user" JOIN (SELECT "user".id AS id, "user".name AS name - FROM "user" - WHERE "user".id = :id_1) AS anon_1 ON "user".id = anon_1.id - WHERE "user".name = :name_1 - -If we wanted to reverse the order of the left and right elements of the -JOIN, the documentation would lead us to believe we could use -:meth:`_query.Query.select_from` to do so:: - - q = ( - session.query(User) - .select_from(select_stmt) - .join(User, User.id == select_stmt.c.id) - .filter(User.name == "ed") - ) - -However, in version 0.8 and earlier, the above use of :meth:`_query.Query.select_from` -would apply the ``select_stmt`` to **replace** the ``User`` entity, as it -selects from the ``user`` table which is compatible with ``User``: - -.. sourcecode:: sql - - -- SQLAlchemy 0.8 and earlier... - SELECT anon_1.id AS anon_1_id, anon_1.name AS anon_1_name - FROM (SELECT "user".id AS id, "user".name AS name - FROM "user" - WHERE "user".id = :id_1) AS anon_1 JOIN "user" ON anon_1.id = anon_1.id - WHERE anon_1.name = :name_1 - -The above statement is a mess, the ON clause refers ``anon_1.id = anon_1.id``, -our WHERE clause has been replaced with ``anon_1`` as well. - -This behavior is quite intentional, but has a different use case from that -which has become popular for :meth:`_query.Query.select_from`. The above behavior -is now available by a new method known as :meth:`_query.Query.select_entity_from`. -This is a lesser used behavior that in modern SQLAlchemy is roughly equivalent -to selecting from a customized :func:`.aliased` construct:: - - select_stmt = select([User]).where(User.id == 7) - user_from_stmt = aliased(User, select_stmt.alias()) - - q = session.query(user_from_stmt).filter(user_from_stmt.name == "ed") - -So with SQLAlchemy 0.9, our query that selects from ``select_stmt`` produces -the SQL we expect: - -.. sourcecode:: sql - - -- SQLAlchemy 0.9 - SELECT "user".id AS user_id, "user".name AS user_name - FROM (SELECT "user".id AS id, "user".name AS name - FROM "user" - WHERE "user".id = :id_1) AS anon_1 JOIN "user" ON "user".id = id - WHERE "user".name = :name_1 - -The :meth:`_query.Query.select_entity_from` method will be available in SQLAlchemy -**0.8.2**, so applications which rely on the old behavior can transition -to this method first, ensure all tests continue to function, then upgrade -to 0.9 without issue. - -:ticket:`2736` - - -.. _migration_2833: - -``viewonly=True`` on ``relationship()`` prevents history from taking effect ---------------------------------------------------------------------------- - -The ``viewonly`` flag on :func:`_orm.relationship` is applied to prevent changes -to the target attribute from having any effect within the flush process. -This is achieved by eliminating the attribute from being considered during -the flush. However, up until now, changes to the attribute would still -register the parent object as "dirty" and trigger a potential flush. The change -is that the ``viewonly`` flag now prevents history from being set for the -target attribute as well. Attribute events like backrefs and user-defined events -still continue to function normally. - -The change is illustrated as follows:: - - from sqlalchemy import Column, Integer, ForeignKey, create_engine - from sqlalchemy.orm import backref, relationship, Session - from sqlalchemy.ext.declarative import declarative_base - from sqlalchemy import inspect - - Base = declarative_base() - - - class A(Base): - __tablename__ = "a" - id = Column(Integer, primary_key=True) - - - class B(Base): - __tablename__ = "b" - - id = Column(Integer, primary_key=True) - a_id = Column(Integer, ForeignKey("a.id")) - a = relationship("A", backref=backref("bs", viewonly=True)) - - - e = create_engine("sqlite://") - Base.metadata.create_all(e) - - a = A() - b = B() - - sess = Session(e) - sess.add_all([a, b]) - sess.commit() - - b.a = a - - assert b in sess.dirty - - # before 0.9.0 - # assert a in sess.dirty - # assert inspect(a).attrs.bs.history.has_changes() - - # after 0.9.0 - assert a not in sess.dirty - assert not inspect(a).attrs.bs.history.has_changes() - -:ticket:`2833` - -.. _migration_2751: - -Association Proxy SQL Expression Improvements and Fixes -------------------------------------------------------- - -The ``==`` and ``!=`` operators as implemented by an association proxy -that refers to a scalar value on a scalar relationship now produces -a more complete SQL expression, intended to take into account -the "association" row being present or not when the comparison is against -``None``. - -Consider this mapping:: - - class A(Base): - __tablename__ = "a" - - id = Column(Integer, primary_key=True) - - b_id = Column(Integer, ForeignKey("b.id"), primary_key=True) - b = relationship("B") - b_value = association_proxy("b", "value") - - - class B(Base): - __tablename__ = "b" - id = Column(Integer, primary_key=True) - value = Column(String) - -Up through 0.8, a query like the following:: - - s.query(A).filter(A.b_value == None).all() - -would produce: - -.. sourcecode:: sql - - SELECT a.id AS a_id, a.b_id AS a_b_id - FROM a - WHERE EXISTS (SELECT 1 - FROM b - WHERE b.id = a.b_id AND b.value IS NULL) - -In 0.9, it now produces: - -.. sourcecode:: sql - - SELECT a.id AS a_id, a.b_id AS a_b_id - FROM a - WHERE (EXISTS (SELECT 1 - FROM b - WHERE b.id = a.b_id AND b.value IS NULL)) OR a.b_id IS NULL - -The difference being, it not only checks ``b.value``, it also checks -if ``a`` refers to no ``b`` row at all. This will return different -results versus prior versions, for a system that uses this type of -comparison where some parent rows have no association row. - -More critically, a correct expression is emitted for ``A.b_value != None``. -In 0.8, this would return ``True`` for ``A`` rows that had no ``b``: - -.. sourcecode:: sql - - SELECT a.id AS a_id, a.b_id AS a_b_id - FROM a - WHERE NOT (EXISTS (SELECT 1 - FROM b - WHERE b.id = a.b_id AND b.value IS NULL)) - -Now in 0.9, the check has been reworked so that it ensures -the A.b_id row is present, in addition to ``B.value`` being -non-NULL: - -.. sourcecode:: sql - - SELECT a.id AS a_id, a.b_id AS a_b_id - FROM a - WHERE EXISTS (SELECT 1 - FROM b - WHERE b.id = a.b_id AND b.value IS NOT NULL) - -In addition, the ``has()`` operator is enhanced such that you can -call it against a scalar column value with no criterion only, -and it will produce criteria that checks for the association row -being present or not:: - - s.query(A).filter(A.b_value.has()).all() - -output: - -.. sourcecode:: sql - - SELECT a.id AS a_id, a.b_id AS a_b_id - FROM a - WHERE EXISTS (SELECT 1 - FROM b - WHERE b.id = a.b_id) - -This is equivalent to ``A.b.has()``, but allows one to query -against ``b_value`` directly. - -:ticket:`2751` - -.. _migration_2810: - -Association Proxy Missing Scalar returns None ---------------------------------------------- - -An association proxy from a scalar attribute to a scalar will now return -``None`` if the proxied object isn't present. This is consistent with the -fact that missing many-to-ones return None in SQLAlchemy, so should the -proxied value. E.g.:: - - from sqlalchemy import * - from sqlalchemy.orm import * - from sqlalchemy.ext.declarative import declarative_base - from sqlalchemy.ext.associationproxy import association_proxy - - Base = declarative_base() - - - class A(Base): - __tablename__ = "a" - - id = Column(Integer, primary_key=True) - b = relationship("B", uselist=False) - - bname = association_proxy("b", "name") - - - class B(Base): - __tablename__ = "b" - - id = Column(Integer, primary_key=True) - a_id = Column(Integer, ForeignKey("a.id")) - name = Column(String) - - - a1 = A() - - # this is how m2o's always have worked - assert a1.b is None - - # but prior to 0.9, this would raise AttributeError, - # now returns None just like the proxied value. - assert a1.bname is None - -:ticket:`2810` - - -.. _change_2787: - -attributes.get_history() will query from the DB by default if value not present -------------------------------------------------------------------------------- - -A bugfix regarding :func:`.attributes.get_history` allows a column-based attribute -to query out to the database for an unloaded value, assuming the ``passive`` -flag is left at its default of ``PASSIVE_OFF``. Previously, this flag would -not be honored. Additionally, a new method :meth:`.AttributeState.load_history` -is added to complement the :attr:`.AttributeState.history` attribute, which -will emit loader callables for an unloaded attribute. - -This is a small change demonstrated as follows:: - - from sqlalchemy import Column, Integer, String, create_engine, inspect - from sqlalchemy.orm import Session, attributes - from sqlalchemy.ext.declarative import declarative_base - - Base = declarative_base() - - - class A(Base): - __tablename__ = "a" - id = Column(Integer, primary_key=True) - data = Column(String) - - - e = create_engine("sqlite://", echo=True) - Base.metadata.create_all(e) - - sess = Session(e) - - a1 = A(data="a1") - sess.add(a1) - sess.commit() # a1 is now expired - - # history doesn't emit loader callables - assert inspect(a1).attrs.data.history == (None, None, None) - - # in 0.8, this would fail to load the unloaded state. - assert attributes.get_history(a1, "data") == ( - (), - [ - "a1", - ], - (), - ) - - # load_history() is now equivalent to get_history() with - # passive=PASSIVE_OFF ^ INIT_OK - assert inspect(a1).attrs.data.load_history() == ( - (), - [ - "a1", - ], - (), - ) - -:ticket:`2787` - -.. _behavioral_changes_core_09: - -Behavioral Changes - Core -========================= - -Type objects no longer accept ignored keyword arguments -------------------------------------------------------- - -Up through the 0.8 series, most type objects accepted arbitrary keyword -arguments which were silently ignored:: - - from sqlalchemy import Date, Integer - - # storage_format argument here has no effect on any backend; - # it needs to be on the SQLite-specific type - d = Date(storage_format="%(day)02d.%(month)02d.%(year)04d") - - # display_width argument here has no effect on any backend; - # it needs to be on the MySQL-specific type - i = Integer(display_width=5) - -This was a very old bug for which a deprecation warning was added to the -0.8 series, but because nobody ever runs Python with the "-W" flag, it -was mostly never seen: - -.. sourcecode:: text - - $ python -W always::DeprecationWarning ~/dev/sqlalchemy/test.py - /Users/classic/dev/sqlalchemy/test.py:5: SADeprecationWarning: Passing arguments to - type object constructor is deprecated - d = Date(storage_format="%(day)02d.%(month)02d.%(year)04d") - /Users/classic/dev/sqlalchemy/test.py:9: SADeprecationWarning: Passing arguments to - type object constructor is deprecated - i = Integer(display_width=5) - -As of the 0.9 series the "catch all" constructor is removed from -:class:`.TypeEngine`, and these meaningless arguments are no longer accepted. - -The correct way to make use of dialect-specific arguments such as -``storage_format`` and ``display_width`` is to use the appropriate -dialect-specific types:: - - from sqlalchemy.dialects.sqlite import DATE - from sqlalchemy.dialects.mysql import INTEGER - - d = DATE(storage_format="%(day)02d.%(month)02d.%(year)04d") - - i = INTEGER(display_width=5) - -What about the case where we want the dialect-agnostic type also? We -use the :meth:`.TypeEngine.with_variant` method:: - - from sqlalchemy import Date, Integer - from sqlalchemy.dialects.sqlite import DATE - from sqlalchemy.dialects.mysql import INTEGER - - d = Date().with_variant( - DATE(storage_format="%(day)02d.%(month)02d.%(year)04d"), "sqlite" - ) - - i = Integer().with_variant(INTEGER(display_width=5), "mysql") - -:meth:`.TypeEngine.with_variant` isn't new, it was added in SQLAlchemy -0.7.2. So code that is running on the 0.8 series can be corrected to use -this approach and tested before upgrading to 0.9. - -``None`` can no longer be used as a "partial AND" constructor --------------------------------------------------------------- - -``None`` can no longer be used as the "backstop" to form an AND condition piecemeal. -This pattern was not a documented pattern even though some SQLAlchemy internals -made use of it:: - - condition = None - - for cond in conditions: - condition = condition & cond - - if condition is not None: - stmt = stmt.where(condition) - -The above sequence, when ``conditions`` is non-empty, will on 0.9 produce -``SELECT .. WHERE AND NULL``. The ``None`` is no longer implicitly -ignored, and is instead consistent with when ``None`` is interpreted in other -contexts besides that of a conjunction. - -The correct code for both 0.8 and 0.9 should read:: - - from sqlalchemy.sql import and_ - - if conditions: - stmt = stmt.where(and_(*conditions)) - -Another variant that works on all backends on 0.9, but on 0.8 only works on -backends that support boolean constants:: - - from sqlalchemy.sql import true - - condition = true() - - for cond in conditions: - condition = cond & condition - - stmt = stmt.where(condition) - -On 0.8, this will produce a SELECT statement that always has ``AND true`` -in the WHERE clause, which is not accepted by backends that don't support -boolean constants (MySQL, MSSQL). On 0.9, the ``true`` constant will be dropped -within an ``and_()`` conjunction. - -.. seealso:: - - :ref:`migration_2804` - -.. _migration_2873: - -The "password" portion of a ``create_engine()`` no longer considers the ``+`` sign as an encoded space ------------------------------------------------------------------------------------------------------- - -For whatever reason, the Python function ``unquote_plus()`` was applied to the -"password" field of a URL, which is an incorrect application of the -encoding rules described in `RFC 1738 `_ -in that it escaped spaces as plus signs. The stringification of a URL -now only encodes ":", "@", or "/" and nothing else, and is now applied to both the -``username`` and ``password`` fields (previously it only applied to the -password). On parsing, encoded characters are converted, but plus signs and -spaces are passed through as is: - -.. sourcecode:: text - - # password: "pass word + other:words" - dbtype://user:pass word + other%3Awords@host/dbname - - # password: "apples/oranges" - dbtype://username:apples%2Foranges@hostspec/database - - # password: "apples@oranges@@" - dbtype://username:apples%40oranges%40%40@hostspec/database - - # password: '', username is "username@" - dbtype://username%40:@hostspec/database - - -:ticket:`2873` - -.. _migration_2879: - -The precedence rules for COLLATE have been changed --------------------------------------------------- - -Previously, an expression like the following:: - - print((column("x") == "somevalue").collate("en_EN")) - -would produce an expression like this: - -.. sourcecode:: sql - - -- 0.8 behavior - (x = :x_1) COLLATE en_EN - -The above is misunderstood by MSSQL and is generally not the syntax suggested -for any database. The expression will now produce the syntax illustrated -by that of most database documentation: - -.. sourcecode:: sql - - -- 0.9 behavior - x = :x_1 COLLATE en_EN - -The potentially backwards incompatible change arises if the -:meth:`.ColumnOperators.collate` operator is being applied to the right-hand -column, as follows:: - - print(column("x") == literal("somevalue").collate("en_EN")) - -In 0.8, this produces: - -.. sourcecode:: sql - - x = :param_1 COLLATE en_EN - -However in 0.9, will now produce the more accurate, but probably not what you -want, form of: - -.. sourcecode:: sql - - x = (:param_1 COLLATE en_EN) - -The :meth:`.ColumnOperators.collate` operator now works more appropriately within an -``ORDER BY`` expression as well, as a specific precedence has been given to the -``ASC`` and ``DESC`` operators which will again ensure no parentheses are -generated: - -.. sourcecode:: pycon+sql - - >>> # 0.8 - >>> print(column("x").collate("en_EN").desc()) - {printsql}(x COLLATE en_EN) DESC{stop} - - >>> # 0.9 - >>> print(column("x").collate("en_EN").desc()) - {printsql}x COLLATE en_EN DESC{stop} - -:ticket:`2879` - - - -.. _migration_2878: - -PostgreSQL CREATE TYPE AS ENUM now applies quoting to values ----------------------------------------------------------------- - -The :class:`_postgresql.ENUM` type will now apply escaping to single quote -signs within the enumerated values: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy.dialects import postgresql - >>> type = postgresql.ENUM("one", "two", "three's", name="myenum") - >>> from sqlalchemy.dialects.postgresql import base - >>> print(base.CreateEnumType(type).compile(dialect=postgresql.dialect())) - {printsql}CREATE TYPE myenum AS ENUM ('one','two','three''s') - -Existing workarounds which already escape single quote signs will need to be -modified, else they will now double-escape. - -:ticket:`2878` - -New Features -============ - -.. _feature_2268: - -Event Removal API ------------------ - -Events established using :func:`.event.listen` or :func:`.event.listens_for` -can now be removed using the new :func:`.event.remove` function. The ``target``, -``identifier`` and ``fn`` arguments sent to :func:`.event.remove` need to match -exactly those which were sent for listening, and the event will be removed -from all locations in which it had been established:: - - @event.listens_for(MyClass, "before_insert", propagate=True) - def my_before_insert(mapper, connection, target): - """listen for before_insert""" - # ... - - - event.remove(MyClass, "before_insert", my_before_insert) - -In the example above, the ``propagate=True`` flag is set. This -means ``my_before_insert()`` is established as a listener for ``MyClass`` -as well as all subclasses of ``MyClass``. -The system tracks everywhere that the ``my_before_insert()`` -listener function had been placed as a result of this call and removes it as -a result of calling :func:`.event.remove`. - -The removal system uses a registry to associate arguments passed to -:func:`.event.listen` with collections of event listeners, which are in many -cases wrapped versions of the original user-supplied function. This registry -makes heavy use of weak references in order to allow all the contained contents, -such as listener targets, to be garbage collected when they go out of scope. - -:ticket:`2268` - -.. _feature_1418: - -New Query Options API; ``load_only()`` option ---------------------------------------------- - -The system of loader options such as :func:`_orm.joinedload`, -:func:`_orm.subqueryload`, :func:`_orm.lazyload`, :func:`_orm.defer`, etc. -all build upon a new system known as :class:`_orm.Load`. :class:`_orm.Load` provides -a "method chained" (a.k.a. :term:`generative`) approach to loader options, so that -instead of joining together long paths using dots or multiple attribute names, -an explicit loader style is given for each path. - -While the new way is slightly more verbose, it is simpler to understand -in that there is no ambiguity in what options are being applied to which paths; -it simplifies the method signatures of the options and provides greater flexibility -particularly for column-based options. The old systems are to remain functional -indefinitely as well and all styles can be mixed. - -**Old Way** - -To set a certain style of loading along every link in a multi-element path, the ``_all()`` -option has to be used:: - - query(User).options(joinedload_all("orders.items.keywords")) - -**New Way** - -Loader options are now chainable, so the same ``joinedload(x)`` method is applied -equally to each link, without the need to keep straight between -:func:`_orm.joinedload` and :func:`_orm.joinedload_all`:: - - query(User).options(joinedload("orders").joinedload("items").joinedload("keywords")) - -**Old Way** - -Setting an option on path that is based on a subclass requires that all -links in the path be spelled out as class bound attributes, since the -:meth:`.PropComparator.of_type` method needs to be called:: - - session.query(Company).options( - subqueryload_all(Company.employees.of_type(Engineer), Engineer.machines) - ) - -**New Way** - -Only those elements in the path that actually need :meth:`.PropComparator.of_type` -need to be set as a class-bound attribute, string-based names can be resumed -afterwards:: - - session.query(Company).options( - subqueryload(Company.employees.of_type(Engineer)).subqueryload("machines") - ) - -**Old Way** - -Setting the loader option on the last link in a long path uses a syntax -that looks a lot like it should be setting the option for all links in the -path, causing confusion:: - - query(User).options(subqueryload("orders.items.keywords")) - -**New Way** - -A path can now be spelled out using :func:`.defaultload` for entries in the -path where the existing loader style should be unchanged. More verbose -but the intent is clearer:: - - query(User).options(defaultload("orders").defaultload("items").subqueryload("keywords")) - -The dotted style can still be taken advantage of, particularly in the case -of skipping over several path elements:: - - query(User).options(defaultload("orders.items").subqueryload("keywords")) - -**Old Way** - -The :func:`.defer` option on a path needed to be spelled out with the full -path for each column:: - - query(User).options(defer("orders.description"), defer("orders.isopen")) - -**New Way** - -A single :class:`_orm.Load` object that arrives at the target path can have -:meth:`_orm.Load.defer` called upon it repeatedly:: - - query(User).options(defaultload("orders").defer("description").defer("isopen")) - -The Load Class -^^^^^^^^^^^^^^^ - -The :class:`_orm.Load` class can be used directly to provide a "bound" target, -especially when multiple parent entities are present:: - - from sqlalchemy.orm import Load - - query(User, Address).options(Load(Address).joinedload("entries")) - -Load Only -^^^^^^^^^ - -A new option :func:`.load_only` achieves a "defer everything but" style of load, -loading only the given columns and deferring the rest:: - - from sqlalchemy.orm import load_only - - query(User).options(load_only("name", "fullname")) - - # specify explicit parent entity - query(User, Address).options(Load(User).load_only("name", "fullname")) - - # specify path - query(User).options(joinedload(User.addresses).load_only("email_address")) - -Class-specific Wildcards -^^^^^^^^^^^^^^^^^^^^^^^^^ - -Using :class:`_orm.Load`, a wildcard may be used to set the loading for all -relationships (or perhaps columns) on a given entity, without affecting any -others:: - - # lazyload all User relationships - query(User).options(Load(User).lazyload("*")) - - # undefer all User columns - query(User).options(Load(User).undefer("*")) - - # lazyload all Address relationships - query(User).options(defaultload(User.addresses).lazyload("*")) - - # undefer all Address columns - query(User).options(defaultload(User.addresses).undefer("*")) - -:ticket:`1418` - - -.. _feature_2877: - -New ``text()`` Capabilities ---------------------------- - -The :func:`_expression.text` construct gains new methods: - -* :meth:`_expression.TextClause.bindparams` allows bound parameter types and values - to be set flexibly:: - - # setup values - stmt = text( - "SELECT id, name FROM user WHERE name=:name AND timestamp=:timestamp" - ).bindparams(name="ed", timestamp=datetime(2012, 11, 10, 15, 12, 35)) - - # setup types and/or values - stmt = ( - text("SELECT id, name FROM user WHERE name=:name AND timestamp=:timestamp") - .bindparams(bindparam("name", value="ed"), bindparam("timestamp", type_=DateTime())) - .bindparam(timestamp=datetime(2012, 11, 10, 15, 12, 35)) - ) - -* :meth:`_expression.TextClause.columns` supersedes the ``typemap`` option - of :func:`_expression.text`, returning a new construct :class:`.TextAsFrom`:: - - # turn a text() into an alias(), with a .c. collection: - stmt = text("SELECT id, name FROM user").columns(id=Integer, name=String) - stmt = stmt.alias() - - stmt = select([addresses]).select_from( - addresses.join(stmt), addresses.c.user_id == stmt.c.id - ) - - - # or into a cte(): - stmt = text("SELECT id, name FROM user").columns(id=Integer, name=String) - stmt = stmt.cte("x") - - stmt = select([addresses]).select_from( - addresses.join(stmt), addresses.c.user_id == stmt.c.id - ) - -:ticket:`2877` - -.. _feature_722: - -INSERT from SELECT ------------------- - -After literally years of pointless procrastination this relatively minor -syntactical feature has been added, and is also backported to 0.8.3, -so technically isn't "new" in 0.9. A :func:`_expression.select` construct or other -compatible construct can be passed to the new method :meth:`_expression.Insert.from_select` -where it will be used to render an ``INSERT .. SELECT`` construct: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy.sql import table, column - >>> t1 = table("t1", column("a"), column("b")) - >>> t2 = table("t2", column("x"), column("y")) - >>> print(t1.insert().from_select(["a", "b"], t2.select().where(t2.c.y == 5))) - {printsql}INSERT INTO t1 (a, b) SELECT t2.x, t2.y - FROM t2 - WHERE t2.y = :y_1 - -The construct is smart enough to also accommodate ORM objects such as classes -and :class:`_query.Query` objects:: - - s = Session() - q = s.query(User.id, User.name).filter_by(name="ed") - ins = insert(Address).from_select((Address.id, Address.email_address), q) - -rendering: - -.. sourcecode:: sql - - INSERT INTO addresses (id, email_address) - SELECT users.id AS users_id, users.name AS users_name - FROM users WHERE users.name = :name_1 - -:ticket:`722` - -.. _feature_github_42: - -New FOR UPDATE support on ``select()``, ``Query()`` ---------------------------------------------------- - -An attempt is made to simplify the specification of the ``FOR UPDATE`` -clause on ``SELECT`` statements made within Core and ORM, and support is added -for the ``FOR UPDATE OF`` SQL supported by PostgreSQL and Oracle. - -Using the core :meth:`_expression.GenerativeSelect.with_for_update`, options like ``FOR SHARE`` and -``NOWAIT`` can be specified individually, rather than linking to arbitrary -string codes:: - - stmt = select([table]).with_for_update(read=True, nowait=True, of=table) - -On Posgtresql the above statement might render like: - -.. sourcecode:: sql - - SELECT table.a, table.b FROM table FOR SHARE OF table NOWAIT - -The :class:`_query.Query` object gains a similar method :meth:`_query.Query.with_for_update` -which behaves in the same way. This method supersedes the existing -:meth:`_query.Query.with_lockmode` method, which translated ``FOR UPDATE`` clauses -using a different system. At the moment, the "lockmode" string argument is still -accepted by the :meth:`.Session.refresh` method. - - -.. _feature_2867: - -Floating Point String-Conversion Precision Configurable for Native Floating Point Types ---------------------------------------------------------------------------------------- - -The conversion which SQLAlchemy does whenever a DBAPI returns a Python -floating point type which is to be converted into a Python ``Decimal()`` -necessarily involves an intermediary step which converts the floating point -value to a string. The scale used for this string conversion was previously -hardcoded to 10, and is now configurable. The setting is available on -both the :class:`.Numeric` as well as the :class:`.Float` -type, as well as all SQL- and dialect-specific descendant types, using the -parameter ``decimal_return_scale``. If the type supports a ``.scale`` parameter, -as is the case with :class:`.Numeric` and some float types such as -:class:`.mysql.DOUBLE`, the value of ``.scale`` is used as the default -for ``.decimal_return_scale`` if it is not otherwise specified. If both -``.scale`` and ``.decimal_return_scale`` are absent, then the default of -10 takes place. E.g.:: - - from sqlalchemy.dialects.mysql import DOUBLE - import decimal - - data = Table( - "data", - metadata, - Column("double_value", mysql.DOUBLE(decimal_return_scale=12, asdecimal=True)), - ) - - conn.execute( - data.insert(), - double_value=45.768392065789, - ) - result = conn.scalar(select([data.c.double_value])) - - # previously, this would typically be Decimal("45.7683920658"), - # e.g. trimmed to 10 decimal places - - # now we get 12, as requested, as MySQL can support this - # much precision for DOUBLE - assert result == decimal.Decimal("45.768392065789") - -:ticket:`2867` - - -.. _change_2824: - -Column Bundles for ORM queries ------------------------------- - -The :class:`.Bundle` allows for querying of sets of columns, which are then -grouped into one name under the tuple returned by the query. The initial -purposes of :class:`.Bundle` are 1. to allow "composite" ORM columns to be -returned as a single value in a column-based result set, rather than expanding -them out into individual columns and 2. to allow the creation of custom result-set -constructs within the ORM, using ad-hoc columns and return types, without involving -the more heavyweight mechanics of mapped classes. - -.. seealso:: - - :ref:`migration_2824` - - :ref:`bundles` - -:ticket:`2824` - - -Server Side Version Counting ------------------------------ - -The versioning feature of the ORM (now also documented at :ref:`mapper_version_counter`) -can now make use of server-side version counting schemes, such as those produced -by triggers or database system columns, as well as conditional programmatic schemes outside -of the version_id_counter function itself. By providing the value ``False`` -to the ``version_id_generator`` parameter, the ORM will use the already-set version -identifier, or alternatively fetch the version identifier -from each row at the same time the INSERT or UPDATE is emitted. When using a -server-generated version identifier, it is strongly -recommended that this feature be used only on a backend with strong RETURNING -support (PostgreSQL, SQL Server; Oracle also supports RETURNING but the cx_oracle -driver has only limited support), else the additional SELECT statements will -add significant performance -overhead. The example provided at :ref:`server_side_version_counter` illustrates -the usage of the PostgreSQL ``xmin`` system column in order to integrate it with -the ORM's versioning feature. - -.. seealso:: - - :ref:`server_side_version_counter` - -:ticket:`2793` - -.. _feature_1535: - -``include_backrefs=False`` option for ``@validates`` ----------------------------------------------------- - -The :func:`.validates` function now accepts an option ``include_backrefs=True``, -which will bypass firing the validator for the case where the event initiated -from a backref:: - - from sqlalchemy import Column, Integer, ForeignKey - from sqlalchemy.orm import relationship, validates - from sqlalchemy.ext.declarative import declarative_base - - Base = declarative_base() - - - class A(Base): - __tablename__ = "a" - - id = Column(Integer, primary_key=True) - bs = relationship("B", backref="a") - - @validates("bs") - def validate_bs(self, key, item): - print("A.bs validator") - return item - - - class B(Base): - __tablename__ = "b" - - id = Column(Integer, primary_key=True) - a_id = Column(Integer, ForeignKey("a.id")) - - @validates("a", include_backrefs=False) - def validate_a(self, key, item): - print("B.a validator") - return item - - - a1 = A() - a1.bs.append(B()) # prints only "A.bs validator" - -:ticket:`1535` - - -PostgreSQL JSON Type --------------------- - -The PostgreSQL dialect now features a :class:`_postgresql.JSON` type to -complement the :class:`_postgresql.HSTORE` type. - -.. seealso:: - - :class:`_postgresql.JSON` - -:ticket:`2581` - -.. _feature_automap: - -Automap Extension ------------------ - -A new extension is added in **0.9.1** known as :mod:`sqlalchemy.ext.automap`. This is an -**experimental** extension which expands upon the functionality of Declarative -as well as the :class:`.DeferredReflection` class. Essentially, the extension -provides a base class :class:`.AutomapBase` which automatically generates -mapped classes and relationships between them based on given table metadata. - -The :class:`_schema.MetaData` in use normally might be produced via reflection, but -there is no requirement that reflection is used. The most basic usage -illustrates how :mod:`sqlalchemy.ext.automap` is able to deliver mapped -classes, including relationships, based on a reflected schema:: - - from sqlalchemy.ext.automap import automap_base - from sqlalchemy.orm import Session - from sqlalchemy import create_engine - - Base = automap_base() - - # engine, suppose it has two tables 'user' and 'address' set up - engine = create_engine("sqlite:///mydatabase.db") - - # reflect the tables - Base.prepare(engine, reflect=True) - - # mapped classes are now created with names matching that of the table - # name. - User = Base.classes.user - Address = Base.classes.address - - session = Session(engine) - - # rudimentary relationships are produced - session.add(Address(email_address="foo@bar.com", user=User(name="foo"))) - session.commit() - - # collection-based relationships are by default named "_collection" - print(u1.address_collection) - -Beyond that, the :class:`.AutomapBase` class is a declarative base, and supports -all the features that declarative does. The "automapping" feature can be used -with an existing, explicitly declared schema to generate relationships and -missing classes only. Naming schemes and relationship-production routines -can be dropped in using callable functions. - -It is hoped that the :class:`.AutomapBase` system provides a quick -and modernized solution to the problem that the very famous -`SQLSoup `_ -also tries to solve, that of generating a quick and rudimentary object -model from an existing database on the fly. By addressing the issue strictly -at the mapper configuration level, and integrating fully with existing -Declarative class techniques, :class:`.AutomapBase` seeks to provide -a well-integrated approach to the issue of expediently auto-generating ad-hoc -mappings. - -.. seealso:: - - :ref:`automap_toplevel` - -Behavioral Improvements -======================= - -Improvements that should produce no compatibility issues except in exceedingly -rare and unusual hypothetical cases, but are good to be aware of in case there are -unexpected issues. - -.. _feature_joins_09: - -Many JOIN and LEFT OUTER JOIN expressions will no longer be wrapped in (SELECT * FROM ..) AS ANON_1 ---------------------------------------------------------------------------------------------------- - -For many years, the SQLAlchemy ORM has been held back from being able to nest -a JOIN inside the right side of an existing JOIN (typically a LEFT OUTER JOIN, -as INNER JOINs could always be flattened): - -.. sourcecode:: sql - - SELECT a.*, b.*, c.* FROM a LEFT OUTER JOIN (b JOIN c ON b.id = c.id) ON a.id - -This was due to the fact that SQLite up until version **3.7.16** cannot parse a statement of the above format: - -.. sourcecode:: text - - SQLite version 3.7.15.2 2013-01-09 11:53:05 - Enter ".help" for instructions - Enter SQL statements terminated with a ";" - sqlite> create table a(id integer); - sqlite> create table b(id integer); - sqlite> create table c(id integer); - sqlite> select a.id, b.id, c.id from a left outer join (b join c on b.id=c.id) on b.id=a.id; - Error: no such column: b.id - -Right-outer-joins are of course another way to work around right-side -parenthesization; this would be significantly complicated and visually unpleasant -to implement, but fortunately SQLite doesn't support RIGHT OUTER JOIN either :): - -.. sourcecode:: sql - - sqlite> select a.id, b.id, c.id from b join c on b.id=c.id - ...> right outer join a on b.id=a.id; - Error: RIGHT and FULL OUTER JOINs are not currently supported - -Back in 2005, it wasn't clear if other databases had trouble with this form, -but today it seems clear every database tested except SQLite now supports it -(Oracle 8, a very old database, doesn't support the JOIN keyword at all, -but SQLAlchemy has always had a simple rewriting scheme in place for Oracle's syntax). -To make matters worse, SQLAlchemy's usual workaround of applying a -SELECT often degrades performance on platforms like PostgreSQL and MySQL: - -.. sourcecode:: sql - - SELECT a.*, anon_1.* FROM a LEFT OUTER JOIN ( - SELECT b.id AS b_id, c.id AS c_id - FROM b JOIN c ON b.id = c.id - ) AS anon_1 ON a.id=anon_1.b_id - -A JOIN like the above form is commonplace when working with joined-table inheritance structures; -any time :meth:`_query.Query.join` is used to join from some parent to a joined-table subclass, or -when :func:`_orm.joinedload` is used similarly, SQLAlchemy's ORM would always make sure a nested -JOIN was never rendered, lest the query wouldn't be able to run on SQLite. Even though -the Core has always supported a JOIN of the more compact form, the ORM had to avoid it. - -An additional issue would arise when producing joins across many-to-many relationships -where special criteria is present in the ON clause. Consider an eager load join like the following:: - - session.query(Order).outerjoin(Order.items) - -Assuming a many-to-many from ``Order`` to ``Item`` which actually refers to a subclass -like ``Subitem``, the SQL for the above would look like: - -.. sourcecode:: sql - - SELECT order.id, order.name - FROM order LEFT OUTER JOIN order_item ON order.id = order_item.order_id - LEFT OUTER JOIN item ON order_item.item_id = item.id AND item.type = 'subitem' - -What's wrong with the above query? Basically, that it will load many ``order`` / -``order_item`` rows where the criteria of ``item.type == 'subitem'`` is not true. - -As of SQLAlchemy 0.9, an entirely new approach has been taken. The ORM no longer -worries about nesting JOINs in the right side of an enclosing JOIN, and it now will -render these as often as possible while still returning the correct results. When -the SQL statement is passed to be compiled, the **dialect compiler** will **rewrite the join** -to suit the target backend, if that backend is known to not support a right-nested -JOIN (which currently is only SQLite - if other backends have this issue please -let us know!). - -So a regular ``query(Parent).join(Subclass)`` will now usually produce a simpler -expression: - -.. sourcecode:: sql - - SELECT parent.id AS parent_id - FROM parent JOIN ( - base_table JOIN subclass_table - ON base_table.id = subclass_table.id) ON parent.id = base_table.parent_id - -Joined eager loads like ``query(Parent).options(joinedload(Parent.subclasses))`` -will alias the individual tables instead of wrapping in an ``ANON_1``: - -.. sourcecode:: sql - - SELECT parent.*, base_table_1.*, subclass_table_1.* FROM parent - LEFT OUTER JOIN ( - base_table AS base_table_1 JOIN subclass_table AS subclass_table_1 - ON base_table_1.id = subclass_table_1.id) - ON parent.id = base_table_1.parent_id - -Many-to-many joins and eagerloads will right nest the "secondary" and "right" tables: - -.. sourcecode:: sql - - SELECT order.id, order.name - FROM order LEFT OUTER JOIN - (order_item JOIN item ON order_item.item_id = item.id AND item.type = 'subitem') - ON order_item.order_id = order.id - -All of these joins, when rendered with a :class:`_expression.Select` statement that specifically -specifies ``use_labels=True``, which is true for all the queries the ORM emits, -are candidates for "join rewriting", which is the process of rewriting all those right-nested -joins into nested SELECT statements, while maintaining the identical labeling used by -the :class:`_expression.Select`. So SQLite, the one database that won't support this very -common SQL syntax even in 2013, shoulders the extra complexity itself, -with the above queries rewritten as: - -.. sourcecode:: sql - - -- sqlite only! - SELECT parent.id AS parent_id - FROM parent JOIN ( - SELECT base_table.id AS base_table_id, - base_table.parent_id AS base_table_parent_id, - subclass_table.id AS subclass_table_id - FROM base_table JOIN subclass_table ON base_table.id = subclass_table.id - ) AS anon_1 ON parent.id = anon_1.base_table_parent_id - - -- sqlite only! - SELECT parent.id AS parent_id, anon_1.subclass_table_1_id AS subclass_table_1_id, - anon_1.base_table_1_id AS base_table_1_id, - anon_1.base_table_1_parent_id AS base_table_1_parent_id - FROM parent LEFT OUTER JOIN ( - SELECT base_table_1.id AS base_table_1_id, - base_table_1.parent_id AS base_table_1_parent_id, - subclass_table_1.id AS subclass_table_1_id - FROM base_table AS base_table_1 - JOIN subclass_table AS subclass_table_1 ON base_table_1.id = subclass_table_1.id - ) AS anon_1 ON parent.id = anon_1.base_table_1_parent_id - - -- sqlite only! - SELECT "order".id AS order_id - FROM "order" LEFT OUTER JOIN ( - SELECT order_item_1.order_id AS order_item_1_order_id, - order_item_1.item_id AS order_item_1_item_id, - item.id AS item_id, item.type AS item_type - FROM order_item AS order_item_1 - JOIN item ON item.id = order_item_1.item_id AND item.type IN (?) - ) AS anon_1 ON "order".id = anon_1.order_item_1_order_id - -.. note:: - - As of SQLAlchemy 1.1, the workarounds present in this feature for SQLite - will automatically disable themselves when SQLite version **3.7.16** - or greater is detected, as SQLite has repaired support for right-nested joins. - -The :meth:`_expression.Join.alias`, :func:`.aliased` and :func:`.with_polymorphic` functions now -support a new argument, ``flat=True``, which is used to construct aliases of joined-table -entities without embedding into a SELECT. This flag is not on by default, to help with -backwards compatibility - but now a "polymorphic" selectable can be joined as a target -without any subqueries generated:: - - employee_alias = with_polymorphic(Person, [Engineer, Manager], flat=True) - - session.query(Company).join(Company.employees.of_type(employee_alias)).filter( - or_(Engineer.primary_language == "python", Manager.manager_name == "dilbert") - ) - -Generates (everywhere except SQLite): - -.. sourcecode:: sql - - SELECT companies.company_id AS companies_company_id, companies.name AS companies_name - FROM companies JOIN ( - people AS people_1 - LEFT OUTER JOIN engineers AS engineers_1 ON people_1.person_id = engineers_1.person_id - LEFT OUTER JOIN managers AS managers_1 ON people_1.person_id = managers_1.person_id - ) ON companies.company_id = people_1.company_id - WHERE engineers.primary_language = %(primary_language_1)s - OR managers.manager_name = %(manager_name_1)s - -:ticket:`2369` :ticket:`2587` - -.. _feature_2976: - -Right-nested inner joins available in joined eager loads ---------------------------------------------------------- - -As of version 0.9.4, the above mentioned right-nested joining can be enabled -in the case of a joined eager load where an "outer" join is linked to an "inner" -on the right side. - -Normally, a joined eager load chain like the following:: - - query(User).options( - joinedload("orders", innerjoin=False).joinedload("items", innerjoin=True) - ) - -Would not produce an inner join; because of the LEFT OUTER JOIN from user->order, -joined eager loading could not use an INNER join from order->items without changing -the user rows that are returned, and would instead ignore the "chained" ``innerjoin=True`` -directive. How 0.9.0 should have delivered this would be that instead of: - -.. sourcecode:: sql - - FROM users LEFT OUTER JOIN orders ON LEFT OUTER JOIN items ON - -the new "right-nested joins are OK" logic would kick in, and we'd get: - -.. sourcecode:: sql - - FROM users LEFT OUTER JOIN (orders JOIN items ON ) ON - -Since we missed the boat on that, to avoid further regressions we've added the above -functionality by specifying the string ``"nested"`` to :paramref:`_orm.joinedload.innerjoin`:: - - query(User).options( - joinedload("orders", innerjoin=False).joinedload("items", innerjoin="nested") - ) - -This feature is new in 0.9.4. - -:ticket:`2976` - - - -ORM can efficiently fetch just-generated INSERT/UPDATE defaults using RETURNING -------------------------------------------------------------------------------- - -The :class:`_orm.Mapper` has long supported an undocumented flag known as -``eager_defaults=True``. The effect of this flag is that when an INSERT or UPDATE -proceeds, and the row is known to have server-generated default values, -a SELECT would immediately follow it in order to "eagerly" load those new values. -Normally, the server-generated columns are marked as "expired" on the object, -so that no overhead is incurred unless the application actually accesses these -columns soon after the flush. The ``eager_defaults`` flag was therefore not -of much use as it could only decrease performance, and was present only to support -exotic event schemes where users needed default values to be available -immediately within the flush process. - -In 0.9, as a result of the version id enhancements, ``eager_defaults`` can now -emit a RETURNING clause for these values, so on a backend with strong RETURNING -support in particular PostgreSQL, the ORM can fetch newly generated default -and SQL expression values inline with the INSERT or UPDATE. ``eager_defaults``, -when enabled, makes use of RETURNING automatically when the target backend -and :class:`_schema.Table` supports "implicit returning". - -.. _change_2836: - -Subquery Eager Loading will apply DISTINCT to the innermost SELECT for some queries ------------------------------------------------------------------------------------- - -In an effort to reduce the number of duplicate rows that can be generated -by subquery eager loading when a many-to-one relationship is involved, a -DISTINCT keyword will be applied to the innermost SELECT when the join is -targeting columns that do not comprise the primary key, as in when loading -along a many to one. - -That is, when subquery loading on a many-to-one from A->B: - -.. sourcecode:: sql - - SELECT b.id AS b_id, b.name AS b_name, anon_1.b_id AS a_b_id - FROM (SELECT DISTINCT a_b_id FROM a) AS anon_1 - JOIN b ON b.id = anon_1.a_b_id - -Since ``a.b_id`` is a non-distinct foreign key, DISTINCT is applied so that -redundant ``a.b_id`` are eliminated. The behavior can be turned on or off -unconditionally for a particular :func:`_orm.relationship` using the flag -``distinct_target_key``, setting the value to ``True`` for unconditionally -on, ``False`` for unconditionally off, and ``None`` for the feature to take -effect when the target SELECT is against columns that do not comprise a full -primary key. In 0.9, ``None`` is the default. - -The option is also backported to 0.8 where the ``distinct_target_key`` -option defaults to ``False``. - -While the feature here is designed to help performance by eliminating -duplicate rows, the ``DISTINCT`` keyword in SQL itself can have a negative -performance impact. If columns in the SELECT are not indexed, ``DISTINCT`` -will likely perform an ``ORDER BY`` on the rowset which can be expensive. -By keeping the feature limited just to foreign keys which are hopefully -indexed in any case, it's expected that the new defaults are reasonable. - -The feature also does not eliminate every possible dupe-row scenario; if -a many-to-one is present elsewhere in the chain of joins, dupe rows may still -be present. - -:ticket:`2836` - -.. _migration_2789: - -Backref handlers can now propagate more than one level deep ------------------------------------------------------------ - -The mechanism by which attribute events pass along their "initiator", that is -the object associated with the start of the event, has been changed; instead -of a :class:`.AttributeImpl` being passed, a new object :class:`.attributes.Event` -is passed instead; this object refers to the :class:`.AttributeImpl` as well as -to an "operation token", representing if the operation is an append, remove, -or replace operation. - -The attribute event system no longer looks at this "initiator" object in order to halt a -recursive series of attribute events. Instead, the system of preventing endless -recursion due to mutually-dependent backref handlers has been moved -to the ORM backref event handlers specifically, which now take over the role -of ensuring that a chain of mutually-dependent events (such as append to collection -A.bs, set many-to-one attribute B.a in response) doesn't go into an endless recursion -stream. The rationale here is that the backref system, given more detail and control -over event propagation, can finally allow operations more than one level deep -to occur; the typical scenario is when a collection append results in a many-to-one -replacement operation, which in turn should cause the item to be removed from a -previous collection:: - - class Parent(Base): - __tablename__ = "parent" - - id = Column(Integer, primary_key=True) - children = relationship("Child", backref="parent") - - - class Child(Base): - __tablename__ = "child" - - id = Column(Integer, primary_key=True) - parent_id = Column(ForeignKey("parent.id")) - - - p1 = Parent() - p2 = Parent() - c1 = Child() - - p1.children.append(c1) - - assert c1.parent is p1 # backref event establishes c1.parent as p1 - - p2.children.append(c1) - - assert c1.parent is p2 # backref event establishes c1.parent as p2 - assert c1 not in p1.children # second backref event removes c1 from p1.children - -Above, prior to this change, the ``c1`` object would still have been present -in ``p1.children``, even though it is also present in ``p2.children`` at the -same time; the backref handlers would have stopped at replacing ``c1.parent`` with -``p2`` instead of ``p1``. In 0.9, using the more detailed :class:`.Event` -object as well as letting the backref handlers make more detailed decisions about -these objects, the propagation can continue onto removing ``c1`` from ``p1.children`` -while maintaining a check against the propagation from going into an endless -recursive loop. - -End-user code which a. makes use of the :meth:`.AttributeEvents.set`, -:meth:`.AttributeEvents.append`, or :meth:`.AttributeEvents.remove` events, -and b. initiates further attribute modification operations as a result of these -events may need to be modified to prevent recursive loops, as the attribute system -no longer stops a chain of events from propagating endlessly in the absence of the backref -event handlers. Additionally, code which depends upon the value of the ``initiator`` -will need to be adjusted to the new API, and furthermore must be ready for the -value of ``initiator`` to change from its original value within a string of -backref-initiated events, as the backref handlers may now swap in a -new ``initiator`` value for some operations. - -:ticket:`2789` - -.. _change_2838: - -The typing system now handles the task of rendering "literal bind" values -------------------------------------------------------------------------- - -A new method is added to :class:`.TypeEngine` :meth:`.TypeEngine.literal_processor` -as well as :meth:`.TypeDecorator.process_literal_param` for :class:`.TypeDecorator` -which take on the task of rendering so-called "inline literal parameters" - parameters -that normally render as "bound" values, but are instead being rendered inline -into the SQL statement due to the compiler configuration. This feature is used -when generating DDL for constructs such as :class:`.CheckConstraint`, as well -as by Alembic when using constructs such as ``op.inline_literal()``. Previously, -a simple "isinstance" check checked for a few basic types, and the "bind processor" -was used unconditionally, leading to such issues as strings being encoded into utf-8 -prematurely. - -Custom types written with :class:`.TypeDecorator` should continue to work in -"inline literal" scenarios, as the :meth:`.TypeDecorator.process_literal_param` -falls back to :meth:`.TypeDecorator.process_bind_param` by default, as these methods -usually handle a data manipulation, not as much how the data is presented to the -database. :meth:`.TypeDecorator.process_literal_param` can be specified to -specifically produce a string representing how a value should be rendered -into an inline DDL statement. - -:ticket:`2838` - - -.. _change_2812: - -Schema identifiers now carry along their own quoting information ---------------------------------------------------------------------- - -This change simplifies the Core's usage of so-called "quote" flags, such -as the ``quote`` flag passed to :class:`_schema.Table` and :class:`_schema.Column`. The flag -is now internalized within the string name itself, which is now represented -as an instance of :class:`.quoted_name`, a string subclass. The -:class:`.IdentifierPreparer` now relies solely on the quoting preferences -reported by the :class:`.quoted_name` object rather than checking for any -explicit ``quote`` flags in most cases. The issue resolved here includes -that various case-sensitive methods such as :meth:`_engine.Engine.has_table` as well -as similar methods within dialects now function with explicitly quoted names, -without the need to complicate or introduce backwards-incompatible changes -to those APIs (many of which are 3rd party) with the details of quoting flags - -in particular, a wider range of identifiers now function correctly with the -so-called "uppercase" backends like Oracle, Firebird, and DB2 (backends that -store and report upon table and column names using all uppercase for case -insensitive names). - -The :class:`.quoted_name` object is used internally as needed; however if -other keywords require fixed quoting preferences, the class is available -publicly. - -:ticket:`2812` - -.. _migration_2804: - -Improved rendering of Boolean constants, NULL constants, conjunctions ----------------------------------------------------------------------- - -New capabilities have been added to the :func:`.true` and :func:`.false` -constants, in particular in conjunction with :func:`.and_` and :func:`.or_` -functions as well as the behavior of the WHERE/HAVING clauses in conjunction -with these types, boolean types overall, and the :func:`.null` constant. - -Starting with a table such as this:: - - from sqlalchemy import Table, Boolean, Integer, Column, MetaData - - t1 = Table("t", MetaData(), Column("x", Boolean()), Column("y", Integer)) - -A select construct will now render the boolean column as a binary expression -on backends that don't feature ``true``/``false`` constant behavior: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy import select, and_, false, true - >>> from sqlalchemy.dialects import mysql, postgresql - - >>> print(select([t1]).where(t1.c.x).compile(dialect=mysql.dialect())) - {printsql}SELECT t.x, t.y FROM t WHERE t.x = 1 - -The :func:`.and_` and :func:`.or_` constructs will now exhibit quasi -"short circuit" behavior, that is truncating a rendered expression, when a -:func:`.true` or :func:`.false` constant is present: - -.. sourcecode:: pycon+sql - - >>> print( - ... select([t1]).where(and_(t1.c.y > 5, false())).compile(dialect=postgresql.dialect()) - ... ) - {printsql}SELECT t.x, t.y FROM t WHERE false - -:func:`.true` can be used as the base to build up an expression: - -.. sourcecode:: pycon+sql - - >>> expr = true() - >>> expr = expr & (t1.c.y > 5) - >>> print(select([t1]).where(expr)) - {printsql}SELECT t.x, t.y FROM t WHERE t.y > :y_1 - -The boolean constants :func:`.true` and :func:`.false` themselves render as -``0 = 1`` and ``1 = 1`` for a backend with no boolean constants: - -.. sourcecode:: pycon+sql - - >>> print(select([t1]).where(and_(t1.c.y > 5, false())).compile(dialect=mysql.dialect())) - {printsql}SELECT t.x, t.y FROM t WHERE 0 = 1 - -Interpretation of ``None``, while not particularly valid SQL, is at least -now consistent: - -.. sourcecode:: pycon+sql - - >>> print(select([t1.c.x]).where(None)) - {printsql}SELECT t.x FROM t WHERE NULL{stop} - - >>> print(select([t1.c.x]).where(None).where(None)) - {printsql}SELECT t.x FROM t WHERE NULL AND NULL{stop} - - >>> print(select([t1.c.x]).where(and_(None, None))) - {printsql}SELECT t.x FROM t WHERE NULL AND NULL{stop} - -:ticket:`2804` - -.. _migration_1068: - -Label constructs can now render as their name alone in an ORDER BY ------------------------------------------------------------------- - -For the case where a :class:`.Label` is used in both the columns clause -as well as the ORDER BY clause of a SELECT, the label will render as -just its name in the ORDER BY clause, assuming the underlying dialect -reports support of this feature. - -E.g. an example like:: - - from sqlalchemy.sql import table, column, select, func - - t = table("t", column("c1"), column("c2")) - expr = (func.foo(t.c.c1) + t.c.c2).label("expr") - - stmt = select([expr]).order_by(expr) - - print(stmt) - -Prior to 0.9 would render as: - -.. sourcecode:: sql - - SELECT foo(t.c1) + t.c2 AS expr - FROM t ORDER BY foo(t.c1) + t.c2 - -And now renders as: - -.. sourcecode:: sql - - SELECT foo(t.c1) + t.c2 AS expr - FROM t ORDER BY expr - -The ORDER BY only renders the label if the label isn't further -embedded into an expression within the ORDER BY, other than a simple -``ASC`` or ``DESC``. - -The above format works on all databases tested, but might have -compatibility issues with older database versions (MySQL 4? Oracle 8? -etc.). Based on user reports we can add rules that will disable the -feature based on database version detection. - -:ticket:`1068` - -.. _migration_2848: - -``RowProxy`` now has tuple-sorting behavior -------------------------------------------- - -The :class:`.RowProxy` object acts much like a tuple, but up until now -would not sort as a tuple if a list of them were sorted using ``sorted()``. -The ``__eq__()`` method now compares both sides as a tuple and also -an ``__lt__()`` method has been added:: - - users.insert().execute( - dict(user_id=1, user_name="foo"), - dict(user_id=2, user_name="bar"), - dict(user_id=3, user_name="def"), - ) - - rows = users.select().order_by(users.c.user_name).execute().fetchall() - - eq_(rows, [(2, "bar"), (3, "def"), (1, "foo")]) - - eq_(sorted(rows), [(1, "foo"), (2, "bar"), (3, "def")]) - -:ticket:`2848` - -.. _migration_2850: - -A bindparam() construct with no type gets upgraded via copy when a type is available ------------------------------------------------------------------------------------- - -The logic which "upgrades" a :func:`.bindparam` construct to take on the -type of the enclosing expression has been improved in two ways. First, the -:func:`.bindparam` object is **copied** before the new type is assigned, so that -the given :func:`.bindparam` is not mutated in place. Secondly, this same -operation occurs when an :class:`_expression.Insert` or :class:`_expression.Update` construct is compiled, -regarding the "values" that were set in the statement via the :meth:`.ValuesBase.values` -method. - -If given an untyped :func:`.bindparam`:: - - bp = bindparam("some_col") - -If we use this parameter as follows:: - - expr = mytable.c.col == bp - -The type for ``bp`` remains as ``NullType``, however if ``mytable.c.col`` -is of type ``String``, then ``expr.right``, that is the right side of the -binary expression, will take on the ``String`` type. Previously, ``bp`` itself -would have been changed in place to have ``String`` as its type. - -Similarly, this operation occurs in an :class:`_expression.Insert` or :class:`_expression.Update`:: - - stmt = mytable.update().values(col=bp) - -Above, ``bp`` remains unchanged, but the ``String`` type will be used when -the statement is executed, which we can see by examining the ``binds`` dictionary:: - - >>> compiled = stmt.compile() - >>> compiled.binds["some_col"].type - String - -The feature allows custom types to take their expected effect within INSERT/UPDATE -statements without needing to explicitly specify those types within every -:func:`.bindparam` expression. - -The potentially backwards-compatible changes involve two unlikely -scenarios. Since the bound parameter is -**cloned**, users should not be relying upon making in-place changes to a -:func:`.bindparam` construct once created. Additionally, code which uses -:func:`.bindparam` within an :class:`_expression.Insert` or :class:`_expression.Update` statement -which is relying on the fact that the :func:`.bindparam` is not typed according -to the column being assigned towards will no longer function in that way. - -:ticket:`2850` - - -.. _migration_1765: - -Columns can reliably get their type from a column referred to via ForeignKey ----------------------------------------------------------------------------- - -There's a long standing behavior which says that a :class:`_schema.Column` can be -declared without a type, as long as that :class:`_schema.Column` is referred to -by a :class:`_schema.ForeignKeyConstraint`, and the type from the referenced column -will be copied into this one. The problem has been that this feature never -worked very well and wasn't maintained. The core issue was that the -:class:`_schema.ForeignKey` object doesn't know what target :class:`_schema.Column` it -refers to until it is asked, typically the first time the foreign key is used -to construct a :class:`_expression.Join`. So until that time, the parent :class:`_schema.Column` -would not have a type, or more specifically, it would have a default type -of :class:`.NullType`. - -While it's taken a long time, the work to reorganize the initialization of -:class:`_schema.ForeignKey` objects has been completed such that this feature can -finally work acceptably. At the core of the change is that the :attr:`_schema.ForeignKey.column` -attribute no longer lazily initializes the location of the target :class:`_schema.Column`; -the issue with this system was that the owning :class:`_schema.Column` would be stuck -with :class:`.NullType` as its type until the :class:`_schema.ForeignKey` happened to -be used. - -In the new version, the :class:`_schema.ForeignKey` coordinates with the eventual -:class:`_schema.Column` it will refer to using internal attachment events, so that the -moment the referencing :class:`_schema.Column` is associated with the -:class:`_schema.MetaData`, all :class:`_schema.ForeignKey` objects that -refer to it will be sent a message that they need to initialize their parent -column. This system is more complicated but works more solidly; as a bonus, -there are now tests in place for a wide variety of :class:`_schema.Column` / -:class:`_schema.ForeignKey` configuration scenarios and error messages have been -improved to be very specific to no less than seven different error conditions. - -Scenarios which now work correctly include: - -1. The type on a :class:`_schema.Column` is immediately present as soon as the - target :class:`_schema.Column` becomes associated with the same :class:`_schema.MetaData`; - this works no matter which side is configured first:: - - >>> from sqlalchemy import Table, MetaData, Column, Integer, ForeignKey - >>> metadata = MetaData() - >>> t2 = Table("t2", metadata, Column("t1id", ForeignKey("t1.id"))) - >>> t2.c.t1id.type - NullType() - >>> t1 = Table("t1", metadata, Column("id", Integer, primary_key=True)) - >>> t2.c.t1id.type - Integer() - -2. The system now works with :class:`_schema.ForeignKeyConstraint` as well:: - - >>> from sqlalchemy import Table, MetaData, Column, Integer, ForeignKeyConstraint - >>> metadata = MetaData() - >>> t2 = Table( - ... "t2", - ... metadata, - ... Column("t1a"), - ... Column("t1b"), - ... ForeignKeyConstraint(["t1a", "t1b"], ["t1.a", "t1.b"]), - ... ) - >>> t2.c.t1a.type - NullType() - >>> t2.c.t1b.type - NullType() - >>> t1 = Table( - ... "t1", - ... metadata, - ... Column("a", Integer, primary_key=True), - ... Column("b", Integer, primary_key=True), - ... ) - >>> t2.c.t1a.type - Integer() - >>> t2.c.t1b.type - Integer() - -3. It even works for "multiple hops" - that is, a :class:`_schema.ForeignKey` that refers to a - :class:`_schema.Column` that refers to another :class:`_schema.Column`:: - - >>> from sqlalchemy import Table, MetaData, Column, Integer, ForeignKey - >>> metadata = MetaData() - >>> t2 = Table("t2", metadata, Column("t1id", ForeignKey("t1.id"))) - >>> t3 = Table("t3", metadata, Column("t2t1id", ForeignKey("t2.t1id"))) - >>> t2.c.t1id.type - NullType() - >>> t3.c.t2t1id.type - NullType() - >>> t1 = Table("t1", metadata, Column("id", Integer, primary_key=True)) - >>> t2.c.t1id.type - Integer() - >>> t3.c.t2t1id.type - Integer() - -:ticket:`1765` - - -Dialect Changes -=============== - -Firebird ``fdb`` is now the default Firebird dialect. ------------------------------------------------------ - -The ``fdb`` dialect is now used if an engine is created without a dialect -specifier, i.e. ``firebird://``. ``fdb`` is a ``kinterbasdb`` compatible -DBAPI which per the Firebird project is now their official Python driver. - -:ticket:`2504` - -Firebird ``fdb`` and ``kinterbasdb`` set ``retaining=False`` by default ------------------------------------------------------------------------ - -Both the ``fdb`` and ``kinterbasdb`` DBAPIs support a flag ``retaining=True`` -which can be passed to the ``commit()`` and ``rollback()`` methods of its -connection. The documented rationale for this flag is so that the DBAPI -can re-use internal transaction state for subsequent transactions, for the -purposes of improving performance. However, newer documentation refers -to analyses of Firebird's "garbage collection" which expresses that this flag -can have a negative effect on the database's ability to process cleanup -tasks, and has been reported as *lowering* performance as a result. - -It's not clear how this flag is actually usable given this information, -and as it appears to be only a performance enhancing feature, it now defaults -to ``False``. The value can be controlled by passing the flag ``retaining=True`` -to the :func:`_sa.create_engine` call. This is a new flag which is added as of -0.8.2, so applications on 0.8.2 can begin setting this to ``True`` or ``False`` -as desired. - -.. seealso:: - - :mod:`sqlalchemy.dialects.firebird.fdb` - - :mod:`sqlalchemy.dialects.firebird.kinterbasdb` - - https://pythonhosted.org/fdb/usage-guide.html#retaining-transactions - information - on the "retaining" flag. - -:ticket:`2763` - - - - - diff --git a/doc/build/changelog/migration_10.rst b/doc/build/changelog/migration_10.rst deleted file mode 100644 index c7988b3cd55..00000000000 --- a/doc/build/changelog/migration_10.rst +++ /dev/null @@ -1,2778 +0,0 @@ -============================= -What's New in SQLAlchemy 1.0? -============================= - -.. admonition:: About this Document - - This document describes changes between SQLAlchemy version 0.9, - undergoing maintenance releases as of May, 2014, - and SQLAlchemy version 1.0, released in April, 2015. - - Document last updated: June 9, 2015 - -Introduction -============ - -This guide introduces what's new in SQLAlchemy version 1.0, -and also documents changes which affect users migrating -their applications from the 0.9 series of SQLAlchemy to 1.0. - -Please carefully review the sections on behavioral changes for -potentially backwards-incompatible changes in behavior. - - -New Features and Improvements - ORM -=================================== - -New Session Bulk INSERT/UPDATE API ----------------------------------- - -A new series of :class:`.Session` methods which provide hooks directly -into the unit of work's facility for emitting INSERT and UPDATE -statements has been created. When used correctly, this expert-oriented system -can allow ORM-mappings to be used to generate bulk insert and update -statements batched into executemany groups, allowing the statements -to proceed at speeds that rival direct use of the Core. - -.. seealso:: - - :ref:`bulk_operations` - introduction and full documentation - -:ticket:`3100` - -New Performance Example Suite ------------------------------ - -Inspired by the benchmarking done for the :ref:`bulk_operations` feature -as well as for the :ref:`faq_how_to_profile` section of the FAQ, a new -example section has been added which features several scripts designed -to illustrate the relative performance profile of various Core and ORM -techniques. The scripts are organized into use cases, and are packaged -under a single console interface such that any combination of demonstrations -can be run, dumping out timings, Python profile results and/or RunSnake profile -displays. - -.. seealso:: - - :ref:`examples_performance` - -"Baked" Queries ---------------- - -The "baked" query feature is an unusual new approach which allows for -straightforward construction an invocation of :class:`_query.Query` objects -using caching, which upon successive calls features vastly reduced -Python function call overhead (over 75%). By specifying a -:class:`_query.Query` object as a series of lambdas which are only invoked -once, a query as a pre-compiled unit begins to be feasible:: - - from sqlalchemy.ext import baked - from sqlalchemy import bindparam - - bakery = baked.bakery() - - - def search_for_user(session, username, email=None): - - baked_query = bakery(lambda session: session.query(User)) - baked_query += lambda q: q.filter(User.name == bindparam("username")) - - baked_query += lambda q: q.order_by(User.id) - - if email: - baked_query += lambda q: q.filter(User.email == bindparam("email")) - - result = baked_query(session).params(username=username, email=email).all() - - return result - -.. seealso:: - - :ref:`baked_toplevel` - -:ticket:`3054` - -.. _feature_3150: - -Improvements to declarative mixins, ``@declared_attr`` and related features ---------------------------------------------------------------------------- - -The declarative system in conjunction with :class:`.declared_attr` has been -overhauled to support new capabilities. - -A function decorated with :class:`.declared_attr` is now called only **after** -any mixin-based column copies are generated. This means the function can -call upon mixin-established columns and will receive a reference to the correct -:class:`_schema.Column` object:: - - class HasFooBar(object): - foobar = Column(Integer) - - @declared_attr - def foobar_prop(cls): - return column_property("foobar: " + cls.foobar) - - - class SomeClass(HasFooBar, Base): - __tablename__ = "some_table" - id = Column(Integer, primary_key=True) - -Above, ``SomeClass.foobar_prop`` will be invoked against ``SomeClass``, -and ``SomeClass.foobar`` will be the final :class:`_schema.Column` object that is -to be mapped to ``SomeClass``, as opposed to the non-copied object present -directly on ``HasFooBar``, even though the columns aren't mapped yet. - -The :class:`.declared_attr` function now **memoizes** the value -that's returned on a per-class basis, so that repeated calls to the same -attribute will return the same value. We can alter the example to illustrate -this:: - - class HasFooBar(object): - @declared_attr - def foobar(cls): - return Column(Integer) - - @declared_attr - def foobar_prop(cls): - return column_property("foobar: " + cls.foobar) - - - class SomeClass(HasFooBar, Base): - __tablename__ = "some_table" - id = Column(Integer, primary_key=True) - -Previously, ``SomeClass`` would be mapped with one particular copy of -the ``foobar`` column, but the ``foobar_prop`` by calling upon ``foobar`` -a second time would produce a different column. The value of -``SomeClass.foobar`` is now memoized during declarative setup time, so that -even before the attribute is mapped by the mapper, the interim column -value will remain consistent no matter how many times the -:class:`.declared_attr` is called upon. - -The two behaviors above should help considerably with declarative definition -of many types of mapper properties that derive from other attributes, where -the :class:`.declared_attr` function is called upon from other -:class:`.declared_attr` functions locally present before the class is -actually mapped. - -For a pretty slim edge case where one wishes to build a declarative mixin -that establishes distinct columns per subclass, a new modifier -:attr:`.declared_attr.cascading` is added. With this modifier, the -decorated function will be invoked individually for each class in the -mapped inheritance hierarchy. While this is already the behavior for -special attributes such as ``__table_args__`` and ``__mapper_args__``, -for columns and other properties the behavior by default assumes that attribute -is affixed to the base class only, and just inherited from subclasses. -With :attr:`.declared_attr.cascading`, individual behaviors can be -applied:: - - class HasIdMixin(object): - @declared_attr.cascading - def id(cls): - if has_inherited_table(cls): - return Column(ForeignKey("myclass.id"), primary_key=True) - else: - return Column(Integer, primary_key=True) - - - class MyClass(HasIdMixin, Base): - __tablename__ = "myclass" - # ... - - - class MySubClass(MyClass): - """ """ - - # ... - -.. seealso:: - - :ref:`mixin_inheritance_columns` - -Finally, the :class:`.AbstractConcreteBase` class has been reworked -so that a relationship or other mapper property can be set up inline -on the abstract base:: - - from sqlalchemy import Column, Integer, ForeignKey - from sqlalchemy.orm import relationship - from sqlalchemy.ext.declarative import ( - declarative_base, - declared_attr, - AbstractConcreteBase, - ) - - Base = declarative_base() - - - class Something(Base): - __tablename__ = "something" - id = Column(Integer, primary_key=True) - - - class Abstract(AbstractConcreteBase, Base): - id = Column(Integer, primary_key=True) - - @declared_attr - def something_id(cls): - return Column(ForeignKey(Something.id)) - - @declared_attr - def something(cls): - return relationship(Something) - - - class Concrete(Abstract): - __tablename__ = "cca" - __mapper_args__ = {"polymorphic_identity": "cca", "concrete": True} - -The above mapping will set up a table ``cca`` with both an ``id`` and -a ``something_id`` column, and ``Concrete`` will also have a relationship -``something``. The new feature is that ``Abstract`` will also have an -independently configured relationship ``something`` that builds against -the polymorphic union of the base. - -:ticket:`3150` :ticket:`2670` :ticket:`3149` :ticket:`2952` :ticket:`3050` - -ORM full object fetches 25% faster ----------------------------------- - -The mechanics of the ``loading.py`` module as well as the identity map -have undergone several passes of inlining, refactoring, and pruning, so -that a raw load of rows now populates ORM-based objects around 25% faster. -Assuming a 1M row table, a script like the following illustrates the type -of load that's improved the most:: - - import time - from sqlalchemy import Integer, Column, create_engine, Table - from sqlalchemy.orm import Session - from sqlalchemy.ext.declarative import declarative_base - - Base = declarative_base() - - - class Foo(Base): - __table__ = Table( - "foo", - Base.metadata, - Column("id", Integer, primary_key=True), - Column("a", Integer(), nullable=False), - Column("b", Integer(), nullable=False), - Column("c", Integer(), nullable=False), - ) - - - engine = create_engine("mysql+mysqldb://scott:tiger@localhost/test", echo=True) - - sess = Session(engine) - - now = time.time() - - # avoid using all() so that we don't have the overhead of building - # a large list of full objects in memory - for obj in sess.query(Foo).yield_per(100).limit(1000000): - pass - - print("Total time: %d" % (time.time() - now)) - -Local MacBookPro results bench from 19 seconds for 0.9 down to 14 seconds for -1.0. The :meth:`_query.Query.yield_per` call is always a good idea when batching -huge numbers of rows, as it prevents the Python interpreter from having -to allocate a huge amount of memory for all objects and their instrumentation -at once. Without the :meth:`_query.Query.yield_per`, the above script on the -MacBookPro is 31 seconds on 0.9 and 26 seconds on 1.0, the extra time spent -setting up very large memory buffers. - -.. _feature_3176: - -New KeyedTuple implementation dramatically faster -------------------------------------------------- - -We took a look into the :class:`.KeyedTuple` implementation in the hopes -of improving queries like this:: - - rows = sess.query(Foo.a, Foo.b, Foo.c).all() - -The :class:`.KeyedTuple` class is used rather than Python's -``collections.namedtuple()``, because the latter has a very complex -type-creation routine that benchmarks much slower than :class:`.KeyedTuple`. -However, when fetching hundreds of thousands of rows, -``collections.namedtuple()`` quickly overtakes :class:`.KeyedTuple` which -becomes dramatically slower as instance invocation goes up. What to do? -A new type that hedges between the approaches of both. Benching -all three types for "size" (number of rows returned) and "num" -(number of distinct queries), the new "lightweight keyed tuple" either -outperforms both, or lags very slightly behind the faster object, based on -which scenario. In the "sweet spot", where we are both creating a good number -of new types as well as fetching a good number of rows, the lightweight -object totally smokes both namedtuple and KeyedTuple: - -.. sourcecode:: text - - ----------------- - size=10 num=10000 # few rows, lots of queries - namedtuple: 3.60302400589 # namedtuple falls over - keyedtuple: 0.255059957504 # KeyedTuple very fast - lw keyed tuple: 0.582715034485 # lw keyed trails right on KeyedTuple - ----------------- - size=100 num=1000 # <--- sweet spot - namedtuple: 0.365247011185 - keyedtuple: 0.24896979332 - lw keyed tuple: 0.0889317989349 # lw keyed blows both away! - ----------------- - size=10000 num=100 - namedtuple: 0.572599887848 - keyedtuple: 2.54251694679 - lw keyed tuple: 0.613876104355 - ----------------- - size=1000000 num=10 # few queries, lots of rows - namedtuple: 5.79669594765 # namedtuple very fast - keyedtuple: 28.856498003 # KeyedTuple falls over - lw keyed tuple: 6.74346804619 # lw keyed trails right on namedtuple - - -:ticket:`3176` - -.. _feature_slots: - -Significant Improvements in Structural Memory Use -------------------------------------------------- - -Structural memory use has been improved via much more significant use -of ``__slots__`` for many internal objects. This optimization is -particularly geared towards the base memory size of large applications -that have lots of tables and columns, and reduces memory -size for a variety of high-volume objects including event listening -internals, comparator objects and parts of the ORM attribute and -loader strategy system. - -A bench that makes use of heapy measure the startup size of Nova -illustrates a difference of about 3.7 fewer megs, or 46%, -taken up by SQLAlchemy's objects, associated dictionaries, as -well as weakrefs, within a basic import of "nova.db.sqlalchemy.models": - -.. sourcecode:: text - - # reported by heapy, summation of SQLAlchemy objects + - # associated dicts + weakref-related objects with core of Nova imported: - - Before: total count 26477 total bytes 7975712 - After: total count 18181 total bytes 4236456 - - # reported for the Python module space overall with the - # core of Nova imported: - - Before: Partition of a set of 355558 objects. Total size = 61661760 bytes. - After: Partition of a set of 346034 objects. Total size = 57808016 bytes. - - -.. _feature_updatemany: - -UPDATE statements are now batched with executemany() in a flush ---------------------------------------------------------------- - -UPDATE statements can now be batched within an ORM flush -into more performant executemany() call, similarly to how INSERT -statements can be batched; this will be invoked within flush -based on the following criteria: - -* two or more UPDATE statements in sequence involve the identical set of - columns to be modified. - -* The statement has no embedded SQL expressions in the SET clause. - -* The mapping does not use a :paramref:`~.orm.mapper.version_id_col`, or - the backend dialect supports a "sane" rowcount for an executemany() - operation; most DBAPIs support this correctly now. - -.. _feature_3178: - - -.. _bug_3035: - -Session.get_bind() handles a wider variety of inheritance scenarios -------------------------------------------------------------------- - -The :meth:`.Session.get_bind` method is invoked whenever a query or unit -of work flush process seeks to locate the database engine that corresponds -to a particular class. The method has been improved to handle a variety -of inheritance-oriented scenarios, including: - -* Binding to a Mixin or Abstract Class:: - - class MyClass(SomeMixin, Base): - __tablename__ = "my_table" - # ... - - - session = Session(binds={SomeMixin: some_engine}) - -* Binding to inherited concrete subclasses individually based on table:: - - class BaseClass(Base): - __tablename__ = "base" - - # ... - - - class ConcreteSubClass(BaseClass): - __tablename__ = "concrete" - - # ... - - __mapper_args__ = {"concrete": True} - - - session = Session(binds={base_table: some_engine, concrete_table: some_other_engine}) - -:ticket:`3035` - - -.. _bug_3227: - -Session.get_bind() will receive the Mapper in all relevant Query cases ----------------------------------------------------------------------- - -A series of issues were repaired where the :meth:`.Session.get_bind` -would not receive the primary :class:`_orm.Mapper` of the :class:`_query.Query`, -even though this mapper was readily available (the primary mapper is the -single mapper, or alternatively the first mapper, that is associated with -a :class:`_query.Query` object). - -The :class:`_orm.Mapper` object, when passed to :meth:`.Session.get_bind`, -is typically used by sessions that make use of the -:paramref:`.Session.binds` parameter to associate mappers with a -series of engines (although in this use case, things frequently -"worked" in most cases anyway as the bind would be located via the -mapped table object), or more specifically implement a user-defined -:meth:`.Session.get_bind` method that provides some pattern of -selecting engines based on mappers, such as horizontal sharding or a -so-called "routing" session that routes queries to different backends. - -These scenarios include: - -* :meth:`_query.Query.count`:: - - session.query(User).count() - -* :meth:`_query.Query.update` and :meth:`_query.Query.delete`, both for the UPDATE/DELETE - statement as well as for the SELECT used by the "fetch" strategy:: - - session.query(User).filter(User.id == 15).update( - {"name": "foob"}, synchronize_session="fetch" - ) - - session.query(User).filter(User.id == 15).delete(synchronize_session="fetch") - -* Queries against individual columns:: - - session.query(User.id, User.name).all() - -* SQL functions and other expressions against indirect mappings such as - :obj:`.column_property`:: - - class User(Base): - ... - - score = column_property(func.coalesce(self.tables.users.c.name, None)) - - - session.query(func.max(User.score)).scalar() - -:ticket:`3227` :ticket:`3242` :ticket:`1326` - -.. _feature_2963: - -.info dictionary improvements ------------------------------ - -The :attr:`.InspectionAttr.info` collection is now available on every kind -of object that one would retrieve from the :attr:`_orm.Mapper.all_orm_descriptors` -collection. This includes :class:`.hybrid_property` and :func:`.association_proxy`. -However, as these objects are class-bound descriptors, they must be accessed -**separately** from the class to which they are attached in order to get -at the attribute. Below this is illustrated using the -:attr:`_orm.Mapper.all_orm_descriptors` namespace:: - - class SomeObject(Base): - # ... - - @hybrid_property - def some_prop(self): - return self.value + 5 - - - inspect(SomeObject).all_orm_descriptors.some_prop.info["foo"] = "bar" - -It is also available as a constructor argument for all :class:`.SchemaItem` -objects (e.g. :class:`_schema.ForeignKey`, :class:`.UniqueConstraint` etc.) as well -as remaining ORM constructs such as :func:`_orm.synonym`. - -:ticket:`2971` - -:ticket:`2963` - -.. _bug_3188: - -ColumnProperty constructs work a lot better with aliases, order_by ------------------------------------------------------------------- - -A variety of issues regarding :func:`.column_property` have been fixed, -most specifically with regards to the :func:`.aliased` construct as well -as the "order by label" logic introduced in 0.9 (see :ref:`migration_1068`). - -Given a mapping like the following:: - - class A(Base): - __tablename__ = "a" - - id = Column(Integer, primary_key=True) - - - class B(Base): - __tablename__ = "b" - - id = Column(Integer, primary_key=True) - a_id = Column(ForeignKey("a.id")) - - - A.b = column_property(select([func.max(B.id)]).where(B.a_id == A.id).correlate(A)) - -A simple scenario that included "A.b" twice would fail to render -correctly:: - - print(sess.query(A, a1).order_by(a1.b)) - -This would order by the wrong column: - -.. sourcecode:: sql - - SELECT a.id AS a_id, (SELECT max(b.id) AS max_1 FROM b - WHERE b.a_id = a.id) AS anon_1, a_1.id AS a_1_id, - (SELECT max(b.id) AS max_2 - FROM b WHERE b.a_id = a_1.id) AS anon_2 - FROM a, a AS a_1 ORDER BY anon_1 - -New output: - -.. sourcecode:: sql - - SELECT a.id AS a_id, (SELECT max(b.id) AS max_1 - FROM b WHERE b.a_id = a.id) AS anon_1, a_1.id AS a_1_id, - (SELECT max(b.id) AS max_2 - FROM b WHERE b.a_id = a_1.id) AS anon_2 - FROM a, a AS a_1 ORDER BY anon_2 - -There were also many scenarios where the "order by" logic would fail -to order by label, for example if the mapping were "polymorphic":: - - class A(Base): - __tablename__ = "a" - - id = Column(Integer, primary_key=True) - type = Column(String) - - __mapper_args__ = {"polymorphic_on": type, "with_polymorphic": "*"} - -The order_by would fail to use the label, as it would be anonymized due -to the polymorphic loading: - -.. sourcecode:: sql - - SELECT a.id AS a_id, a.type AS a_type, (SELECT max(b.id) AS max_1 - FROM b WHERE b.a_id = a.id) AS anon_1 - FROM a ORDER BY (SELECT max(b.id) AS max_2 - FROM b WHERE b.a_id = a.id) - -Now that the order by label tracks the anonymized label, this now works: - -.. sourcecode:: sql - - SELECT a.id AS a_id, a.type AS a_type, (SELECT max(b.id) AS max_1 - FROM b WHERE b.a_id = a.id) AS anon_1 - FROM a ORDER BY anon_1 - -Included in these fixes are a variety of heisenbugs that could corrupt -the state of an ``aliased()`` construct such that the labeling logic -would again fail; these have also been fixed. - -:ticket:`3148` :ticket:`3188` - -New Features and Improvements - Core -==================================== - -.. _feature_3034: - -Select/Query LIMIT / OFFSET may be specified as an arbitrary SQL expression ---------------------------------------------------------------------------- - -The :meth:`_expression.Select.limit` and :meth:`_expression.Select.offset` methods now accept -any SQL expression, in addition to integer values, as arguments. The ORM -:class:`_query.Query` object also passes through any expression to the underlying -:class:`_expression.Select` object. Typically -this is used to allow a bound parameter to be passed, which can be substituted -with a value later:: - - sel = select([table]).limit(bindparam("mylimit")).offset(bindparam("myoffset")) - -Dialects which don't support non-integer LIMIT or OFFSET expressions may continue -to not support this behavior; third party dialects may also need modification -in order to take advantage of the new behavior. A dialect which currently -uses the ``._limit`` or ``._offset`` attributes will continue to function -for those cases where the limit/offset was specified as a simple integer value. -However, when a SQL expression is specified, these two attributes will -instead raise a :class:`.CompileError` on access. A third-party dialect which -wishes to support the new feature should now call upon the ``._limit_clause`` -and ``._offset_clause`` attributes to receive the full SQL expression, rather -than the integer value. - -.. _feature_3282: - -The ``use_alter`` flag on ``ForeignKeyConstraint`` is (usually) no longer needed --------------------------------------------------------------------------------- - -The :meth:`_schema.MetaData.create_all` and :meth:`_schema.MetaData.drop_all` methods will -now make use of a system that automatically renders an ALTER statement -for foreign key constraints that are involved in mutually-dependent cycles -between tables, without the -need to specify :paramref:`_schema.ForeignKeyConstraint.use_alter`. Additionally, -the foreign key constraints no longer need to have a name in order to be -created via ALTER; only the DROP operation requires a name. In the case -of a DROP, the feature will ensure that only constraints which have -explicit names are actually included as ALTER statements. In the -case of an unresolvable cycle within a DROP, the system emits -a succinct and clear error message now if the DROP cannot proceed. - -The :paramref:`_schema.ForeignKeyConstraint.use_alter` and -:paramref:`_schema.ForeignKey.use_alter` flags remain in place, and continue to have -the same effect of establishing those constraints for which ALTER is -required during a CREATE/DROP scenario. - -As of version 1.0.1, special logic takes over in the case of SQLite, which -does not support ALTER, in the case that during a DROP, the given tables have -an unresolvable cycle; in this case a warning is emitted, and the tables -are dropped with **no** ordering, which is usually fine on SQLite unless -constraints are enabled. To resolve the warning and proceed with at least -a partial ordering on a SQLite database, particularly one where constraints -are enabled, re-apply "use_alter" flags to those -:class:`_schema.ForeignKey` and :class:`_schema.ForeignKeyConstraint` objects which should -be explicitly omitted from the sort. - -.. seealso:: - - :ref:`use_alter` - full description of the new behavior. - -:ticket:`3282` - -.. _change_3330: - -ResultProxy "auto close" is now a "soft" close ----------------------------------------------- - -For many releases, the :class:`_engine.ResultProxy` object has always been -automatically closed out at the point at which all result rows have been -fetched. This was to allow usage of the object without the need to call -upon :meth:`_engine.ResultProxy.close` explicitly; as all DBAPI resources had been -freed, the object was safe to discard. However, the object maintained -a strict "closed" behavior, which meant that any subsequent calls to -:meth:`_engine.ResultProxy.fetchone`, :meth:`_engine.ResultProxy.fetchmany` or -:meth:`_engine.ResultProxy.fetchall` would now raise a :class:`.ResourceClosedError`:: - - >>> result = connection.execute(stmt) - >>> result.fetchone() - (1, 'x') - >>> result.fetchone() - None # indicates no more rows - >>> result.fetchone() - exception: ResourceClosedError - -This behavior is inconsistent vs. what pep-249 states, which is -that you can call upon the fetch methods repeatedly even after results -are exhausted. It also interferes with behavior for some implementations of -result proxy, such as the :class:`.BufferedColumnResultProxy` used by the -cx_oracle dialect for certain datatypes. - -To solve this, the "closed" state of the :class:`_engine.ResultProxy` has been -broken into two states; a "soft close" which does the majority of what -"close" does, in that it releases the DBAPI cursor and in the case of a -"close with result" object will also release the connection, and a -"closed" state which is everything included by "soft close" as well as -establishing the fetch methods as "closed". The :meth:`_engine.ResultProxy.close` -method is now never called implicitly, only the :meth:`_engine.ResultProxy._soft_close` -method which is non-public:: - - >>> result = connection.execute(stmt) - >>> result.fetchone() - (1, 'x') - >>> result.fetchone() - None # indicates no more rows - >>> result.fetchone() - None # still None - >>> result.fetchall() - [] - >>> result.close() - >>> result.fetchone() - exception: ResourceClosedError # *now* it raises - -:ticket:`3330` -:ticket:`3329` - -CHECK Constraints now support the ``%(column_0_name)s`` token in naming conventions ------------------------------------------------------------------------------------ - -The ``%(column_0_name)s`` will derive from the first column found in the -expression of a :class:`.CheckConstraint`:: - - metadata = MetaData(naming_convention={"ck": "ck_%(table_name)s_%(column_0_name)s"}) - - foo = Table("foo", metadata, Column("value", Integer)) - - CheckConstraint(foo.c.value > 5) - -Will render: - -.. sourcecode:: sql - - CREATE TABLE foo ( - value INTEGER, - CONSTRAINT ck_foo_value CHECK (value > 5) - ) - -The combination of naming conventions with the constraint produced by a -:class:`.SchemaType` such as :class:`.Boolean` or :class:`.Enum` will also -now make use of all CHECK constraint conventions. - -.. seealso:: - - :ref:`naming_check_constraints` - - :ref:`naming_schematypes` - -:ticket:`3299` - -.. _change_3341: - -Constraints referring to unattached Columns can auto-attach to the Table when their referred columns are attached ------------------------------------------------------------------------------------------------------------------ - -Since at least version 0.8, a :class:`.Constraint` has had the ability to -"auto-attach" itself to a :class:`_schema.Table` based on being passed table-attached columns:: - - from sqlalchemy import Table, Column, MetaData, Integer, UniqueConstraint - - m = MetaData() - - t = Table("t", m, Column("a", Integer), Column("b", Integer)) - - uq = UniqueConstraint(t.c.a, t.c.b) # will auto-attach to Table - - assert uq in t.constraints - -In order to assist with some cases that tend to come up with declarative, -this same auto-attachment logic can now function even if the :class:`_schema.Column` -objects are not yet associated with the :class:`_schema.Table`; additional events -are established such that when those :class:`_schema.Column` objects are associated, -the :class:`.Constraint` is also added:: - - from sqlalchemy import Table, Column, MetaData, Integer, UniqueConstraint - - m = MetaData() - - a = Column("a", Integer) - b = Column("b", Integer) - - uq = UniqueConstraint(a, b) - - t = Table("t", m, a, b) - - assert uq in t.constraints # constraint auto-attached - -The above feature was a late add as of version 1.0.0b3. A fix as of -version 1.0.4 for :ticket:`3411` ensures that this logic -does not occur if the :class:`.Constraint` refers to a mixture of -:class:`_schema.Column` objects and string column names; as we do not yet have -tracking for the addition of names to a :class:`_schema.Table`:: - - from sqlalchemy import Table, Column, MetaData, Integer, UniqueConstraint - - m = MetaData() - - a = Column("a", Integer) - b = Column("b", Integer) - - uq = UniqueConstraint(a, "b") - - t = Table("t", m, a, b) - - # constraint *not* auto-attached, as we do not have tracking - # to locate when a name 'b' becomes available on the table - assert uq not in t.constraints - -Above, the attachment event for column "a" to table "t" will fire off before -column "b" is attached (as "a" is stated in the :class:`_schema.Table` constructor -before "b"), and the constraint will fail to locate "b" if it were to attempt -an attachment. For consistency, if the constraint refers to any string names, -the autoattach-on-column-attach logic is skipped. - -The original auto-attach logic of course remains in place, if the :class:`_schema.Table` -already contains all the target :class:`_schema.Column` objects at the time -the :class:`.Constraint` is constructed:: - - from sqlalchemy import Table, Column, MetaData, Integer, UniqueConstraint - - m = MetaData() - - a = Column("a", Integer) - b = Column("b", Integer) - - - t = Table("t", m, a, b) - - uq = UniqueConstraint(a, "b") - - # constraint auto-attached normally as in older versions - assert uq in t.constraints - -:ticket:`3341` -:ticket:`3411` - -.. _change_2051: - -.. _feature_insert_from_select_defaults: - -INSERT FROM SELECT now includes Python and SQL-expression defaults ------------------------------------------------------------------- - -:meth:`_expression.Insert.from_select` now includes Python and SQL-expression defaults if -otherwise unspecified; the limitation where non-server column defaults -aren't included in an INSERT FROM SELECT is now lifted and these -expressions are rendered as constants into the SELECT statement:: - - from sqlalchemy import Table, Column, MetaData, Integer, select, func - - m = MetaData() - - t = Table( - "t", m, Column("x", Integer), Column("y", Integer, default=func.somefunction()) - ) - - stmt = select([t.c.x]) - print(t.insert().from_select(["x"], stmt)) - -Will render: - -.. sourcecode:: sql - - INSERT INTO t (x, y) SELECT t.x, somefunction() AS somefunction_1 - FROM t - -The feature can be disabled using -:paramref:`.Insert.from_select.include_defaults`. - -.. _change_3087: - -Column server defaults now render literal values ------------------------------------------------- - -The "literal binds" compiler flag is switched on when a -:class:`.DefaultClause`, set up by :paramref:`_schema.Column.server_default` -is present as a SQL expression to be compiled. This allows literals -embedded in SQL to render correctly, such as:: - - from sqlalchemy import Table, Column, MetaData, Text - from sqlalchemy.schema import CreateTable - from sqlalchemy.dialects.postgresql import ARRAY, array - from sqlalchemy.dialects import postgresql - - metadata = MetaData() - - tbl = Table( - "derp", - metadata, - Column("arr", ARRAY(Text), server_default=array(["foo", "bar", "baz"])), - ) - - print(CreateTable(tbl).compile(dialect=postgresql.dialect())) - -Now renders: - -.. sourcecode:: sql - - CREATE TABLE derp ( - arr TEXT[] DEFAULT ARRAY['foo', 'bar', 'baz'] - ) - -Previously, the literal values ``"foo", "bar", "baz"`` would render as -bound parameters, which are useless in DDL. - -:ticket:`3087` - -.. _feature_3184: - -UniqueConstraint is now part of the Table reflection process ------------------------------------------------------------- - -A :class:`_schema.Table` object populated using ``autoload=True`` will now -include :class:`.UniqueConstraint` constructs as well as -:class:`.Index` constructs. This logic has a few caveats for -PostgreSQL and MySQL: - -PostgreSQL -^^^^^^^^^^ - -PostgreSQL has the behavior such that when a UNIQUE constraint is -created, it implicitly creates a UNIQUE INDEX corresponding to that -constraint as well. The :meth:`_reflection.Inspector.get_indexes` and the -:meth:`_reflection.Inspector.get_unique_constraints` methods will continue to -**both** return these entries distinctly, where -:meth:`_reflection.Inspector.get_indexes` now features a token -``duplicates_constraint`` within the index entry indicating the -corresponding constraint when detected. However, when performing -full table reflection using ``Table(..., autoload=True)``, the -:class:`.Index` construct is detected as being linked to the -:class:`.UniqueConstraint`, and is **not** present within the -:attr:`_schema.Table.indexes` collection; only the :class:`.UniqueConstraint` -will be present in the :attr:`_schema.Table.constraints` collection. This -deduplication logic works by joining to the ``pg_constraint`` table -when querying ``pg_index`` to see if the two constructs are linked. - -MySQL -^^^^^ - -MySQL does not have separate concepts for a UNIQUE INDEX and a UNIQUE -constraint. While it supports both syntaxes when creating tables and indexes, -it does not store them any differently. The -:meth:`_reflection.Inspector.get_indexes` -and the :meth:`_reflection.Inspector.get_unique_constraints` methods will continue to -**both** return an entry for a UNIQUE index in MySQL, -where :meth:`_reflection.Inspector.get_unique_constraints` features a new token -``duplicates_index`` within the constraint entry indicating that this is a -dupe entry corresponding to that index. However, when performing -full table reflection using ``Table(..., autoload=True)``, -the :class:`.UniqueConstraint` construct is -**not** part of the fully reflected :class:`_schema.Table` construct under any -circumstances; this construct is always represented by a :class:`.Index` -with the ``unique=True`` setting present in the :attr:`_schema.Table.indexes` -collection. - -.. seealso:: - - :ref:`postgresql_index_reflection` - - :ref:`mysql_unique_constraints` - -:ticket:`3184` - - -New systems to safely emit parameterized warnings -------------------------------------------------- - -For a long time, there has been a restriction that warning messages could not -refer to data elements, such that a particular function might emit an -infinite number of unique warnings. The key place this occurs is in the -``Unicode type received non-unicode bind param value`` warning. Placing -the data value in this message would mean that the Python ``__warningregistry__`` -for that module, or in some cases the Python-global ``warnings.onceregistry``, -would grow unbounded, as in most warning scenarios, one of these two collections -is populated with every distinct warning message. - -The change here is that by using a special ``string`` type that purposely -changes how the string is hashed, we can control that a large number of -parameterized messages are hashed only on a small set of possible hash -values, such that a warning such as ``Unicode type received non-unicode -bind param value`` can be tailored to be emitted only a specific number -of times; beyond that, the Python warnings registry will begin recording -them as duplicates. - -To illustrate, the following test script will show only ten warnings being -emitted for ten of the parameter sets, out of a total of 1000:: - - from sqlalchemy import create_engine, Unicode, select, cast - import random - import warnings - - e = create_engine("sqlite://") - - # Use the "once" filter (which is also the default for Python - # warnings). Exactly ten of these warnings will - # be emitted; beyond that, the Python warnings registry will accumulate - # new values as dupes of one of the ten existing. - warnings.filterwarnings("once") - - for i in range(1000): - e.execute( - select([cast(("foo_%d" % random.randint(0, 1000000)).encode("ascii"), Unicode)]) - ) - -The format of the warning here is: - -.. sourcecode:: text - - /path/lib/sqlalchemy/sql/sqltypes.py:186: SAWarning: Unicode type received - non-unicode bind param value 'foo_4852'. (this warning may be - suppressed after 10 occurrences) - - -:ticket:`3178` - -Key Behavioral Changes - ORM -============================ - -.. _bug_3228: - -query.update() now resolves string names into mapped attribute names --------------------------------------------------------------------- - -The documentation for :meth:`_query.Query.update` states that the given -``values`` dictionary is "a dictionary with attributes names as keys", -implying that these are mapped attribute names. Unfortunately, the function -was designed more in mind to receive attributes and SQL expressions and -not as much strings; when strings -were passed, these strings would be passed through straight to the core -update statement without any resolution as far as how these names are -represented on the mapped class, meaning the name would have to match that -of a table column exactly, not how an attribute of that name was mapped -onto the class. - -The string names are now resolved as attribute names in earnest:: - - class User(Base): - __tablename__ = "user" - - id = Column(Integer, primary_key=True) - name = Column("user_name", String(50)) - -Above, the column ``user_name`` is mapped as ``name``. Previously, -a call to :meth:`_query.Query.update` that was passed strings would have to -have been called as follows:: - - session.query(User).update({"user_name": "moonbeam"}) - -The given string is now resolved against the entity:: - - session.query(User).update({"name": "moonbeam"}) - -It is typically preferable to use the attribute directly, to avoid any -ambiguity:: - - session.query(User).update({User.name: "moonbeam"}) - -The change also indicates that synonyms and hybrid attributes can be referred -to by string name as well:: - - class User(Base): - __tablename__ = "user" - - id = Column(Integer, primary_key=True) - name = Column("user_name", String(50)) - - @hybrid_property - def fullname(self): - return self.name - - - session.query(User).update({"fullname": "moonbeam"}) - -:ticket:`3228` - -.. _bug_3371: - -Warnings emitted when comparing objects with None values to relationships -------------------------------------------------------------------------- - -This change is new as of 1.0.1. Some users are performing -queries that are essentially of this form:: - - session.query(Address).filter(Address.user == User(id=None)) - -This pattern is not currently supported in SQLAlchemy. For all versions, -it emits SQL resembling: - -.. sourcecode:: sql - - SELECT address.id AS address_id, address.user_id AS address_user_id, - address.email_address AS address_email_address - FROM address WHERE ? = address.user_id - (None,) - -Note above, there is a comparison ``WHERE ? = address.user_id`` where the -bound value ``?`` is receiving ``None``, or ``NULL`` in SQL. **This will -always return False in SQL**. The comparison here would in theory -generate SQL as follows: - -.. sourcecode:: sql - - SELECT address.id AS address_id, address.user_id AS address_user_id, - address.email_address AS address_email_address - FROM address WHERE address.user_id IS NULL - -But right now, **it does not**. Applications which are relying upon the -fact that "NULL = NULL" produces False in all cases run the risk that -someday, SQLAlchemy might fix this issue to generate "IS NULL", and the queries -will then produce different results. Therefore with this kind of operation, -you will see a warning: - -.. sourcecode:: text - - SAWarning: Got None for value of column user.id; this is unsupported - for a relationship comparison and will not currently produce an - IS comparison (but may in a future release) - -Note that this pattern was broken in most cases for release 1.0.0 including -all of the betas; a value like ``SYMBOL('NEVER_SET')`` would be generated. -This issue has been fixed, but as a result of identifying this pattern, -the warning is now there so that we can more safely repair this broken -behavior (now captured in :ticket:`3373`) in a future release. - -:ticket:`3371` - -.. _bug_3374: - -A "negated contains or equals" relationship comparison will use the current value of attributes, not the database value -------------------------------------------------------------------------------------------------------------------------- - -This change is new as of 1.0.1; while we would have preferred for this to be in 1.0.0, -it only became apparent as a result of :ticket:`3371`. - -Given a mapping:: - - class A(Base): - __tablename__ = "a" - id = Column(Integer, primary_key=True) - - - class B(Base): - __tablename__ = "b" - id = Column(Integer, primary_key=True) - a_id = Column(ForeignKey("a.id")) - a = relationship("A") - -Given ``A``, with primary key of 7, but which we changed to be 10 -without flushing:: - - s = Session(autoflush=False) - a1 = A(id=7) - s.add(a1) - s.commit() - - a1.id = 10 - -A query against a many-to-one relationship with this object as the target -will use the value 10 in the bound parameters:: - - s.query(B).filter(B.a == a1) - -Produces: - -.. sourcecode:: sql - - SELECT b.id AS b_id, b.a_id AS b_a_id - FROM b - WHERE ? = b.a_id - (10,) - -However, before this change, the negation of this criteria would **not** use -10, it would use 7, unless the object were flushed first:: - - s.query(B).filter(B.a != a1) - -Produces (in 0.9 and all versions prior to 1.0.1): - -.. sourcecode:: sql - - SELECT b.id AS b_id, b.a_id AS b_a_id - FROM b - WHERE b.a_id != ? OR b.a_id IS NULL - (7,) - -For a transient object, it would produce a broken query: - -.. sourcecode:: sql - - SELECT b.id, b.a_id - FROM b - WHERE b.a_id != :a_id_1 OR b.a_id IS NULL - -- {u'a_id_1': symbol('NEVER_SET')} - -This inconsistency has been repaired, and in all queries the current attribute -value, in this example ``10``, will now be used. - -:ticket:`3374` - -.. _migration_3061: - -Changes to attribute events and other operations regarding attributes that have no pre-existing value ------------------------------------------------------------------------------------------------------- - -In this change, the default return value of ``None`` when accessing an object -is now returned dynamically on each access, rather than implicitly setting the -attribute's state with a special "set" operation when it is first accessed. -The visible result of this change is that ``obj.__dict__`` is not implicitly -modified on get, and there are also some minor behavioral changes -for :func:`.attributes.get_history` and related functions. - -Given an object with no state:: - - >>> obj = Foo() - -It has always been SQLAlchemy's behavior such that if we access a scalar -or many-to-one attribute that was never set, it is returned as ``None``:: - - >>> obj.someattr - None - -This value of ``None`` is in fact now part of the state of ``obj``, and is -not unlike as though we had set the attribute explicitly, e.g. -``obj.someattr = None``. However, the "set on get" here would behave -differently as far as history and events. It would not emit any attribute -event, and additionally if we view history, we see this:: - - >>> inspect(obj).attrs.someattr.history - History(added=(), unchanged=[None], deleted=()) # 0.9 and below - -That is, it's as though the attribute were always ``None`` and were -never changed. This is explicitly different from if we had set the -attribute first instead:: - - >>> obj = Foo() - >>> obj.someattr = None - >>> inspect(obj).attrs.someattr.history - History(added=[None], unchanged=(), deleted=()) # all versions - -The above means that the behavior of our "set" operation can be corrupted -by the fact that the value was accessed via "get" earlier. In 1.0, this -inconsistency has been resolved, by no longer actually setting anything -when the default "getter" is used. - - >>> obj = Foo() - >>> obj.someattr - None - >>> inspect(obj).attrs.someattr.history - History(added=(), unchanged=(), deleted=()) # 1.0 - >>> obj.someattr = None - >>> inspect(obj).attrs.someattr.history - History(added=[None], unchanged=(), deleted=()) - -The reason the above behavior hasn't had much impact is because the -INSERT statement in relational databases considers a missing value to be -the same as NULL in most cases. Whether SQLAlchemy received a history -event for a particular attribute set to None or not would usually not matter; -as the difference between sending None/NULL or not wouldn't have an impact. -However, as :ticket:`3060` (described here in :ref:`migration_3060`) -illustrates, there are some seldom edge cases -where we do in fact want to positively have ``None`` set. Also, allowing -the attribute event here means it's now possible to create "default value" -functions for ORM mapped attributes. - -As part of this change, the generation of the implicit "None" is now disabled -for other situations where this used to occur; this includes when an -attribute set operation on a many-to-one is received; previously, the "old" value -would be "None" if it had been not set otherwise; it now will send the -value :data:`.orm.attributes.NEVER_SET`, which is a value that may be sent -to an attribute listener now. This symbol may also be received when -calling on mapper utility functions such as :meth:`_orm.Mapper.primary_key_from_instance`; -if the primary key attributes have no setting at all, whereas the value -would be ``None`` before, it will now be the :data:`.orm.attributes.NEVER_SET` -symbol, and no change to the object's state occurs. - -:ticket:`3061` - -.. _migration_3060: - -Priority of attribute changes on relationship-bound attributes vs. FK-bound may appear to change ------------------------------------------------------------------------------------------------- - -As a side effect of :ticket:`3060`, setting a relationship-bound attribute to ``None`` -is now a tracked history event which refers to the intention of persisting -``None`` to that attribute. As it has always been the case that setting a -relationship-bound attribute will trump direct assignment to the foreign key -attributes, a change in behavior can be seen here when assigning None. -Given a mapping:: - - class A(Base): - __tablename__ = "table_a" - - id = Column(Integer, primary_key=True) - - - class B(Base): - __tablename__ = "table_b" - - id = Column(Integer, primary_key=True) - a_id = Column(ForeignKey("table_a.id")) - a = relationship(A) - -In 1.0, the relationship-bound attribute takes precedence over the FK-bound -attribute in all cases, whether or not -the value we assign is a reference to an ``A`` object or is ``None``. -In 0.9, the behavior is inconsistent and -only takes effect if a value is assigned; the None is not considered:: - - a1 = A(id=1) - a2 = A(id=2) - session.add_all([a1, a2]) - session.flush() - - b1 = B() - b1.a = a1 # we expect a_id to be '1'; takes precedence in 0.9 and 1.0 - - b2 = B() - b2.a = None # we expect a_id to be None; takes precedence only in 1.0 - - b1.a_id = 2 - b2.a_id = 2 - - session.add_all([b1, b2]) - session.commit() - - assert b1.a is a1 # passes in both 0.9 and 1.0 - assert b2.a is None # passes in 1.0, in 0.9 it's a2 - -:ticket:`3060` - -.. _bug_3139: - -session.expunge() will fully detach an object that's been deleted ------------------------------------------------------------------ - -The behavior of :meth:`.Session.expunge` had a bug that caused an -inconsistency in behavior regarding deleted objects. The -:func:`.object_session` function as well as the :attr:`.InstanceState.session` -attribute would still report object as belonging to the :class:`.Session` -subsequent to the expunge:: - - u1 = sess.query(User).first() - sess.delete(u1) - - sess.flush() - - assert u1 not in sess - assert inspect(u1).session is sess # this is normal before commit - - sess.expunge(u1) - - assert u1 not in sess - assert inspect(u1).session is None # would fail - -Note that it is normal for ``u1 not in sess`` to be True while -``inspect(u1).session`` still refers to the session, while the transaction -is ongoing subsequent to the delete operation and :meth:`.Session.expunge` -has not been called; the full detachment normally completes once the -transaction is committed. This issue would also impact functions -that rely on :meth:`.Session.expunge` such as :func:`.make_transient`. - -:ticket:`3139` - -.. _migration_yield_per_eager_loading: - -Joined/Subquery eager loading explicitly disallowed with yield_per ------------------------------------------------------------------- - -In order to make the :meth:`_query.Query.yield_per` method easier to use, -an exception is raised if any subquery eager loaders, or joined -eager loaders that would use collections, are -to take effect when yield_per is used, as these are currently not compatible -with yield-per (subquery loading could be in theory, however). -When this error is raised, the :func:`.lazyload` option can be sent with -an asterisk:: - - q = sess.query(Object).options(lazyload("*")).yield_per(100) - -or use :meth:`_query.Query.enable_eagerloads`:: - - q = sess.query(Object).enable_eagerloads(False).yield_per(100) - -The :func:`.lazyload` option has the advantage that additional many-to-one -joined loader options can still be used:: - - q = ( - sess.query(Object) - .options(lazyload("*"), joinedload("some_manytoone")) - .yield_per(100) - ) - -.. _bug_3233: - -Changes and fixes in handling of duplicate join targets -------------------------------------------------------- - -Changes here encompass bugs where an unexpected and inconsistent -behavior would occur in some scenarios when joining to an entity -twice, or to multiple single-table entities against the same table, -without using a relationship-based ON clause, as well as when joining -multiple times to the same target relationship. - -Starting with a mapping as:: - - from sqlalchemy import Integer, Column, String, ForeignKey - from sqlalchemy.orm import Session, relationship - from sqlalchemy.ext.declarative import declarative_base - - Base = declarative_base() - - - class A(Base): - __tablename__ = "a" - id = Column(Integer, primary_key=True) - bs = relationship("B") - - - class B(Base): - __tablename__ = "b" - id = Column(Integer, primary_key=True) - a_id = Column(ForeignKey("a.id")) - -A query that joins to ``A.bs`` twice:: - - print(s.query(A).join(A.bs).join(A.bs)) - -Will render: - -.. sourcecode:: sql - - SELECT a.id AS a_id - FROM a JOIN b ON a.id = b.a_id - -The query deduplicates the redundant ``A.bs`` because it is attempting -to support a case like the following:: - - s.query(A).join(A.bs).filter(B.foo == "bar").reset_joinpoint().join(A.bs, B.cs).filter( - C.bar == "bat" - ) - -That is, the ``A.bs`` is part of a "path". As part of :ticket:`3367`, -arriving at the same endpoint twice without it being part of a -larger path will now emit a warning: - -.. sourcecode:: text - - SAWarning: Pathed join target A.bs has already been joined to; skipping - -The bigger change involves when joining to an entity without using a -relationship-bound path. If we join to ``B`` twice:: - - print(s.query(A).join(B, B.a_id == A.id).join(B, B.a_id == A.id)) - -In 0.9, this would render as follows: - -.. sourcecode:: sql - - SELECT a.id AS a_id - FROM a JOIN b ON b.a_id = a.id JOIN b AS b_1 ON b_1.a_id = a.id - -This is problematic since the aliasing is implicit and in the case of different -ON clauses can lead to unpredictable results. - -In 1.0, no automatic aliasing is applied and we get: - -.. sourcecode:: sql - - SELECT a.id AS a_id - FROM a JOIN b ON b.a_id = a.id JOIN b ON b.a_id = a.id - -This will raise an error from the database. While it might be nice if -the "duplicate join target" acted identically if we joined both from -redundant relationships vs. redundant non-relationship based targets, -for now we are only changing the behavior in the more serious case where -implicit aliasing would have occurred previously, and only emitting a warning -in the relationship case. Ultimately, joining to the same thing twice without -any aliasing to disambiguate should raise an error in all cases. - -The change also has an impact on single-table inheritance targets. Using -a mapping as follows:: - - from sqlalchemy import Integer, Column, String, ForeignKey - from sqlalchemy.orm import Session, relationship - from sqlalchemy.ext.declarative import declarative_base - - Base = declarative_base() - - - class A(Base): - __tablename__ = "a" - - id = Column(Integer, primary_key=True) - type = Column(String) - - __mapper_args__ = {"polymorphic_on": type, "polymorphic_identity": "a"} - - - class ASub1(A): - __mapper_args__ = {"polymorphic_identity": "asub1"} - - - class ASub2(A): - __mapper_args__ = {"polymorphic_identity": "asub2"} - - - class B(Base): - __tablename__ = "b" - - id = Column(Integer, primary_key=True) - - a_id = Column(Integer, ForeignKey("a.id")) - - a = relationship("A", primaryjoin="B.a_id == A.id", backref="b") - - - s = Session() - - print(s.query(ASub1).join(B, ASub1.b).join(ASub2, B.a)) - - print(s.query(ASub1).join(B, ASub1.b).join(ASub2, ASub2.id == B.a_id)) - -The two queries at the bottom are equivalent, and should both render -the identical SQL: - -.. sourcecode:: sql - - SELECT a.id AS a_id, a.type AS a_type - FROM a JOIN b ON b.a_id = a.id JOIN a ON b.a_id = a.id AND a.type IN (:type_1) - WHERE a.type IN (:type_2) - -The above SQL is invalid, as it renders "a" within the FROM list twice. -However, the implicit aliasing bug would occur with the second query only -and render this instead: - -.. sourcecode:: sql - - SELECT a.id AS a_id, a.type AS a_type - FROM a JOIN b ON b.a_id = a.id JOIN a AS a_1 - ON a_1.id = b.a_id AND a_1.type IN (:type_1) - WHERE a_1.type IN (:type_2) - -Where above, the second join to "a" is aliased. While this seems convenient, -it's not how single-inheritance queries work in general and is misleading -and inconsistent. - -The net effect is that applications which were relying on this bug will now -have an error raised by the database. The solution is to use the expected -form. When referring to multiple subclasses of a single-inheritance -entity in a query, you must manually use aliases to disambiguate the table, -as all the subclasses normally refer to the same table:: - - asub2_alias = aliased(ASub2) - - print(s.query(ASub1).join(B, ASub1.b).join(asub2_alias, B.a.of_type(asub2_alias))) - -:ticket:`3233` -:ticket:`3367` - - -Deferred Columns No Longer Implicitly Undefer ---------------------------------------------- - -Mapped attributes marked as deferred without explicit undeferral -will now remain "deferred" even if their column is otherwise -present in the result set in some way. This is a performance -enhancement in that an ORM load no longer spends time searching -for each deferred column when the result set is obtained. However, -for an application that has been relying upon this, an explicit -:func:`.undefer` or similar option should now be used, in order -to prevent a SELECT from being emitted when the attribute is accessed. - - -.. _migration_deprecated_orm_events: - -Deprecated ORM Event Hooks Removed ----------------------------------- - -The following ORM event hooks, some of which have been deprecated since -0.5, have been removed: ``translate_row``, ``populate_instance``, -``append_result``, ``create_instance``. The use cases for these hooks -originated in the very early 0.1 / 0.2 series of SQLAlchemy and have long -since been unnecessary. In particular, the hooks were largely unusable -as the behavioral contracts within these events was strongly linked to -the surrounding internals, such as how an instance needs to be created -and initialized as well as how columns are located within an ORM-generated -row. The removal of these hooks greatly simplifies the mechanics of ORM -object loading. - -.. _bundle_api_change: - -API Change for new Bundle feature when custom row loaders are used ------------------------------------------------------------------- - -The new :class:`.Bundle` object of 0.9 has a small change in API, -when the ``create_row_processor()`` method is overridden on a custom class. -Previously, the sample code looked like:: - - from sqlalchemy.orm import Bundle - - - class DictBundle(Bundle): - def create_row_processor(self, query, procs, labels): - """Override create_row_processor to return values as dictionaries""" - - def proc(row, result): - return dict(zip(labels, (proc(row, result) for proc in procs))) - - return proc - -The unused ``result`` member is now removed:: - - from sqlalchemy.orm import Bundle - - - class DictBundle(Bundle): - def create_row_processor(self, query, procs, labels): - """Override create_row_processor to return values as dictionaries""" - - def proc(row): - return dict(zip(labels, (proc(row) for proc in procs))) - - return proc - -.. seealso:: - - :ref:`bundles` - -.. _migration_3008: - -Right inner join nesting now the default for joinedload with innerjoin=True ---------------------------------------------------------------------------- - -The behavior of :paramref:`_orm.joinedload.innerjoin` as well as -:paramref:`_orm.relationship.innerjoin` is now to use "nested" -inner joins, that is, right-nested, as the default behavior when an -inner join joined eager load is chained to an outer join eager load. In -order to get the old behavior of chaining all joined eager loads as -outer join when an outer join is present, use ``innerjoin="unnested"``. - -As introduced in :ref:`feature_2976` from version 0.9, the behavior of -``innerjoin="nested"`` is that an inner join eager load chained to an outer -join eager load will use a right-nested join. ``"nested"`` is now implied -when using ``innerjoin=True``:: - - query(User).options( - joinedload("orders", innerjoin=False).joinedload("items", innerjoin=True) - ) - -With the new default, this will render the FROM clause in the form:\ - -.. sourcecode:: text - - FROM users LEFT OUTER JOIN (orders JOIN items ON ) ON - -That is, using a right-nested join for the INNER join so that the full -result of ``users`` can be returned. The use of an INNER join is more efficient -than using an OUTER join, and allows the :paramref:`_orm.joinedload.innerjoin` -optimization parameter to take effect in all cases. - -To get the older behavior, use ``innerjoin="unnested"``:: - - query(User).options( - joinedload("orders", innerjoin=False).joinedload("items", innerjoin="unnested") - ) - -This will avoid right-nested joins and chain the joins together using all -OUTER joins despite the innerjoin directive: - -.. sourcecode:: text - - FROM users LEFT OUTER JOIN orders ON LEFT OUTER JOIN items ON - -As noted in the 0.9 notes, the only database backend that has difficulty -with right-nested joins is SQLite; SQLAlchemy as of 0.9 converts a right-nested -join into a subquery as a join target on SQLite. - -.. seealso:: - - :ref:`feature_2976` - description of the feature as introduced in 0.9.4. - -:ticket:`3008` - -.. _change_3249: - -Subqueries no longer applied to uselist=False joined eager loads ----------------------------------------------------------------- - -Given a joined eager load like the following:: - - class A(Base): - __tablename__ = "a" - id = Column(Integer, primary_key=True) - b = relationship("B", uselist=False) - - - class B(Base): - __tablename__ = "b" - id = Column(Integer, primary_key=True) - a_id = Column(ForeignKey("a.id")) - - - s = Session() - print(s.query(A).options(joinedload(A.b)).limit(5)) - -SQLAlchemy considers the relationship ``A.b`` to be a "one to many, -loaded as a single value", which is essentially a "one to one" -relationship. However, joined eager loading has always treated the -above as a situation where the main query needs to be inside a -subquery, as would normally be needed for a collection of B objects -where the main query has a LIMIT applied: - -.. sourcecode:: sql - - SELECT anon_1.a_id AS anon_1_a_id, b_1.id AS b_1_id, b_1.a_id AS b_1_a_id - FROM (SELECT a.id AS a_id - FROM a LIMIT :param_1) AS anon_1 - LEFT OUTER JOIN b AS b_1 ON anon_1.a_id = b_1.a_id - -However, since the relationship of the inner query to the outer one is -that at most only one row is shared in the case of ``uselist=False`` -(in the same way as a many-to-one), the "subquery" used with LIMIT + -joined eager loading is now dropped in this case: - -.. sourcecode:: sql - - SELECT a.id AS a_id, b_1.id AS b_1_id, b_1.a_id AS b_1_a_id - FROM a LEFT OUTER JOIN b AS b_1 ON a.id = b_1.a_id - LIMIT :param_1 - -In the case that the LEFT OUTER JOIN returns more than one row, the ORM -has always emitted a warning here and ignored additional results for -``uselist=False``, so the results in that error situation should not change. - -:ticket:`3249` - - -query.update() / query.delete() raises if used with join(), select_from(), from_self() --------------------------------------------------------------------------------------- - -A warning is emitted in SQLAlchemy 0.9.10 (not yet released as of -June 9, 2015) when the :meth:`_query.Query.update` or :meth:`_query.Query.delete` methods -are invoked against a query which has also called upon :meth:`_query.Query.join`, -:meth:`_query.Query.outerjoin`, -:meth:`_query.Query.select_from` or :meth:`_query.Query.from_self`. These are unsupported -use cases which silently fail in the 0.9 series up until 0.9.10 where it emits -a warning. In 1.0, these cases raise an exception. - -:ticket:`3349` - - -query.update() with ``synchronize_session='evaluate'`` raises on multi-table update ------------------------------------------------------------------------------------ - -The "evaluator" for :meth:`_query.Query.update` won't work with multi-table -updates, and needs to be set to ``synchronize_session=False`` or -``synchronize_session='fetch'`` when multiple tables are present. -The new behavior is that an explicit exception is now raised, with a message -to change the synchronize setting. -This is upgraded from a warning emitted as of 0.9.7. - -:ticket:`3117` - -Resurrect Event has been Removed --------------------------------- - -The "resurrect" ORM event has been removed entirely. This event ceased to -have any function since version 0.8 removed the older "mutable" system -from the unit of work. - - -.. _migration_3177: - -Change to single-table-inheritance criteria when using from_self(), count() ---------------------------------------------------------------------------- - -Given a single-table inheritance mapping, such as:: - - class Widget(Base): - __table__ = "widget_table" - - - class FooWidget(Widget): - pass - -Using :meth:`_query.Query.from_self` or :meth:`_query.Query.count` against a subclass -would produce a subquery, but then add the "WHERE" criteria for subtypes -to the outside:: - - sess.query(FooWidget).from_self().all() - -rendering: - -.. sourcecode:: sql - - SELECT - anon_1.widgets_id AS anon_1_widgets_id, - anon_1.widgets_type AS anon_1_widgets_type - FROM (SELECT widgets.id AS widgets_id, widgets.type AS widgets_type, - FROM widgets) AS anon_1 - WHERE anon_1.widgets_type IN (?) - -The issue with this is that if the inner query does not specify all -columns, then we can't add the WHERE clause on the outside (it actually tries, -and produces a bad query). This decision -apparently goes way back to 0.6.5 with the note "may need to make more -adjustments to this". Well, those adjustments have arrived! So now the -above query will render: - -.. sourcecode:: sql - - SELECT - anon_1.widgets_id AS anon_1_widgets_id, - anon_1.widgets_type AS anon_1_widgets_type - FROM (SELECT widgets.id AS widgets_id, widgets.type AS widgets_type, - FROM widgets - WHERE widgets.type IN (?)) AS anon_1 - -So that queries that don't include "type" will still work!:: - - sess.query(FooWidget.id).count() - -Renders: - -.. sourcecode:: sql - - SELECT count(*) AS count_1 - FROM (SELECT widgets.id AS widgets_id - FROM widgets - WHERE widgets.type IN (?)) AS anon_1 - - -:ticket:`3177` - - -.. _migration_3222: - - -single-table-inheritance criteria added to all ON clauses unconditionally -------------------------------------------------------------------------- - -When joining to a single-table inheritance subclass target, the ORM always adds -the "single table criteria" when joining on a relationship. Given a -mapping as:: - - class Widget(Base): - __tablename__ = "widget" - id = Column(Integer, primary_key=True) - type = Column(String) - related_id = Column(ForeignKey("related.id")) - related = relationship("Related", backref="widget") - __mapper_args__ = {"polymorphic_on": type} - - - class FooWidget(Widget): - __mapper_args__ = {"polymorphic_identity": "foo"} - - - class Related(Base): - __tablename__ = "related" - id = Column(Integer, primary_key=True) - -It's been the behavior for quite some time that a JOIN on the relationship -will render a "single inheritance" clause for the type:: - - s.query(Related).join(FooWidget, Related.widget).all() - -SQL output: - -.. sourcecode:: sql - - SELECT related.id AS related_id - FROM related JOIN widget ON related.id = widget.related_id AND widget.type IN (:type_1) - -Above, because we joined to a subclass ``FooWidget``, :meth:`_query.Query.join` -knew to add the ``AND widget.type IN ('foo')`` criteria to the ON clause. - -The change here is that the ``AND widget.type IN()`` criteria is now appended -to *any* ON clause, not just those generated from a relationship, -including one that is explicitly stated:: - - # ON clause will now render as - # related.id = widget.related_id AND widget.type IN (:type_1) - s.query(Related).join(FooWidget, FooWidget.related_id == Related.id).all() - -As well as the "implicit" join when no ON clause of any kind is stated:: - - # ON clause will now render as - # related.id = widget.related_id AND widget.type IN (:type_1) - s.query(Related).join(FooWidget).all() - -Previously, the ON clause for these would not include the single-inheritance -criteria. Applications that are already adding this criteria to work around -this will want to remove its explicit use, though it should continue to work -fine if the criteria happens to be rendered twice in the meantime. - -.. seealso:: - - :ref:`bug_3233` - -:ticket:`3222` - -Key Behavioral Changes - Core -============================= - -.. _migration_2992: - -Warnings emitted when coercing full SQL fragments into text() -------------------------------------------------------------- - -Since SQLAlchemy's inception, there has always been an emphasis on not getting -in the way of the usage of plain text. The Core and ORM expression systems -were intended to allow any number of points at which the user can just -use plain text SQL expressions, not just in the sense that you can send a -full SQL string to :meth:`_engine.Connection.execute`, but that you can send strings -with SQL expressions into many functions, such as :meth:`_expression.Select.where`, -:meth:`_query.Query.filter`, and :meth:`_expression.Select.order_by`. - -Note that by "SQL expressions" we mean a **full fragment of a SQL string**, -such as:: - - # the argument sent to where() is a full SQL expression - stmt = select([sometable]).where("somecolumn = 'value'") - -and we are **not talking about string arguments**, that is, the normal -behavior of passing string values that become parameterized:: - - # This is a normal Core expression with a string argument - - # we aren't talking about this!! - stmt = select([sometable]).where(sometable.c.somecolumn == "value") - -The Core tutorial has long featured an example of the use of this technique, -using a :func:`_expression.select` construct where virtually all components of it -are specified as straight strings. However, despite this long-standing -behavior and example, users are apparently surprised that this behavior -exists, and when asking around the community, I was unable to find any user -that was in fact *not* surprised that you can send a full string into a method -like :meth:`_query.Query.filter`. - -So the change here is to encourage the user to qualify textual strings when -composing SQL that is partially or fully composed from textual fragments. -When composing a select as below:: - - stmt = select(["a", "b"]).where("a = b").select_from("sometable") - -The statement is built up normally, with all the same coercions as before. -However, one will see the following warnings emitted: - -.. sourcecode:: text - - SAWarning: Textual column expression 'a' should be explicitly declared - with text('a'), or use column('a') for more specificity - (this warning may be suppressed after 10 occurrences) - - SAWarning: Textual column expression 'b' should be explicitly declared - with text('b'), or use column('b') for more specificity - (this warning may be suppressed after 10 occurrences) - - SAWarning: Textual SQL expression 'a = b' should be explicitly declared - as text('a = b') (this warning may be suppressed after 10 occurrences) - - SAWarning: Textual SQL FROM expression 'sometable' should be explicitly - declared as text('sometable'), or use table('sometable') for more - specificity (this warning may be suppressed after 10 occurrences) - -These warnings attempt to show exactly where the issue is by displaying -the parameters as well as where the string was received. -The warnings make use of the :ref:`feature_3178` so that parameterized warnings -can be emitted safely without running out of memory, and as always, if -one wishes the warnings to be exceptions, the -`Python Warnings Filter `_ -should be used:: - - import warnings - - warnings.simplefilter("error") # all warnings raise an exception - -Given the above warnings, our statement works just fine, but -to get rid of the warnings we would rewrite our statement as follows:: - - from sqlalchemy import select, text - - stmt = ( - select([text("a"), text("b")]).where(text("a = b")).select_from(text("sometable")) - ) - -and as the warnings suggest, we can give our statement more specificity -about the text if we use :func:`_expression.column` and :func:`.table`:: - - from sqlalchemy import select, text, column, table - - stmt = ( - select([column("a"), column("b")]) - .where(text("a = b")) - .select_from(table("sometable")) - ) - -Where note also that :func:`.table` and :func:`_expression.column` can now -be imported from "sqlalchemy" without the "sql" part. - -The behavior here applies to :func:`_expression.select` as well as to key methods -on :class:`_query.Query`, including :meth:`_query.Query.filter`, -:meth:`_query.Query.from_statement` and :meth:`_query.Query.having`. - -ORDER BY and GROUP BY are special cases -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -There is one case where usage of a string has special meaning, and as part -of this change we have enhanced its functionality. When we have a -:func:`_expression.select` or :class:`_query.Query` that refers to some column name or named -label, we might want to GROUP BY and/or ORDER BY known columns or labels:: - - stmt = ( - select([user.c.name, func.count(user.c.id).label("id_count")]) - .group_by("name") - .order_by("id_count") - ) - -In the above statement we expect to see "ORDER BY id_count", as opposed to a -re-statement of the function. The string argument given is actively -matched to an entry in the columns clause during compilation, so the above -statement would produce as we expect, without warnings (though note that -the ``"name"`` expression has been resolved to ``users.name``!): - -.. sourcecode:: sql - - SELECT users.name, count(users.id) AS id_count - FROM users GROUP BY users.name ORDER BY id_count - -However, if we refer to a name that cannot be located, then we get -the warning again, as below:: - - stmt = select([user.c.name, func.count(user.c.id).label("id_count")]).order_by( - "some_label" - ) - -The output does what we say, but again it warns us: - -.. sourcecode:: text - - SAWarning: Can't resolve label reference 'some_label'; converting to - text() (this warning may be suppressed after 10 occurrences) - -.. sourcecode:: sql - - SELECT users.name, count(users.id) AS id_count - FROM users ORDER BY some_label - -The above behavior applies to all those places where we might want to refer -to a so-called "label reference"; ORDER BY and GROUP BY, but also within an -OVER clause as well as a DISTINCT ON clause that refers to columns (e.g. the -PostgreSQL syntax). - -We can still specify any arbitrary expression for ORDER BY or others using -:func:`_expression.text`:: - - stmt = select([users]).order_by(text("some special expression")) - -The upshot of the whole change is that SQLAlchemy now would like us -to tell it when a string is sent that this string is explicitly -a :func:`_expression.text` construct, or a column, table, etc., and if we use it as a -label name in an order by, group by, or other expression, SQLAlchemy expects -that the string resolves to something known, else it should again -be qualified with :func:`_expression.text` or similar. - -:ticket:`2992` - -.. _bug_3288: - -Python-side defaults invoked for each row individually when using a multivalued insert --------------------------------------------------------------------------------------- - -Support for Python-side column defaults when using the multi-valued -version of :meth:`_expression.Insert.values` were essentially not implemented, and -would only work "by accident" in specific situations, when the dialect in -use was using a non-positional (e.g. named) style of bound parameter, and -when it was not necessary that a Python-side callable be invoked for each -row. - -The feature has been overhauled so that it works more similarly to -that of an "executemany" style of invocation:: - - import itertools - - counter = itertools.count(1) - t = Table( - "my_table", - metadata, - Column("id", Integer, default=lambda: next(counter)), - Column("data", String), - ) - - conn.execute( - t.insert().values( - [ - {"data": "d1"}, - {"data": "d2"}, - {"data": "d3"}, - ] - ) - ) - -The above example will invoke ``next(counter)`` for each row individually -as would be expected: - -.. sourcecode:: sql - - INSERT INTO my_table (id, data) VALUES (?, ?), (?, ?), (?, ?) - (1, 'd1', 2, 'd2', 3, 'd3') - -Previously, a positional dialect would fail as a bind would not be generated -for additional positions: - -.. sourcecode:: text - - Incorrect number of bindings supplied. The current statement uses 6, - and there are 4 supplied. - [SQL: u'INSERT INTO my_table (id, data) VALUES (?, ?), (?, ?), (?, ?)'] - [parameters: (1, 'd1', 'd2', 'd3')] - -And with a "named" dialect, the same value for "id" would be re-used in -each row (hence this change is backwards-incompatible with a system that -relied on this): - -.. sourcecode:: sql - - INSERT INTO my_table (id, data) VALUES (:id, :data_0), (:id, :data_1), (:id, :data_2) - -- {u'data_2': 'd3', u'data_1': 'd2', u'data_0': 'd1', 'id': 1} - -The system will also refuse to invoke a "server side" default as inline-rendered -SQL, since it cannot be guaranteed that a server side default is compatible -with this. If the VALUES clause renders for a specific column, then a Python-side -value is required; if an omitted value only refers to a server-side default, -an exception is raised:: - - t = Table( - "my_table", - metadata, - Column("id", Integer, primary_key=True), - Column("data", String, server_default="some default"), - ) - - conn.execute( - t.insert().values( - [ - {"data": "d1"}, - {"data": "d2"}, - {}, - ] - ) - ) - -will raise: - -.. sourcecode:: text - - sqlalchemy.exc.CompileError: INSERT value for column my_table.data is - explicitly rendered as a boundparameter in the VALUES clause; a - Python-side value or SQL expression is required - -Previously, the value "d1" would be copied into that of the third -row (but again, only with named format!): - -.. sourcecode:: sql - - INSERT INTO my_table (data) VALUES (:data_0), (:data_1), (:data_0) - -- {u'data_1': 'd2', u'data_0': 'd1'} - -:ticket:`3288` - -.. _change_3163: - -Event listeners can not be added or removed from within that event's runner ---------------------------------------------------------------------------- - -Removal of an event listener from inside that same event itself would -modify the elements of a list during iteration, which would cause -still-attached event listeners to silently fail to fire. To prevent -this while still maintaining performance, the lists have been replaced -with ``collections.deque()``, which does not allow any additions or -removals during iteration, and instead raises ``RuntimeError``. - -:ticket:`3163` - -.. _change_3169: - -The INSERT...FROM SELECT construct now implies ``inline=True`` --------------------------------------------------------------- - -Using :meth:`_expression.Insert.from_select` now implies ``inline=True`` -on :func:`_expression.insert`. This helps to fix a bug where an -INSERT...FROM SELECT construct would inadvertently be compiled -as "implicit returning" on supporting backends, which would -cause breakage in the case of an INSERT that inserts zero rows -(as implicit returning expects a row), as well as arbitrary -return data in the case of an INSERT that inserts multiple -rows (e.g. only the first row of many). -A similar change is also applied to an INSERT..VALUES -with multiple parameter sets; implicit RETURNING will no longer emit -for this statement either. As both of these constructs deal -with variable numbers of rows, the -:attr:`_engine.ResultProxy.inserted_primary_key` accessor does not -apply. Previously, there was a documentation note that one -may prefer ``inline=True`` with INSERT..FROM SELECT as some databases -don't support returning and therefore can't do "implicit" returning, -but there's no reason an INSERT...FROM SELECT needs implicit returning -in any case. Regular explicit :meth:`_expression.Insert.returning` should -be used to return variable numbers of result rows if inserted -data is needed. - -:ticket:`3169` - -.. _change_3027: - -``autoload_with`` now implies ``autoload=True`` ------------------------------------------------ - -A :class:`_schema.Table` can be set up for reflection by passing -:paramref:`_schema.Table.autoload_with` alone:: - - my_table = Table("my_table", metadata, autoload_with=some_engine) - -:ticket:`3027` - -.. _change_3266: - -DBAPI exception wrapping and handle_error() event improvements --------------------------------------------------------------- - -SQLAlchemy's wrapping of DBAPI exceptions was not taking place in the -case where a :class:`_engine.Connection` object was invalidated, and then tried -to reconnect and encountered an error; this has been resolved. - -Additionally, the recently added :meth:`_events.ConnectionEvents.handle_error` -event is now invoked for errors that occur upon initial connect, upon -reconnect, and when :func:`_sa.create_engine` is used given a custom connection -function via :paramref:`_sa.create_engine.creator`. - -The :class:`.ExceptionContext` object has a new datamember -:attr:`.ExceptionContext.engine` that will always refer to the :class:`_engine.Engine` -in use, in those cases when the :class:`_engine.Connection` object is not available -(e.g. on initial connect). - - -:ticket:`3266` - -.. _change_3243: - -ForeignKeyConstraint.columns is now a ColumnCollection ------------------------------------------------------- - -:attr:`_schema.ForeignKeyConstraint.columns` was previously a plain list -containing either strings or :class:`_schema.Column` objects, depending on -how the :class:`_schema.ForeignKeyConstraint` was constructed and whether it was -associated with a table. The collection is now a :class:`_expression.ColumnCollection`, -and is only initialized after the :class:`_schema.ForeignKeyConstraint` is -associated with a :class:`_schema.Table`. A new accessor -:attr:`_schema.ForeignKeyConstraint.column_keys` -is added to unconditionally return string keys for the local set of -columns regardless of how the object was constructed or its current -state. - - -.. _feature_3084: - -MetaData.sorted_tables accessor is "deterministic" --------------------------------------------------- - -The sorting of tables resulting from the :attr:`_schema.MetaData.sorted_tables` -accessor is "deterministic"; the ordering should be the same in all cases -regardless of Python hashing. This is done by first sorting the tables -by name before passing them to the topological algorithm, which maintains -that ordering as it iterates. - -Note that this change does **not** yet apply to the ordering applied -when emitting :meth:`_schema.MetaData.create_all` or :meth:`_schema.MetaData.drop_all`. - -:ticket:`3084` - -.. _bug_3170: - -null(), false() and true() constants are no longer singletons -------------------------------------------------------------- - -These three constants were changed to return a "singleton" value -in 0.9; unfortunately, that would lead to a query like the following -to not render as expected:: - - select([null(), null()]) - -rendering only ``SELECT NULL AS anon_1``, because the two :func:`.null` -constructs would come out as the same ``NULL`` object, and -SQLAlchemy's Core model is based on object identity in order to -determine lexical significance. The change in 0.9 had no -importance other than the desire to save on object overhead; in general, -an unnamed construct needs to stay lexically unique so that it gets -labeled uniquely. - -:ticket:`3170` - -.. _change_3204: - -SQLite/Oracle have distinct methods for temporary table/view name reporting ---------------------------------------------------------------------------- - -The :meth:`_reflection.Inspector.get_table_names` and :meth:`_reflection.Inspector.get_view_names` -methods in the case of SQLite/Oracle would also return the names of temporary -tables and views, which is not provided by any other dialect (in the case -of MySQL at least it is not even possible). This logic has been moved -out to two new methods :meth:`_reflection.Inspector.get_temp_table_names` and -:meth:`_reflection.Inspector.get_temp_view_names`. - -Note that reflection of a specific named temporary table or temporary view, -either by ``Table('name', autoload=True)`` or via methods like -:meth:`_reflection.Inspector.get_columns` continues to function for most if not all -dialects. For SQLite specifically, there is a bug fix for UNIQUE constraint -reflection from temp tables as well, which is :ticket:`3203`. - -:ticket:`3204` - -Dialect Improvements and Changes - PostgreSQL -============================================= - -.. _change_3319: - -Overhaul of ENUM type create/drop rules ---------------------------------------- - -The rules for PostgreSQL :class:`_postgresql.ENUM` have been made more strict -with regards to creating and dropping of the TYPE. - -An :class:`_postgresql.ENUM` that is created **without** being explicitly -associated with a :class:`_schema.MetaData` object will be created *and* dropped -corresponding to :meth:`_schema.Table.create` and :meth:`_schema.Table.drop`:: - - table = Table( - "sometable", metadata, Column("some_enum", ENUM("a", "b", "c", name="myenum")) - ) - - table.create(engine) # will emit CREATE TYPE and CREATE TABLE - table.drop(engine) # will emit DROP TABLE and DROP TYPE - new for 1.0 - -This means that if a second table also has an enum named 'myenum', the -above DROP operation will now fail. In order to accommodate the use case -of a common shared enumerated type, the behavior of a metadata-associated -enumeration has been enhanced. - -An :class:`_postgresql.ENUM` that is created **with** being explicitly -associated with a :class:`_schema.MetaData` object will *not* be created *or* dropped -corresponding to :meth:`_schema.Table.create` and :meth:`_schema.Table.drop`, with -the exception of :meth:`_schema.Table.create` called with the ``checkfirst=True`` -flag:: - - my_enum = ENUM("a", "b", "c", name="myenum", metadata=metadata) - - table = Table("sometable", metadata, Column("some_enum", my_enum)) - - # will fail: ENUM 'my_enum' does not exist - table.create(engine) - - # will check for enum and emit CREATE TYPE - table.create(engine, checkfirst=True) - - table.drop(engine) # will emit DROP TABLE, *not* DROP TYPE - - metadata.drop_all(engine) # will emit DROP TYPE - - metadata.create_all(engine) # will emit CREATE TYPE - -:ticket:`3319` - -New PostgreSQL Table options ----------------------------- - -Added support for PG table options TABLESPACE, ON COMMIT, -WITH(OUT) OIDS, and INHERITS, when rendering DDL via -the :class:`_schema.Table` construct. - -.. seealso:: - - :ref:`postgresql_table_options` - -:ticket:`2051` - -.. _feature_get_enums: - -New get_enums() method with PostgreSQL Dialect ----------------------------------------------- - -The :func:`_sa.inspect` method returns a :class:`.PGInspector` object in the -case of PostgreSQL, which includes a new :meth:`.PGInspector.get_enums` -method that returns information on all available ``ENUM`` types:: - - from sqlalchemy import inspect, create_engine - - engine = create_engine("postgresql+psycopg2://host/dbname") - insp = inspect(engine) - print(insp.get_enums()) - -.. seealso:: - - :meth:`.PGInspector.get_enums` - -.. _feature_2891: - -PostgreSQL Dialect reflects Materialized Views, Foreign Tables --------------------------------------------------------------- - -Changes are as follows: - -* the :class:`Table` construct with ``autoload=True`` will now match a name - that exists in the database as a materialized view or foreign table. - -* :meth:`_reflection.Inspector.get_view_names` will return plain and materialized view - names. - -* :meth:`_reflection.Inspector.get_table_names` does **not** change for PostgreSQL, it - continues to return only the names of plain tables. - -* A new method :meth:`.PGInspector.get_foreign_table_names` is added which - will return the names of tables that are specifically marked as "foreign" - in the PostgreSQL schema tables. - -The change to reflection involves adding ``'m'`` and ``'f'`` to the list -of qualifiers we use when querying ``pg_class.relkind``, but this change -is new in 1.0.0 to avoid any backwards-incompatible surprises for those -running 0.9 in production. - -:ticket:`2891` - -.. _change_3264: - -PostgreSQL ``has_table()`` now works for temporary tables ---------------------------------------------------------- - -This is a simple fix such that "has table" for temporary tables now works, -so that code like the following may proceed:: - - from sqlalchemy import * - - metadata = MetaData() - user_tmp = Table( - "user_tmp", - metadata, - Column("id", INT, primary_key=True), - Column("name", VARCHAR(50)), - prefixes=["TEMPORARY"], - ) - - e = create_engine("postgresql://scott:tiger@localhost/test", echo="debug") - with e.begin() as conn: - user_tmp.create(conn, checkfirst=True) - - # checkfirst will succeed - user_tmp.create(conn, checkfirst=True) - -The very unlikely case that this behavior will cause a non-failing application -to behave differently, is because PostgreSQL allows a non-temporary table -to silently overwrite a temporary table. So code like the following will -now act completely differently, no longer creating the real table following -the temporary table:: - - from sqlalchemy import * - - metadata = MetaData() - user_tmp = Table( - "user_tmp", - metadata, - Column("id", INT, primary_key=True), - Column("name", VARCHAR(50)), - prefixes=["TEMPORARY"], - ) - - e = create_engine("postgresql://scott:tiger@localhost/test", echo="debug") - with e.begin() as conn: - user_tmp.create(conn, checkfirst=True) - - m2 = MetaData() - user = Table( - "user_tmp", - m2, - Column("id", INT, primary_key=True), - Column("name", VARCHAR(50)), - ) - - # in 0.9, *will create* the new table, overwriting the old one. - # in 1.0, *will not create* the new table - user.create(conn, checkfirst=True) - -:ticket:`3264` - -.. _feature_gh134: - -PostgreSQL FILTER keyword -------------------------- - -The SQL standard FILTER keyword for aggregate functions is now supported -by PostgreSQL as of 9.4. SQLAlchemy allows this using -:meth:`.FunctionElement.filter`:: - - func.count(1).filter(True) - -.. seealso:: - - :meth:`.FunctionElement.filter` - - :class:`.FunctionFilter` - -PG8000 dialect supports client side encoding --------------------------------------------- - -The :paramref:`_sa.create_engine.encoding` parameter is now honored -by the pg8000 dialect, using on connect handler which -emits ``SET CLIENT_ENCODING`` matching the selected encoding. - -PG8000 native JSONB support ---------------------------- - -Support for PG8000 versions greater than 1.10.1 has been added, where -JSONB is supported natively. - - -Support for psycopg2cffi Dialect on PyPy ----------------------------------------- - -Support for the pypy psycopg2cffi dialect is added. - -.. seealso:: - - :mod:`sqlalchemy.dialects.postgresql.psycopg2cffi` - -Dialect Improvements and Changes - MySQL -======================================== - -.. _change_3155: - -MySQL TIMESTAMP Type now renders NULL / NOT NULL in all cases -------------------------------------------------------------- - -The MySQL dialect has always worked around MySQL's implicit NOT NULL -default associated with TIMESTAMP columns by emitting NULL for -such a type, if the column is set up with ``nullable=True``. However, -MySQL 5.6.6 and above features a new flag -`explicit_defaults_for_timestamp `_ which repairs MySQL's non-standard -behavior to make it behave like any other type; to accommodate this, -SQLAlchemy now emits NULL/NOT NULL unconditionally for all TIMESTAMP -columns. - -.. seealso:: - - :ref:`mysql_timestamp_null` - -:ticket:`3155` - - -.. _change_3283: - -MySQL SET Type Overhauled to support empty sets, unicode, blank value handling ------------------------------------------------------------------------------- - -The :class:`.mysql.SET` type historically not included a system of handling -blank sets and empty values separately; as different drivers had different -behaviors for treatment of empty strings and empty-string-set representations, -the SET type tried only to hedge between these behaviors, opting to treat the -empty set as ``set([''])`` as is still the current behavior for the -MySQL-Connector-Python DBAPI. -Part of the rationale here was that it was otherwise impossible to actually -store a blank string within a MySQL SET, as the driver gives us back strings -with no way to discern between ``set([''])`` and ``set()``. It was left -to the user to determine if ``set([''])`` actually meant "empty set" or not. - -The new behavior moves the use case for the blank string, which is an unusual -case that isn't even documented in MySQL's documentation, into a special -case, and the default behavior of :class:`.mysql.SET` is now: - -* to treat the empty string ``''`` as returned by MySQL-python into the empty - set ``set()``; - -* to convert the single-blank value set ``set([''])`` returned by - MySQL-Connector-Python into the empty set ``set()``; - -* To handle the case of a set type that actually wishes includes the blank - value ``''`` in its list of possible values, - a new feature (required in this use case) is implemented whereby the set - value is persisted and loaded as a bitwise integer value; the - flag :paramref:`.mysql.SET.retrieve_as_bitwise` is added in order to - enable this. - -Using the :paramref:`.mysql.SET.retrieve_as_bitwise` flag allows the set -to be persisted and retrieved with no ambiguity of values. Theoretically -this flag can be turned on in all cases, as long as the given list of -values to the type matches the ordering exactly as declared in the -database; it only makes the SQL echo output a bit more unusual. - -The default behavior of :class:`.mysql.SET` otherwise remains the same, -roundtripping values using strings. The string-based behavior now -supports unicode fully including MySQL-python with use_unicode=0. - -:ticket:`3283` - - -MySQL internal "no such table" exceptions not passed to event handlers ----------------------------------------------------------------------- - -The MySQL dialect will now disable :meth:`_events.ConnectionEvents.handle_error` -events from firing for those statements which it uses internally -to detect if a table exists or not. This is achieved using an -execution option ``skip_user_error_events`` that disables the handle -error event for the scope of that execution. In this way, user code -that rewrites exceptions doesn't need to worry about the MySQL -dialect or other dialects that occasionally need to catch -SQLAlchemy specific exceptions. - - -Changed the default value of ``raise_on_warnings`` for MySQL-Connector ----------------------------------------------------------------------- - -Changed the default value of "raise_on_warnings" to False for -MySQL-Connector. This was set at True for some reason. The "buffered" -flag unfortunately must stay at True as MySQLconnector does not allow -a cursor to be closed unless all results are fully fetched. - -:ticket:`2515` - -.. _bug_3186: - -MySQL boolean symbols "true", "false" work again ------------------------------------------------- - -0.9's overhaul of the IS/IS NOT operators as well as boolean types in -:ticket:`2682` disallowed the MySQL dialect from making use of the -"true" and "false" symbols in the context of "IS" / "IS NOT". Apparently, -even though MySQL has no "boolean" type, it supports IS / IS NOT when the -special "true" and "false" symbols are used, even though these are otherwise -synonymous with "1" and "0" (and IS/IS NOT don't work with the numerics). - -So the change here is that the MySQL dialect remains "non native boolean", -but the :func:`.true` and :func:`.false` symbols again produce the -keywords "true" and "false", so that an expression like ``column.is_(true())`` -again works on MySQL. - -:ticket:`3186` - -.. _change_3263: - -The match() operator now returns an agnostic MatchType compatible with MySQL's floating point return value ----------------------------------------------------------------------------------------------------------- - -The return type of a :meth:`.ColumnOperators.match` expression is now a new type -called :class:`.MatchType`. This is a subclass of :class:`.Boolean`, -that can be intercepted by the dialect in order to produce a different -result type at SQL execution time. - -Code like the following will now function correctly and return floating points -on MySQL:: - - >>> connection.execute( - ... select( - ... [ - ... matchtable.c.title.match("Agile Ruby Programming").label("ruby"), - ... matchtable.c.title.match("Dive Python").label("python"), - ... matchtable.c.title, - ... ] - ... ).order_by(matchtable.c.id) - ... ) - [ - (2.0, 0.0, 'Agile Web Development with Ruby On Rails'), - (0.0, 2.0, 'Dive Into Python'), - (2.0, 0.0, "Programming Matz's Ruby"), - (0.0, 0.0, 'The Definitive Guide to Django'), - (0.0, 1.0, 'Python in a Nutshell') - ] - - -:ticket:`3263` - -.. _change_2984: - -Drizzle Dialect is now an External Dialect ------------------------------------------- - -The dialect for `Drizzle `_ is now an external -dialect, available at https://bitbucket.org/zzzeek/sqlalchemy-drizzle. -This dialect was added to SQLAlchemy right before SQLAlchemy was able to -accommodate third party dialects well; going forward, all databases that aren't -within the "ubiquitous use" category are third party dialects. -The dialect's implementation hasn't changed and is still based on the -MySQL + MySQLdb dialects within SQLAlchemy. The dialect is as of yet -unreleased and in "attic" status; however it passes the majority of tests -and is generally in decent working order, if someone wants to pick up -on polishing it. - -Dialect Improvements and Changes - SQLite -========================================= - -SQLite named and unnamed UNIQUE and FOREIGN KEY constraints will inspect and reflect -------------------------------------------------------------------------------------- - -UNIQUE and FOREIGN KEY constraints are now fully reflected on -SQLite both with and without names. Previously, foreign key -names were ignored and unnamed unique constraints were skipped. In particular -this will help with Alembic's new SQLite migration features. - -To achieve this, for both foreign keys and unique constraints, the result -of PRAGMA foreign_keys, index_list, and index_info is combined with regular -expression parsing of the CREATE TABLE statement overall to form a complete -picture of the names of constraints, as well as differentiating UNIQUE -constraints that were created as UNIQUE vs. unnamed INDEXes. - -:ticket:`3244` - -:ticket:`3261` - -Dialect Improvements and Changes - SQL Server -============================================= - -.. _change_3182: - -PyODBC driver name is required with hostname-based SQL Server connections -------------------------------------------------------------------------- - -Connecting to SQL Server with PyODBC using a DSN-less connection, e.g. -with an explicit hostname, now requires a driver name - SQLAlchemy will no -longer attempt to guess a default:: - - engine = create_engine( - "mssql+pyodbc://scott:tiger@myhost:port/databasename?driver=SQL+Server+Native+Client+10.0" - ) - -SQLAlchemy's previously hardcoded default of "SQL Server" is obsolete on -Windows, and SQLAlchemy cannot be tasked with guessing the best driver -based on operation system/driver detection. Using a DSN is always preferred -when using ODBC to avoid this issue entirely. - -:ticket:`3182` - -SQL Server 2012 large text / binary types render as VARCHAR, NVARCHAR, VARBINARY --------------------------------------------------------------------------------- - -The rendering of the :class:`_expression.TextClause`, :class:`.UnicodeText`, and :class:`.LargeBinary` -types has been changed for SQL Server 2012 and greater, with options -to control the behavior completely, based on deprecation guidelines from -Microsoft. See :ref:`mssql_large_type_deprecation` for details. - -Dialect Improvements and Changes - Oracle -========================================= - -.. _change_3220: - -Improved support for CTEs in Oracle ------------------------------------ - -CTE support has been fixed up for Oracle, and there is also a new feature -:meth:`_expression.CTE.with_suffixes` that can assist with Oracle's special directives:: - - included_parts = ( - select([part.c.sub_part, part.c.part, part.c.quantity]) - .where(part.c.part == "p1") - .cte(name="included_parts", recursive=True) - .suffix_with( - "search depth first by part set ord1", - "cycle part set y_cycle to 1 default 0", - dialect="oracle", - ) - ) - -:ticket:`3220` - -New Oracle Keywords for DDL ---------------------------- - -Keywords such as COMPRESS, ON COMMIT, BITMAP: - -:ref:`oracle_table_options` - -:ref:`oracle_index_options` diff --git a/doc/build/changelog/migration_11.rst b/doc/build/changelog/migration_11.rst deleted file mode 100644 index 8a1ba3ba0e6..00000000000 --- a/doc/build/changelog/migration_11.rst +++ /dev/null @@ -1,3036 +0,0 @@ -============================= -What's New in SQLAlchemy 1.1? -============================= - -.. admonition:: About this Document - - This document describes changes between SQLAlchemy version 1.0 - and SQLAlchemy version 1.1. - -Introduction -============ - -This guide introduces what's new in SQLAlchemy version 1.1, -and also documents changes which affect users migrating -their applications from the 1.0 series of SQLAlchemy to 1.1. - -Please carefully review the sections on behavioral changes for -potentially backwards-incompatible changes in behavior. - -Platform / Installer Changes -============================ - -Setuptools is now required for install --------------------------------------- - -SQLAlchemy's ``setup.py`` file has for many years supported operation -both with Setuptools installed and without; supporting a "fallback" mode -that uses straight Distutils. As a Setuptools-less Python environment is -now unheard of, and in order to support the featureset of Setuptools -more fully, in particular to support py.test's integration with it as well -as things like "extras", ``setup.py`` now depends on Setuptools fully. - -.. seealso:: - - :ref:`installation` - -:ticket:`3489` - -Enabling / Disabling C Extension builds is only via environment variable ------------------------------------------------------------------------- - -The C Extensions build by default during install as long as it is possible. -To disable C extension builds, the ``DISABLE_SQLALCHEMY_CEXT`` environment -variable was made available as of SQLAlchemy 0.8.6 / 0.9.4. The previous -approach of using the ``--without-cextensions`` argument has been removed, -as it relies on deprecated features of setuptools. - -.. seealso:: - - :ref:`c_extensions` - -:ticket:`3500` - - -New Features and Improvements - ORM -=================================== - -.. _change_2677: - -New Session lifecycle events ----------------------------- - -The :class:`.Session` has long supported events that allow some degree -of tracking of state changes to objects, including -:meth:`.SessionEvents.before_attach`, :meth:`.SessionEvents.after_attach`, -and :meth:`.SessionEvents.before_flush`. The Session documentation also -documents major object states at :ref:`session_object_states`. However, -there has never been system of tracking objects specifically as they -pass through these transitions. Additionally, the status of "deleted" objects -has historically been murky as the objects act somewhere between -the "persistent" and "detached" states. - -To clean up this area and allow the realm of session state transition -to be fully transparent, a new series of events have been added that -are intended to cover every possible way that an object might transition -between states, and additionally the "deleted" status has been given -its own official state name within the realm of session object states. - -New State Transition Events -^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Transitions between all states of an object such as :term:`persistent`, -:term:`pending` and others can now be intercepted in terms of a -session-level event intended to cover a specific transition. -Transitions as objects move into a :class:`.Session`, move out of a -:class:`.Session`, and even all the transitions which occur when the -transaction is rolled back using :meth:`.Session.rollback` -are explicitly present in the interface of :class:`.SessionEvents`. - -In total, there are **ten new events**. A summary of these events is in a -newly written documentation section :ref:`session_lifecycle_events`. - - -New Object State "deleted" is added, deleted objects no longer "persistent" -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The :term:`persistent` state of an object in the :class:`.Session` has -always been documented as an object that has a valid database identity; -however in the case of objects that were deleted within a flush, they -have always been in a grey area where they are not really "detached" -from the :class:`.Session` yet, because they can still be restored -within a rollback, but are not really "persistent" because their database -identity has been deleted and they aren't present in the identity map. - -To resolve this grey area given the new events, a new object state -:term:`deleted` is introduced. This state exists between the "persistent" and -"detached" states. An object that is marked for deletion via -:meth:`.Session.delete` remains in the "persistent" state until a flush -proceeds; at that point, it is removed from the identity map, moves -to the "deleted" state, and the :meth:`.SessionEvents.persistent_to_deleted` -hook is invoked. If the :class:`.Session` object's transaction is rolled -back, the object is restored as persistent; the -:meth:`.SessionEvents.deleted_to_persistent` transition is called. Otherwise -if the :class:`.Session` object's transaction is committed, -the :meth:`.SessionEvents.deleted_to_detached` transition is invoked. - -Additionally, the :attr:`.InstanceState.persistent` accessor **no longer returns -True** for an object that is in the new "deleted" state; instead, the -:attr:`.InstanceState.deleted` accessor has been enhanced to reliably -report on this new state. When the object is detached, the :attr:`.InstanceState.deleted` -returns False and the :attr:`.InstanceState.detached` accessor is True -instead. To determine if an object was deleted either in the current -transaction or in a previous transaction, use the -:attr:`.InstanceState.was_deleted` accessor. - -Strong Identity Map is Deprecated -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -One of the inspirations for the new series of transition events was to enable -leak-proof tracking of objects as they move in and out of the identity map, -so that a "strong reference" may be maintained mirroring the object -moving in and out of this map. With this new capability, there is no longer -any need for the :paramref:`.Session.weak_identity_map` parameter and the -corresponding :class:`.StrongIdentityMap` object. This option has remained -in SQLAlchemy for many years as the "strong-referencing" behavior used to be -the only behavior available, and many applications were written to assume -this behavior. It has long been recommended that strong-reference tracking -of objects not be an intrinsic job of the :class:`.Session` and instead -be an application-level construct built as needed by the application; the -new event model allows even the exact behavior of the strong identity map -to be replicated. See :ref:`session_referencing_behavior` for a new -recipe illustrating how to replace the strong identity map. - -:ticket:`2677` - -.. _change_1311: - -New init_scalar() event intercepts default values at ORM level --------------------------------------------------------------- - -The ORM produces a value of ``None`` when an attribute that has not been -set is first accessed, for a non-persistent object:: - - >>> obj = MyObj() - >>> obj.some_value - None - -There's a use case for this in-Python value to correspond to that of a -Core-generated default value, even before the object is persisted. -To suit this use case a new event :meth:`.AttributeEvents.init_scalar` -is added. The new example ``active_column_defaults.py`` at -:ref:`examples_instrumentation` illustrates a sample use, so the effect -can instead be:: - - >>> obj = MyObj() - >>> obj.some_value - "my default" - -:ticket:`1311` - -.. _change_3499: - -Changes regarding "unhashable" types, impacts deduping of ORM rows ------------------------------------------------------------------- - -The :class:`_query.Query` object has a well-known behavior of "deduping" -returned rows that contain at least one ORM-mapped entity (e.g., a -full mapped object, as opposed to individual column values). The -primary purpose of this is so that the handling of entities works -smoothly in conjunction with the identity map, including to -accommodate for the duplicate entities normally represented within -joined eager loading, as well as when joins are used for the purposes -of filtering on additional columns. - -This deduplication relies upon the hashability of the elements within -the row. With the introduction of PostgreSQL's special types like -:class:`_postgresql.ARRAY`, :class:`_postgresql.HSTORE` and -:class:`_postgresql.JSON`, the experience of types within rows being -unhashable and encountering problems here is more prevalent than -it was previously. - -In fact, SQLAlchemy has since version 0.8 included a flag on datatypes that -are noted as "unhashable", however this flag was not used consistently -on built in types. As described in :ref:`change_3499_postgresql`, this -flag is now set consistently for all of PostgreSQL's "structural" types. - -The "unhashable" flag is also set on the :class:`.NullType` type, -as :class:`.NullType` is used to refer to any expression of unknown -type. - -Since :class:`.NullType` is applied to most -usages of :attr:`.func`, as :attr:`.func` doesn't actually know anything -about the function names given in most cases, **using func() will -often disable row deduping unless explicit typing is applied**. -The following examples illustrate ``func.substr()`` applied to a string -expression, and ``func.date()`` applied to a datetime expression; both -examples will return duplicate rows due to the joined eager load unless -explicit typing is applied:: - - result = ( - session.query(func.substr(A.some_thing, 0, 4), A).options(joinedload(A.bs)).all() - ) - - users = ( - session.query( - func.date(User.date_created, "start of month").label("month"), - User, - ) - .options(joinedload(User.orders)) - .all() - ) - -The above examples, in order to retain deduping, should be specified as:: - - result = ( - session.query(func.substr(A.some_thing, 0, 4, type_=String), A) - .options(joinedload(A.bs)) - .all() - ) - - users = ( - session.query( - func.date(User.date_created, "start of month", type_=DateTime).label("month"), - User, - ) - .options(joinedload(User.orders)) - .all() - ) - -Additionally, the treatment of a so-called "unhashable" type is slightly -different than its been in previous releases; internally we are using -the ``id()`` function to get a "hash value" from these structures, just -as we would any ordinary mapped object. This replaces the previous -approach which applied a counter to the object. - -:ticket:`3499` - -.. _change_3321: - -Specific checks added for passing mapped classes, instances as SQL literals ---------------------------------------------------------------------------- - -The typing system now has specific checks for passing of SQLAlchemy -"inspectable" objects in contexts where they would otherwise be handled as -literal values. Any SQLAlchemy built-in object that is legal to pass as a -SQL value (which is not already a :class:`_expression.ClauseElement` instance) -includes a method ``__clause_element__()`` which provides a -valid SQL expression for that object. For SQLAlchemy objects that -don't provide this, such as mapped classes, mappers, and mapped -instances, a more informative error message is emitted rather than -allowing the DBAPI to receive the object and fail later. An example -is illustrated below, where a string-based attribute ``User.name`` is -compared to a full instance of ``User()``, rather than against a -string value:: - - >>> some_user = User() - >>> q = s.query(User).filter(User.name == some_user) - sqlalchemy.exc.ArgumentError: Object <__main__.User object at 0x103167e90> is not legal as a SQL literal value - -The exception is now immediate when the comparison is made between -``User.name == some_user``. Previously, a comparison like the above -would produce a SQL expression that would only fail once resolved -into a DBAPI execution call; the mapped ``User`` object would -ultimately become a bound parameter that would be rejected by the -DBAPI. - -Note that in the above example, the expression fails because -``User.name`` is a string-based (e.g. column oriented) attribute. -The change does *not* impact the usual case of comparing a many-to-one -relationship attribute to an object, which is handled distinctly:: - - >>> # Address.user refers to the User mapper, so - >>> # this is of course still OK! - >>> q = s.query(Address).filter(Address.user == some_user) - - -:ticket:`3321` - -.. _feature_indexable: - -New Indexable ORM extension ---------------------------- - -The :ref:`indexable_toplevel` extension is an extension to the hybrid -attribute feature which allows the construction of attributes which -refer to specific elements of an "indexable" data type, such as an array -or JSON field:: - - class Person(Base): - __tablename__ = "person" - - id = Column(Integer, primary_key=True) - data = Column(JSON) - - name = index_property("data", "name") - -Above, the ``name`` attribute will read/write the field ``"name"`` -from the JSON column ``data``, after initializing it to an -empty dictionary:: - - >>> person = Person(name="foobar") - >>> person.name - foobar - -The extension also triggers a change event when the attribute is modified, -so that there's no need to use :class:`~.mutable.MutableDict` in order -to track this change. - -.. seealso:: - - :ref:`indexable_toplevel` - -.. _change_3250: - -New options allowing explicit persistence of NULL over a default ----------------------------------------------------------------- - -Related to the new JSON-NULL support added to PostgreSQL as part of -:ref:`change_3514`, the base :class:`.TypeEngine` class now supports -a method :meth:`.TypeEngine.evaluates_none` which allows a positive set -of the ``None`` value on an attribute to be persisted as NULL, rather than -omitting the column from the INSERT statement, which has the effect of using -the column-level default. This allows a mapper-level -configuration of the existing object-level technique of assigning -:func:`_expression.null` to the attribute. - -.. seealso:: - - :ref:`session_forcing_null` - -:ticket:`3250` - - -.. _change_3582: - -Further Fixes to single-table inheritance querying --------------------------------------------------- - -Continuing from 1.0's :ref:`migration_3177`, the :class:`_query.Query` should -no longer inappropriately add the "single inheritance" criteria when the -query is against a subquery expression such as an exists:: - - class Widget(Base): - __tablename__ = "widget" - id = Column(Integer, primary_key=True) - type = Column(String) - data = Column(String) - __mapper_args__ = {"polymorphic_on": type} - - - class FooWidget(Widget): - __mapper_args__ = {"polymorphic_identity": "foo"} - - - q = session.query(FooWidget).filter(FooWidget.data == "bar").exists() - - session.query(q).all() - -Produces: - -.. sourcecode:: sql - - SELECT EXISTS (SELECT 1 - FROM widget - WHERE widget.data = :data_1 AND widget.type IN (:type_1)) AS anon_1 - -The IN clause on the inside is appropriate, in order to limit to FooWidget -objects, however previously the IN clause would also be generated a second -time on the outside of the subquery. - -:ticket:`3582` - -.. _change_3680: - -Improved Session state when a SAVEPOINT is cancelled by the database --------------------------------------------------------------------- - -A common case with MySQL is that a SAVEPOINT is cancelled when a deadlock -occurs within the transaction. The :class:`.Session` has been modified -to deal with this failure mode slightly more gracefully, such that the -outer, non-savepoint transaction still remains usable:: - - s = Session() - s.begin_nested() - - s.add(SomeObject()) - - try: - # assume the flush fails, flush goes to rollback to the - # savepoint and that also fails - s.flush() - except Exception as err: - print("Something broke, and our SAVEPOINT vanished too") - - # this is the SAVEPOINT transaction, marked as - # DEACTIVE so the rollback() call succeeds - s.rollback() - - # this is the outermost transaction, remains ACTIVE - # so rollback() or commit() can succeed - s.rollback() - -This issue is a continuation of :ticket:`2696` where we emit a warning -so that the original error can be seen when running on Python 2, even though -the SAVEPOINT exception takes precedence. On Python 3, exceptions are chained -so both failures are reported individually. - - -:ticket:`3680` - -.. _change_3677: - -Erroneous "new instance X conflicts with persistent instance Y" flush errors fixed ----------------------------------------------------------------------------------- - -The :meth:`.Session.rollback` method is responsible for removing objects -that were INSERTed into the database, e.g. moved from pending to persistent, -within that now rolled-back transaction. Objects that make this state -change are tracked in a weak-referencing collection, and if an object is -garbage collected from that collection, the :class:`.Session` no longer worries -about it (it would otherwise not scale for operations that insert many new -objects within a transaction). However, an issue arises if the application -re-loads that same garbage-collected row within the transaction, before the -rollback occurs; if a strong reference to this object remains into the next -transaction, the fact that this object was not inserted and should be -removed would be lost, and the flush would incorrectly raise an error:: - - from sqlalchemy import Column, create_engine - from sqlalchemy.orm import Session - from sqlalchemy.ext.declarative import declarative_base - - Base = declarative_base() - - - class A(Base): - __tablename__ = "a" - id = Column(Integer, primary_key=True) - - - e = create_engine("sqlite://", echo=True) - Base.metadata.create_all(e) - - s = Session(e) - - # persist an object - s.add(A(id=1)) - s.flush() - - # rollback buffer loses reference to A - - # load it again, rollback buffer knows nothing - # about it - a1 = s.query(A).first() - - # roll back the transaction; all state is expired but the - # "a1" reference remains - s.rollback() - - # previous "a1" conflicts with the new one because we aren't - # checking that it never got committed - s.add(A(id=1)) - s.commit() - -The above program would raise: - -.. sourcecode:: text - - FlushError: New instance with identity key - (, ('u1',)) conflicts - with persistent instance - -The bug is that when the above exception is raised, the unit of work -is operating upon the original object assuming it's a live row, when in -fact the object is expired and upon testing reveals that it's gone. The -fix tests this condition now, so in the SQL log we see: - -.. sourcecode:: sql - - BEGIN (implicit) - - INSERT INTO a (id) VALUES (?) - (1,) - - SELECT a.id AS a_id FROM a LIMIT ? OFFSET ? - (1, 0) - - ROLLBACK - - BEGIN (implicit) - - SELECT a.id AS a_id FROM a WHERE a.id = ? - (1,) - - INSERT INTO a (id) VALUES (?) - (1,) - - COMMIT - -Above, the unit of work now does a SELECT for the row we're about to report -as a conflict for, sees that it doesn't exist, and proceeds normally. -The expense of this SELECT is only incurred in the case when we would have -erroneously raised an exception in any case. - - -:ticket:`3677` - -.. _change_2349: - -passive_deletes feature for joined-inheritance mappings -------------------------------------------------------- - -A joined-table inheritance mapping may now allow a DELETE to proceed -as a result of :meth:`.Session.delete`, which only emits DELETE for the -base table, and not the subclass table, allowing configured ON DELETE CASCADE -to take place for the configured foreign keys. This is configured using -the :paramref:`.orm.mapper.passive_deletes` option:: - - from sqlalchemy import Column, Integer, String, ForeignKey, create_engine - from sqlalchemy.orm import Session - from sqlalchemy.ext.declarative import declarative_base - - Base = declarative_base() - - - class A(Base): - __tablename__ = "a" - id = Column("id", Integer, primary_key=True) - type = Column(String) - - __mapper_args__ = { - "polymorphic_on": type, - "polymorphic_identity": "a", - "passive_deletes": True, - } - - - class B(A): - __tablename__ = "b" - b_table_id = Column("b_table_id", Integer, primary_key=True) - bid = Column("bid", Integer, ForeignKey("a.id", ondelete="CASCADE")) - data = Column("data", String) - - __mapper_args__ = {"polymorphic_identity": "b"} - -With the above mapping, the :paramref:`.orm.mapper.passive_deletes` option -is configured on the base mapper; it takes effect for all non-base mappers -that are descendants of the mapper with the option set. A DELETE for -an object of type ``B`` no longer needs to retrieve the primary key value -of ``b_table_id`` if unloaded, nor does it need to emit a DELETE statement -for the table itself:: - - session.delete(some_b) - session.commit() - -Will emit SQL as: - -.. sourcecode:: sql - - DELETE FROM a WHERE a.id = %(id)s - -- {'id': 1} - COMMIT - -As always, the target database must have foreign key support with -ON DELETE CASCADE enabled. - -:ticket:`2349` - -.. _change_3630: - -Same-named backrefs will not raise an error when applied to concrete inheritance subclasses -------------------------------------------------------------------------------------------- - -The following mapping has always been possible without issue:: - - class A(Base): - __tablename__ = "a" - id = Column(Integer, primary_key=True) - b = relationship("B", foreign_keys="B.a_id", backref="a") - - - class A1(A): - __tablename__ = "a1" - id = Column(Integer, primary_key=True) - b = relationship("B", foreign_keys="B.a1_id", backref="a1") - __mapper_args__ = {"concrete": True} - - - class B(Base): - __tablename__ = "b" - id = Column(Integer, primary_key=True) - - a_id = Column(ForeignKey("a.id")) - a1_id = Column(ForeignKey("a1.id")) - -Above, even though class ``A`` and class ``A1`` have a relationship -named ``b``, no conflict warning or error occurs because class ``A1`` is -marked as "concrete". - -However, if the relationships were configured the other way, an error -would occur:: - - class A(Base): - __tablename__ = "a" - id = Column(Integer, primary_key=True) - - - class A1(A): - __tablename__ = "a1" - id = Column(Integer, primary_key=True) - __mapper_args__ = {"concrete": True} - - - class B(Base): - __tablename__ = "b" - id = Column(Integer, primary_key=True) - - a_id = Column(ForeignKey("a.id")) - a1_id = Column(ForeignKey("a1.id")) - - a = relationship("A", backref="b") - a1 = relationship("A1", backref="b") - -The fix enhances the backref feature so that an error is not emitted, -as well as an additional check within the mapper logic to bypass warning -for an attribute being replaced. - -:ticket:`3630` - -.. _change_3749: - -Same-named relationships on inheriting mappers no longer warn -------------------------------------------------------------- - -When creating two mappers in an inheritance scenario, placing a relationship -on both with the same name would emit the warning -"relationship '' on mapper supersedes the same relationship -on inherited mapper ''; this can cause dependency issues during flush". -An example is as follows:: - - class A(Base): - __tablename__ = "a" - id = Column(Integer, primary_key=True) - bs = relationship("B") - - - class ASub(A): - __tablename__ = "a_sub" - id = Column(Integer, ForeignKey("a.id"), primary_key=True) - bs = relationship("B") - - - class B(Base): - __tablename__ = "b" - id = Column(Integer, primary_key=True) - a_id = Column(ForeignKey("a.id")) - -This warning dates back to the 0.4 series in 2007 and is based on a version of -the unit of work code that has since been entirely rewritten. Currently, there -is no known issue with the same-named relationships being placed on a base -class and a descendant class, so the warning is lifted. However, note that -this use case is likely not prevalent in real world use due to the warning. -While rudimentary test support is added for this use case, it is possible that -some new issue with this pattern may be identified. - -.. versionadded:: 1.1.0b3 - -:ticket:`3749` - -.. _change_3653: - -Hybrid properties and methods now propagate the docstring as well as .info --------------------------------------------------------------------------- - -A hybrid method or property will now reflect the ``__doc__`` value -present in the original docstring:: - - class A(Base): - __tablename__ = "a" - id = Column(Integer, primary_key=True) - - name = Column(String) - - @hybrid_property - def some_name(self): - """The name field""" - return self.name - -The above value of ``A.some_name.__doc__`` is now honored:: - - >>> A.some_name.__doc__ - The name field - -However, to accomplish this, the mechanics of hybrid properties necessarily -becomes more complex. Previously, the class-level accessor for a hybrid -would be a simple pass-through, that is, this test would succeed:: - - >>> assert A.name is A.some_name - -With the change, the expression returned by ``A.some_name`` is wrapped inside -of its own ``QueryableAttribute`` wrapper:: - - >>> A.some_name - - -A lot of testing went into making sure this wrapper works correctly, including -for elaborate schemes like that of the -`Custom Value Object `_ -recipe, however we'll be looking to see that no other regressions occur for -users. - -As part of this change, the :attr:`.hybrid_property.info` collection is now -also propagated from the hybrid descriptor itself, rather than from the underlying -expression. That is, accessing ``A.some_name.info`` now returns the same -dictionary that you'd get from ``inspect(A).all_orm_descriptors['some_name'].info``:: - - >>> A.some_name.info["foo"] = "bar" - >>> from sqlalchemy import inspect - >>> inspect(A).all_orm_descriptors["some_name"].info - {'foo': 'bar'} - -Note that this ``.info`` dictionary is **separate** from that of a mapped attribute -which the hybrid descriptor may be proxying directly; this is a behavioral -change from 1.0. The wrapper will still proxy other useful attributes -of a mirrored attribute such as :attr:`.QueryableAttribute.property` and -:attr:`.QueryableAttribute.class_`. - -:ticket:`3653` - -.. _change_3601: - -Session.merge resolves pending conflicts the same as persistent ---------------------------------------------------------------- - -The :meth:`.Session.merge` method will now track the identities of objects given -within a graph to maintain primary key uniqueness before emitting an INSERT. -When duplicate objects of the same identity are encountered, non-primary-key -attributes are **overwritten** as the objects are encountered, which is -essentially non-deterministic. This behavior matches that of how persistent -objects, that is objects that are already located in the database via -primary key, are already treated, so this behavior is more internally -consistent. - -Given:: - - u1 = User(id=7, name="x") - u1.orders = [ - Order(description="o1", address=Address(id=1, email_address="a")), - Order(description="o2", address=Address(id=1, email_address="b")), - Order(description="o3", address=Address(id=1, email_address="c")), - ] - - sess = Session() - sess.merge(u1) - -Above, we merge a ``User`` object with three new ``Order`` objects, each referring to -a distinct ``Address`` object, however each is given the same primary key. -The current behavior of :meth:`.Session.merge` is to look in the identity -map for this ``Address`` object, and use that as the target. If the object -is present, meaning that the database already has a row for ``Address`` with -primary key "1", we can see that the ``email_address`` field of the ``Address`` -will be overwritten three times, in this case with the values a, b and finally -c. - -However, if the ``Address`` row for primary key "1" were not present, :meth:`.Session.merge` -would instead create three separate ``Address`` instances, and we'd then get -a primary key conflict upon INSERT. The new behavior is that the proposed -primary key for these ``Address`` objects are tracked in a separate dictionary -so that we merge the state of the three proposed ``Address`` objects onto -one ``Address`` object to be inserted. - -It may have been preferable if the original case emitted some kind of warning -that conflicting data were present in a single merge-tree, however the -non-deterministic merging of values has been the behavior for many -years for the persistent case; it now matches for the pending case. A -feature that warns for conflicting values could still be feasible for both -cases but would add considerable performance overhead as each column value -would have to be compared during the merge. - - -:ticket:`3601` - -.. _change_3708: - -Fix involving many-to-one object moves with user-initiated foreign key manipulations ------------------------------------------------------------------------------------- - -A bug has been fixed involving the mechanics of replacing a many-to-one -reference to an object with another object. During the attribute operation, -the location of the object that was previously referred to now makes use of the -database-committed foreign key value, rather than the current foreign key -value. The main effect of the fix is that a backref event towards a collection -will fire off more accurately when a many-to-one change is made, even if the -foreign key attribute was manually moved to the new value beforehand. Assume a -mapping of the classes ``Parent`` and ``SomeClass``, where ``SomeClass.parent`` -refers to ``Parent`` and ``Parent.items`` refers to the collection of -``SomeClass`` objects:: - - some_object = SomeClass() - session.add(some_object) - some_object.parent_id = some_parent.id - some_object.parent = some_parent - -Above, we've made a pending object ``some_object``, manipulated its foreign key -towards ``Parent`` to refer to it, *then* we actually set up the relationship. -Before the bug fix, the backref would not have fired off:: - - # before the fix - assert some_object not in some_parent.items - -The fix now is that when we seek to locate the previous value of -``some_object.parent``, we disregard the parent id that's been manually set, -and we look for the database-committed value. In this case, it's None because -the object is pending, so the event system logs ``some_object.parent`` -as a net change:: - - # after the fix, backref fired off for some_object.parent = some_parent - assert some_object in some_parent.items - -While it is discouraged to manipulate foreign key attributes that are managed -by relationships, there is limited support for this use case. Applications -that manipulate foreign keys in order to allow loads to proceed will often make -use of the :meth:`.Session.enable_relationship_loading` and -:attr:`.RelationshipProperty.load_on_pending` features, which cause -relationships to emit lazy loads based on in-memory foreign key values that -aren't persisted. Whether or not these features are in use, this behavioral -improvement will now be apparent. - -:ticket:`3708` - -.. _change_3662: - -Improvements to the Query.correlate method with polymorphic entities --------------------------------------------------------------------- - -In recent SQLAlchemy versions, the SQL generated by many forms of -"polymorphic" queries has a more "flat" form than it used to, where -a JOIN of several tables is no longer bundled into a subquery unconditionally. -To accommodate this, the :meth:`_query.Query.correlate` method now extracts the -individual tables from such a polymorphic selectable and ensures that all -are part of the "correlate" for the subquery. Assuming the -``Person/Manager/Engineer->Company`` setup from the mapping documentation, -using with_polymorphic:: - - sess.query(Person.name).filter( - sess.query(Company.name) - .filter(Company.company_id == Person.company_id) - .correlate(Person) - .as_scalar() - == "Elbonia, Inc." - ) - -The above query now produces: - -.. sourcecode:: sql - - SELECT people.name AS people_name - FROM people - LEFT OUTER JOIN engineers ON people.person_id = engineers.person_id - LEFT OUTER JOIN managers ON people.person_id = managers.person_id - WHERE (SELECT companies.name - FROM companies - WHERE companies.company_id = people.company_id) = ? - -Before the fix, the call to ``correlate(Person)`` would inadvertently -attempt to correlate to the join of ``Person``, ``Engineer`` and ``Manager`` -as a single unit, so ``Person`` wouldn't be correlated: - -.. sourcecode:: sql - - -- old, incorrect query - SELECT people.name AS people_name - FROM people - LEFT OUTER JOIN engineers ON people.person_id = engineers.person_id - LEFT OUTER JOIN managers ON people.person_id = managers.person_id - WHERE (SELECT companies.name - FROM companies, people - WHERE companies.company_id = people.company_id) = ? - -Using correlated subqueries against polymorphic mappings still has some -unpolished edges. If for example ``Person`` is polymorphically linked -to a so-called "concrete polymorphic union" query, the above subquery -may not correctly refer to this subquery. In all cases, a way to refer -to the "polymorphic" entity fully is to create an :func:`.aliased` object -from it first:: - - # works with all SQLAlchemy versions and all types of polymorphic - # aliasing. - - paliased = aliased(Person) - sess.query(paliased.name).filter( - sess.query(Company.name) - .filter(Company.company_id == paliased.company_id) - .correlate(paliased) - .as_scalar() - == "Elbonia, Inc." - ) - -The :func:`.aliased` construct guarantees that the "polymorphic selectable" -is wrapped in a subquery. By referring to it explicitly in the correlated -subquery, the polymorphic form is correctly used. - -:ticket:`3662` - -.. _change_3081: - -Stringify of Query will consult the Session for the correct dialect -------------------------------------------------------------------- - -Calling ``str()`` on a :class:`_query.Query` object will consult the :class:`.Session` -for the correct "bind" to use, in order to render the SQL that would be -passed to the database. In particular this allows a :class:`_query.Query` that -refers to dialect-specific SQL constructs to be renderable, assuming the -:class:`_query.Query` is associated with an appropriate :class:`.Session`. -Previously, this behavior would only take effect if the :class:`_schema.MetaData` -to which the mappings were associated were itself bound to the target -:class:`_engine.Engine`. - -If neither the underlying :class:`_schema.MetaData` nor the :class:`.Session` are -associated with any bound :class:`_engine.Engine`, then the fallback to the -"default" dialect is used to generate the SQL string. - -.. seealso:: - - :ref:`change_3631` - -:ticket:`3081` - -.. _change_3431: - -Joined eager loading where the same entity is present multiple times in one row -------------------------------------------------------------------------------- - -A fix has been made to the case has been made whereby an attribute will be -loaded via joined eager loading, even if the entity was already loaded from the -row on a different "path" that doesn't include the attribute. This is a -deep use case that's hard to reproduce, but the general idea is as follows:: - - class A(Base): - __tablename__ = "a" - id = Column(Integer, primary_key=True) - b_id = Column(ForeignKey("b.id")) - c_id = Column(ForeignKey("c.id")) - - b = relationship("B") - c = relationship("C") - - - class B(Base): - __tablename__ = "b" - id = Column(Integer, primary_key=True) - c_id = Column(ForeignKey("c.id")) - - c = relationship("C") - - - class C(Base): - __tablename__ = "c" - id = Column(Integer, primary_key=True) - d_id = Column(ForeignKey("d.id")) - d = relationship("D") - - - class D(Base): - __tablename__ = "d" - id = Column(Integer, primary_key=True) - - - c_alias_1 = aliased(C) - c_alias_2 = aliased(C) - - q = s.query(A) - q = q.join(A.b).join(c_alias_1, B.c).join(c_alias_1.d) - q = q.options( - contains_eager(A.b).contains_eager(B.c, alias=c_alias_1).contains_eager(C.d) - ) - q = q.join(c_alias_2, A.c) - q = q.options(contains_eager(A.c, alias=c_alias_2)) - -The above query emits SQL like this: - -.. sourcecode:: sql - - SELECT - d.id AS d_id, - c_1.id AS c_1_id, c_1.d_id AS c_1_d_id, - b.id AS b_id, b.c_id AS b_c_id, - c_2.id AS c_2_id, c_2.d_id AS c_2_d_id, - a.id AS a_id, a.b_id AS a_b_id, a.c_id AS a_c_id - FROM - a - JOIN b ON b.id = a.b_id - JOIN c AS c_1 ON c_1.id = b.c_id - JOIN d ON d.id = c_1.d_id - JOIN c AS c_2 ON c_2.id = a.c_id - -We can see that the ``c`` table is selected from twice; once in the context -of ``A.b.c -> c_alias_1`` and another in the context of ``A.c -> c_alias_2``. -Also, we can see that it is quite possible that the ``C`` identity for a -single row is the **same** for both ``c_alias_1`` and ``c_alias_2``, meaning -two sets of columns in one row result in only one new object being added -to the identity map. - -The query options above only call for the attribute ``C.d`` to be loaded -in the context of ``c_alias_1``, and not ``c_alias_2``. So whether or not -the final ``C`` object we get in the identity map has the ``C.d`` attribute -loaded depends on how the mappings are traversed, which while not completely -random, is essentially non-deterministic. The fix is that even if the -loader for ``c_alias_1`` is processed after that of ``c_alias_2`` for a -single row where they both refer to the same identity, the ``C.d`` -element will still be loaded. Previously, the loader did not seek to -modify the load of an entity that was already loaded via a different path. -The loader that reaches the entity first has always been non-deterministic, -so this fix may be detectable as a behavioral change in some situations and -not others. - -The fix includes tests for two variants of the "multiple paths to one entity" -case, and the fix should hopefully cover all other scenarios of this nature. - -:ticket:`3431` - - -New MutableList and MutableSet helpers added to the mutation tracking extension -------------------------------------------------------------------------------- - -New helper classes :class:`.MutableList` and :class:`.MutableSet` have been -added to the :ref:`mutable_toplevel` extension, to complement the existing -:class:`.MutableDict` helper. - -:ticket:`3297` - -.. _change_3512: - -New "raise" / "raise_on_sql" loader strategies ----------------------------------------------- - -To assist with the use case of preventing unwanted lazy loads from occurring -after a series of objects are loaded, the new "lazy='raise'" and -"lazy='raise_on_sql'" strategies and -corresponding loader option :func:`_orm.raiseload` may be applied to a -relationship attribute which will cause it to raise ``InvalidRequestError`` -when a non-eagerly-loaded attribute is accessed for read. The two variants -test for either a lazy load of any variety, including those that would -only return None or retrieve from the identity map:: - - >>> from sqlalchemy.orm import raiseload - >>> a1 = s.query(A).options(raiseload(A.some_b)).first() - >>> a1.some_b - Traceback (most recent call last): - ... - sqlalchemy.exc.InvalidRequestError: 'A.some_b' is not available due to lazy='raise' - -Or a lazy load only where SQL would be emitted:: - - >>> from sqlalchemy.orm import raiseload - >>> a1 = s.query(A).options(raiseload(A.some_b, sql_only=True)).first() - >>> a1.some_b - Traceback (most recent call last): - ... - sqlalchemy.exc.InvalidRequestError: 'A.bs' is not available due to lazy='raise_on_sql' - -:ticket:`3512` - -.. _change_3394: - -Mapper.order_by is deprecated ------------------------------ - -This old parameter from the very first versions of SQLAlchemy was part of -the original design of the ORM which featured the :class:`_orm.Mapper` object -as a public-facing query structure. This role has long since been replaced -by the :class:`_query.Query` object, where we use :meth:`_query.Query.order_by` to -indicate the ordering of results in a way that works consistently for any -combination of SELECT statements, entities and SQL expressions. There are -many areas in which :paramref:`_orm.Mapper.order_by` doesn't work as expected -(or what would be expected is not clear), such as when queries are combined -into unions; these cases are not supported. - - -:ticket:`3394` - -New Features and Improvements - Core -==================================== - -.. _change_3803: - -Engines now invalidate connections, run error handlers for BaseException ------------------------------------------------------------------------- - -.. versionadded:: 1.1 this change is a late add to the 1.1 series just - prior to 1.1 final, and is not present in the 1.1 beta releases. - -The Python ``BaseException`` class is below that of ``Exception`` but is the -identifiable base for system-level exceptions such as ``KeyboardInterrupt``, -``SystemExit``, and notably the ``GreenletExit`` exception that's used by -eventlet and gevent. This exception class is now intercepted by the exception- -handling routines of :class:`_engine.Connection`, and includes handling by the -:meth:`_events.ConnectionEvents.handle_error` event. The :class:`_engine.Connection` is now -**invalidated** by default in the case of a system level exception that is not -a subclass of ``Exception``, as it is assumed an operation was interrupted and -the connection may be in an unusable state. The MySQL drivers are most -targeted by this change however the change is across all DBAPIs. - -Note that upon invalidation, the immediate DBAPI connection used by -:class:`_engine.Connection` is disposed, and the :class:`_engine.Connection`, if still -being used subsequent to the exception raise, will use a new -DBAPI connection for subsequent operations upon next use; however, the state of -any transaction in progress is lost and the appropriate ``.rollback()`` method -must be called if applicable before this re-use can proceed. - -In order to identify this change, it was straightforward to demonstrate a pymysql or -mysqlclient / MySQL-Python connection moving into a corrupted state when -these exceptions occur in the middle of the connection doing its work; -the connection would then be returned to the connection pool where subsequent -uses would fail, or even before returning to the pool would cause secondary -failures in context managers that call ``.rollback()`` upon the exception -catch. The behavior here is expected to reduce -the incidence of the MySQL error "commands out of sync", as well as the -``ResourceClosedError`` which can occur when the MySQL driver fails to -report ``cursor.description`` correctly, when running under greenlet -conditions where greenlets are killed, or where ``KeyboardInterrupt`` exceptions -are handled without exiting the program entirely. - -The behavior is distinct from the usual auto-invalidation feature, in that it -does not assume that the backend database itself has been shut down or -restarted; it does not recycle the entire connection pool as is the case -for usual DBAPI disconnect exceptions. - -This change should be a net improvement for all users with the exception -of **any application that currently intercepts ``KeyboardInterrupt`` or -``GreenletExit`` and wishes to continue working within the same transaction**. -Such an operation is theoretically possible with other DBAPIs that do not appear to be -impacted by ``KeyboardInterrupt`` such as psycopg2. For these DBAPIs, -the following workaround will disable the connection from being recycled -for specific exceptions:: - - - engine = create_engine("postgresql+psycopg2://") - - - @event.listens_for(engine, "handle_error") - def cancel_disconnect(ctx): - if isinstance(ctx.original_exception, KeyboardInterrupt): - ctx.is_disconnect = False - -:ticket:`3803` - - -.. _change_2551: - -CTE Support for INSERT, UPDATE, DELETE --------------------------------------- - -One of the most widely requested features is support for common table -expressions (CTE) that work with INSERT, UPDATE, DELETE, and is now implemented. -An INSERT/UPDATE/DELETE can both draw from a WITH clause that's stated at the -top of the SQL, as well as can be used as a CTE itself in the context of -a larger statement. - -As part of this change, an INSERT from SELECT that includes a CTE will now -render the CTE at the top of the entire statement, rather than nested -in the SELECT statement as was the case in 1.0. - -Below is an example that renders UPDATE, INSERT and SELECT all in one -statement: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy import table, column, select, literal, exists - >>> orders = table( - ... "orders", - ... column("region"), - ... column("amount"), - ... column("product"), - ... column("quantity"), - ... ) - >>> - >>> upsert = ( - ... orders.update() - ... .where(orders.c.region == "Region1") - ... .values(amount=1.0, product="Product1", quantity=1) - ... .returning(*(orders.c._all_columns)) - ... .cte("upsert") - ... ) - >>> - >>> insert = orders.insert().from_select( - ... orders.c.keys(), - ... select([literal("Region1"), literal(1.0), literal("Product1"), literal(1)]).where( - ... ~exists(upsert.select()) - ... ), - ... ) - >>> - >>> print(insert) # Note: formatting added for clarity - {printsql}WITH upsert AS - (UPDATE orders SET amount=:amount, product=:product, quantity=:quantity - WHERE orders.region = :region_1 - RETURNING orders.region, orders.amount, orders.product, orders.quantity - ) - INSERT INTO orders (region, amount, product, quantity) - SELECT - :param_1 AS anon_1, :param_2 AS anon_2, - :param_3 AS anon_3, :param_4 AS anon_4 - WHERE NOT ( - EXISTS ( - SELECT upsert.region, upsert.amount, - upsert.product, upsert.quantity - FROM upsert)) - -:ticket:`2551` - -.. _change_3049: - -Support for RANGE and ROWS specification within window functions ----------------------------------------------------------------- - -New :paramref:`.expression.over.range_` and :paramref:`.expression.over.rows` parameters allow -RANGE and ROWS expressions for window functions: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy import func - - >>> print(func.row_number().over(order_by="x", range_=(-5, 10))) - {printsql}row_number() OVER (ORDER BY x RANGE BETWEEN :param_1 PRECEDING AND :param_2 FOLLOWING){stop} - - >>> print(func.row_number().over(order_by="x", rows=(None, 0))) - {printsql}row_number() OVER (ORDER BY x ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW){stop} - - >>> print(func.row_number().over(order_by="x", range_=(-2, None))) - {printsql}row_number() OVER (ORDER BY x RANGE BETWEEN :param_1 PRECEDING AND UNBOUNDED FOLLOWING){stop} - -:paramref:`.expression.over.range_` and :paramref:`.expression.over.rows` are specified as -2-tuples and indicate negative and positive values for specific ranges, -0 for "CURRENT ROW", and None for UNBOUNDED. - -.. seealso:: - - :ref:`tutorial_window_functions` - -:ticket:`3049` - -.. _change_2857: - -Support for the SQL LATERAL keyword ------------------------------------ - -The LATERAL keyword is currently known to only be supported by PostgreSQL 9.3 -and greater, however as it is part of the SQL standard support for this keyword -is added to Core. The implementation of :meth:`_expression.Select.lateral` employs -special logic beyond just rendering the LATERAL keyword to allow for -correlation of tables that are derived from the same FROM clause as the -selectable, e.g. lateral correlation: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy import table, column, select, true - >>> people = table("people", column("people_id"), column("age"), column("name")) - >>> books = table("books", column("book_id"), column("owner_id")) - >>> subq = ( - ... select([books.c.book_id]) - ... .where(books.c.owner_id == people.c.people_id) - ... .lateral("book_subq") - ... ) - >>> print(select([people]).select_from(people.join(subq, true()))) - {printsql}SELECT people.people_id, people.age, people.name - FROM people JOIN LATERAL (SELECT books.book_id AS book_id - FROM books WHERE books.owner_id = people.people_id) - AS book_subq ON true - -.. seealso:: - - :ref:`tutorial_lateral_correlation` - - :class:`_expression.Lateral` - - :meth:`_expression.Select.lateral` - - -:ticket:`2857` - -.. _change_3718: - -Support for TABLESAMPLE ------------------------ - -The SQL standard TABLESAMPLE can be rendered using the -:meth:`_expression.FromClause.tablesample` method, which returns a :class:`_expression.TableSample` -construct similar to an alias:: - - from sqlalchemy import func - - selectable = people.tablesample(func.bernoulli(1), name="alias", seed=func.random()) - stmt = select([selectable.c.people_id]) - -Assuming ``people`` with a column ``people_id``, the above -statement would render as: - -.. sourcecode:: sql - - SELECT alias.people_id FROM - people AS alias TABLESAMPLE bernoulli(:bernoulli_1) - REPEATABLE (random()) - -:ticket:`3718` - -.. _change_3216: - -The ``.autoincrement`` directive is no longer implicitly enabled for a composite primary key column ---------------------------------------------------------------------------------------------------- - -SQLAlchemy has always had the convenience feature of enabling the backend database's -"autoincrement" feature for a single-column integer primary key; by "autoincrement" -we mean that the database column will include whatever DDL directives the -database provides in order to indicate an auto-incrementing integer identifier, -such as the SERIAL keyword on PostgreSQL or AUTO_INCREMENT on MySQL, and additionally -that the dialect will receive these generated values from the execution -of a :meth:`_schema.Table.insert` construct using techniques appropriate to that -backend. - -What's changed is that this feature no longer turns on automatically for a -*composite* primary key; previously, a table definition such as:: - - Table( - "some_table", - metadata, - Column("x", Integer, primary_key=True), - Column("y", Integer, primary_key=True), - ) - -Would have "autoincrement" semantics applied to the ``'x'`` column, only -because it's first in the list of primary key columns. In order to -disable this, one would have to turn off ``autoincrement`` on all columns:: - - # old way - Table( - "some_table", - metadata, - Column("x", Integer, primary_key=True, autoincrement=False), - Column("y", Integer, primary_key=True, autoincrement=False), - ) - -With the new behavior, the composite primary key will not have autoincrement -semantics unless a column is marked explicitly with ``autoincrement=True``:: - - # column 'y' will be SERIAL/AUTO_INCREMENT/ auto-generating - Table( - "some_table", - metadata, - Column("x", Integer, primary_key=True), - Column("y", Integer, primary_key=True, autoincrement=True), - ) - -In order to anticipate some potential backwards-incompatible scenarios, -the :meth:`_schema.Table.insert` construct will perform more thorough checks -for missing primary key values on composite primary key columns that don't -have autoincrement set up; given a table such as:: - - Table( - "b", - metadata, - Column("x", Integer, primary_key=True), - Column("y", Integer, primary_key=True), - ) - -An INSERT emitted with no values for this table will produce this warning: - -.. sourcecode:: text - - SAWarning: Column 'b.x' is marked as a member of the primary - key for table 'b', but has no Python-side or server-side default - generator indicated, nor does it indicate 'autoincrement=True', - and no explicit value is passed. Primary key columns may not - store NULL. Note that as of SQLAlchemy 1.1, 'autoincrement=True' - must be indicated explicitly for composite (e.g. multicolumn) - primary keys if AUTO_INCREMENT/SERIAL/IDENTITY behavior is - expected for one of the columns in the primary key. CREATE TABLE - statements are impacted by this change as well on most backends. - -For a column that is receiving primary key values from a server-side -default or something less common such as a trigger, the presence of a -value generator can be indicated using :class:`.FetchedValue`:: - - Table( - "b", - metadata, - Column("x", Integer, primary_key=True, server_default=FetchedValue()), - Column("y", Integer, primary_key=True, server_default=FetchedValue()), - ) - -For the very unlikely case where a composite primary key is actually intended -to store NULL in one or more of its columns (only supported on SQLite and MySQL), -specify the column with ``nullable=True``:: - - Table( - "b", - metadata, - Column("x", Integer, primary_key=True), - Column("y", Integer, primary_key=True, nullable=True), - ) - -In a related change, the ``autoincrement`` flag may be set to True -on a column that has a client-side or server-side default. This typically -will not have much impact on the behavior of the column during an INSERT. - - -.. seealso:: - - :ref:`change_mysql_3216` - -:ticket:`3216` - -.. _change_is_distinct_from: - -Support for IS DISTINCT FROM and IS NOT DISTINCT FROM ------------------------------------------------------ - -New operators :meth:`.ColumnOperators.is_distinct_from` and -:meth:`.ColumnOperators.isnot_distinct_from` allow the IS DISTINCT -FROM and IS NOT DISTINCT FROM sql operation: - -.. sourcecode:: pycon+sql - - >>> print(column("x").is_distinct_from(None)) - {printsql}x IS DISTINCT FROM NULL{stop} - -Handling is provided for NULL, True and False: - -.. sourcecode:: pycon+sql - - >>> print(column("x").isnot_distinct_from(False)) - {printsql}x IS NOT DISTINCT FROM false{stop} - -For SQLite, which doesn't have this operator, "IS" / "IS NOT" is rendered, -which on SQLite works for NULL unlike other backends: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy.dialects import sqlite - >>> print(column("x").is_distinct_from(None).compile(dialect=sqlite.dialect())) - {printsql}x IS NOT NULL{stop} - -.. _change_1957: - -Core and ORM support for FULL OUTER JOIN ----------------------------------------- - -The new flag :paramref:`.FromClause.outerjoin.full`, available at the Core -and ORM level, instructs the compiler to render ``FULL OUTER JOIN`` -where it would normally render ``LEFT OUTER JOIN``:: - - stmt = select([t1]).select_from(t1.outerjoin(t2, full=True)) - -The flag also works at the ORM level:: - - q = session.query(MyClass).outerjoin(MyOtherClass, full=True) - -:ticket:`1957` - -.. _change_3501: - -ResultSet column matching enhancements; positional column setup for textual SQL -------------------------------------------------------------------------------- - -A series of improvements were made to the :class:`_engine.ResultProxy` system -in the 1.0 series as part of :ticket:`918`, which reorganizes the internals -to match cursor-bound result columns with table/ORM metadata positionally, -rather than by matching names, for compiled SQL constructs that contain full -information about the result rows to be returned. This allows a dramatic savings -on Python overhead as well as much greater accuracy in linking ORM and Core -SQL expressions to result rows. In 1.1, this reorganization has been taken -further internally, and also has been made available to pure-text SQL -constructs via the use of the recently added :meth:`_expression.TextClause.columns` method. - -TextAsFrom.columns() now works positionally -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The :meth:`_expression.TextClause.columns` method, added in 0.9, accepts column-based arguments -positionally; in 1.1, when all columns are passed positionally, the correlation -of these columns to the ultimate result set is also performed positionally. -The key advantage here is that textual SQL can now be linked to an ORM- -level result set without the need to deal with ambiguous or duplicate column -names, or with having to match labeling schemes to ORM-level labeling schemes. All -that's needed now is the same ordering of columns within the textual SQL -and the column arguments passed to :meth:`_expression.TextClause.columns`:: - - - from sqlalchemy import text - - stmt = text( - "SELECT users.id, addresses.id, users.id, " - "users.name, addresses.email_address AS email " - "FROM users JOIN addresses ON users.id=addresses.user_id " - "WHERE users.id = 1" - ).columns(User.id, Address.id, Address.user_id, User.name, Address.email_address) - - query = session.query(User).from_statement(stmt).options(contains_eager(User.addresses)) - result = query.all() - -Above, the textual SQL contains the column "id" three times, which would -normally be ambiguous. Using the new feature, we can apply the mapped -columns from the ``User`` and ``Address`` class directly, even linking -the ``Address.user_id`` column to the ``users.id`` column in textual SQL -for fun, and the :class:`_query.Query` object will receive rows that are correctly -targetable as needed, including for an eager load. - -This change is **backwards incompatible** with code that passes the columns -to the method with a different ordering than is present in the textual statement. -It is hoped that this impact will be low due to the fact that this -method has always been documented illustrating the columns being passed in the same order as that of the -textual SQL statement, as would seem intuitive, even though the internals -weren't checking for this. The method itself was only added as of 0.9 in -any case and may not yet have widespread use. Notes on exactly how to handle -this behavioral change for applications using it are at :ref:`behavior_change_3501`. - -.. seealso:: - - :ref:`tutorial_select_arbitrary_text` - - :ref:`behavior_change_3501` - backwards compatibility remarks - -Positional matching is trusted over name-based matching for Core/ORM SQL constructs -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Another aspect of this change is that the rules for matching columns have also been modified -to rely upon "positional" matching more fully for compiled SQL constructs -as well. Given a statement like the following:: - - ua = users.alias("ua") - stmt = select([users.c.user_id, ua.c.user_id]) - -The above statement will compile to: - -.. sourcecode:: sql - - SELECT users.user_id, ua.user_id FROM users, users AS ua - -In 1.0, the above statement when executed would be matched to its original -compiled construct using positional matching, however because the statement -contains the ``'user_id'`` label duplicated, the "ambiguous column" rule -would still get involved and prevent the columns from being fetched from a row. -As of 1.1, the "ambiguous column" rule does not affect an exact match from -a column construct to the SQL column, which is what the ORM uses to -fetch columns:: - - result = conn.execute(stmt) - row = result.first() - - # these both match positionally, so no error - user_id = row[users.c.user_id] - ua_id = row[ua.c.user_id] - - # this still raises, however - user_id = row["user_id"] - -Much less likely to get an "ambiguous column" error message -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -As part of this change, the wording of the error message ``Ambiguous column -name '' in result set! try 'use_labels' option on select statement.`` -has been dialed back; as this message should now be extremely rare when using -the ORM or Core compiled SQL constructs, it merely states -``Ambiguous column name '' in result set column descriptions``, and -only when a result column is retrieved using the string name that is actually -ambiguous, e.g. ``row['user_id']`` in the above example. It also now refers -to the actual ambiguous name from the rendered SQL statement itself, -rather than indicating the key or name that was local to the construct being -used for the fetch. - -:ticket:`3501` - -.. _change_3292: - -Support for Python's native ``enum`` type and compatible forms --------------------------------------------------------------- - -The :class:`.Enum` type can now be constructed using any -PEP-435 compliant enumerated type. When using this mode, input values -and return values are the actual enumerated objects, not the -string/integer/etc values:: - - import enum - from sqlalchemy import Table, MetaData, Column, Enum, create_engine - - - class MyEnum(enum.Enum): - one = 1 - two = 2 - three = 3 - - - t = Table("data", MetaData(), Column("value", Enum(MyEnum))) - - e = create_engine("sqlite://") - t.create(e) - - e.execute(t.insert(), {"value": MyEnum.two}) - assert e.scalar(t.select()) is MyEnum.two - -The ``Enum.enums`` collection is now a list instead of a tuple -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -As part of the changes to :class:`.Enum`, the :attr:`.Enum.enums` collection -of elements is now a list instead of a tuple. This because lists -are appropriate for variable length sequences of homogeneous items where -the position of the element is not semantically significant. - -:ticket:`3292` - -.. _change_gh_231: - -Negative integer indexes accommodated by Core result rows ---------------------------------------------------------- - -The :class:`.RowProxy` object now accommodates single negative integer indexes -like a regular Python sequence, both in the pure Python and C-extension -version. Previously, negative values would only work in slices:: - - >>> from sqlalchemy import create_engine - >>> e = create_engine("sqlite://") - >>> row = e.execute("select 1, 2, 3").first() - >>> row[-1], row[-2], row[1], row[-2:2] - 3 2 2 (2,) - -.. _change_3095: - -The ``Enum`` type now does in-Python validation of values ---------------------------------------------------------- - -To accommodate for Python native enumerated objects, as well as for edge -cases such as that of where a non-native ENUM type is used within an ARRAY -and a CHECK constraint is infeasible, the :class:`.Enum` datatype now adds -in-Python validation of input values when the :paramref:`.Enum.validate_strings` -flag is used (1.1.0b2):: - - - >>> from sqlalchemy import Table, MetaData, Column, Enum, create_engine - >>> t = Table( - ... "data", - ... MetaData(), - ... Column("value", Enum("one", "two", "three", validate_strings=True)), - ... ) - >>> e = create_engine("sqlite://") - >>> t.create(e) - >>> e.execute(t.insert(), {"value": "four"}) - Traceback (most recent call last): - ... - sqlalchemy.exc.StatementError: (exceptions.LookupError) - "four" is not among the defined enum values - [SQL: u'INSERT INTO data (value) VALUES (?)'] - [parameters: [{'value': 'four'}]] - -This validation is turned off by default as there are already use cases -identified where users don't want such validation (such as string comparisons). -For non-string types, it necessarily takes place in all cases. The -check also occurs unconditionally on the result-handling side as well, when -values coming from the database are returned. - -This validation is in addition to the existing behavior of creating a -CHECK constraint when a non-native enumerated type is used. The creation of -this CHECK constraint can now be disabled using the new -:paramref:`.Enum.create_constraint` flag. - -:ticket:`3095` - -.. _change_3730: - -Non-native boolean integer values coerced to zero/one/None in all cases ------------------------------------------------------------------------ - -The :class:`.Boolean` datatype coerces Python booleans to integer values -for backends that don't have a native boolean type, such as SQLite and -MySQL. On these backends, a CHECK constraint is normally set up which -ensures the values in the database are in fact one of these two values. -However, MySQL ignores CHECK constraints, the constraint is optional, and -an existing database might not have this constraint. The :class:`.Boolean` -datatype has been repaired such that an incoming Python-side value that is -already an integer value is coerced to zero or one, not just passed as-is; -additionally, the C-extension version of the int-to-boolean processor for -results now uses the same Python boolean interpretation of the value, -rather than asserting an exact one or zero value. This is now consistent -with the pure-Python int-to-boolean processor and is more forgiving of -existing data already within the database. Values of None/NULL are as before -retained as None/NULL. - -.. note:: - - this change had an unintended side effect that the interpretation of non- - integer values, such as strings, also changed in behavior such that the - string value ``"0"`` would be interpreted as "true", but only on backends - that don't have a native boolean datatype - on "native boolean" backends - like PostgreSQL, the string value ``"0"`` is passed directly to the driver - and is interpreted as "false". This is an inconsistency that did not occur - with the previous implementation. It should be noted that passing strings or - any other value outside of ``None``, ``True``, ``False``, ``1``, ``0`` to - the :class:`.Boolean` datatype is **not supported** and version 1.2 will - raise an error for this scenario (or possibly just emit a warning, TBD). - See also :ticket:`4102`. - - -:ticket:`3730` - -.. _change_2837: - -Large parameter and row values are now truncated in logging and exception displays ----------------------------------------------------------------------------------- - -A large value present as a bound parameter for a SQL statement, as well as a -large value present in a result row, will now be truncated during display -within logging, exception reporting, as well as ``repr()`` of the row itself:: - - >>> from sqlalchemy import create_engine - >>> import random - >>> e = create_engine("sqlite://", echo="debug") - >>> some_value = "".join(chr(random.randint(52, 85)) for i in range(5000)) - >>> row = e.execute("select ?", [some_value]).first() - ... # (lines are wrapped for clarity) ... - 2016-02-17 13:23:03,027 INFO sqlalchemy.engine.base.Engine select ? - 2016-02-17 13:23:03,027 INFO sqlalchemy.engine.base.Engine - ('E6@?>9HPOJB<:=TSTLA;9K;9FPM4M8M@;NM6GU - LUAEBT9QGHNHTHR5EP75@OER4?SKC;D:TFUMD:M>;C6U:JLM6R67GEK4=4:P - GJ7HQ6 ... (4702 characters truncated) ... J6IK546AJMB4N6S9L;;9AKI;=RJP - HDSSOTNBUEEC9@Q:RCL:I@5?FO<9K>KJAGAO@E6@A7JI8O:J7B69T6<8;F:S;4BEIJS9HM - K:;5OLPM@JR;R:J6Q>7T@I::OTDC:CC<=NGP6C>BC8N',) - 2016-02-17 13:23:03,027 DEBUG sqlalchemy.engine.base.Engine Col ('?',) - 2016-02-17 13:23:03,027 DEBUG sqlalchemy.engine.base.Engine - Row (u'E6@?>9HPOJB<:=TSTLA;9K;9FPM4M8M@; - NM6GULUAEBT9QGHNHTHR5EP75@OER4?SKC;D:TFUMD:M>;C6U:JLM6R67GEK4=4:PGJ7HQ ... (4703 characters truncated) ... J6IK546AJMB4N6S9L;;9AKI;= - RJPHDSSOTNBUEEC9@Q:RCL:I@5?FO<9K>KJAGAO@E6@A7JI8O:J7B69T6<8;F:S;4BEIJS9HM - K:;5OLPM@JR;R:J6Q>7T@I::OTDC:CC<=NGP6C>BC8N',) - >>> print(row) - (u'E6@?>9HPOJB<:=TSTLA;9K;9FPM4M8M@;NM6 - GULUAEBT9QGHNHTHR5EP75@OER4?SKC;D:TFUMD:M>;C6U:JLM6R67GEK4 - =4:PGJ7HQ ... (4703 characters truncated) ... J6IK546AJMB4N6S9L;;9AKI; - =RJPHDSSOTNBUEEC9@Q:RCL:I@5?FO<9K>KJAGAO@E6@A7JI8O:J7B69T6<8;F:S;4BEIJS9H - MK:;5OLPM@JR;R:J6Q>7T@I::OTDC:CC<=NGP6C>BC8N',) - - -:ticket:`2837` - - -.. _change_3619: - -JSON support added to Core --------------------------- - -As MySQL now has a JSON datatype in addition to the PostgreSQL JSON datatype, -the core now gains a :class:`sqlalchemy.types.JSON` datatype that is the basis -for both of these. Using this type allows access to the "getitem" operator -as well as the "getpath" operator in a way that is agnostic across PostgreSQL -and MySQL. - -The new datatype also has a series of improvements to the handling of -NULL values as well as expression handling. - -.. seealso:: - - :ref:`change_3547` - - :class:`_types.JSON` - - :class:`_postgresql.JSON` - - :class:`.mysql.JSON` - -:ticket:`3619` - -.. _change_3514: - -JSON "null" is inserted as expected with ORM operations, omitted when not present -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The :class:`_types.JSON` type and its descendant types :class:`_postgresql.JSON` -and :class:`.mysql.JSON` have a flag :paramref:`.types.JSON.none_as_null` which -when set to True indicates that the Python value ``None`` should translate -into a SQL NULL rather than a JSON NULL value. This flag defaults to False, -which means that the Python value ``None`` should result in a JSON NULL value. - -This logic would fail, and is now corrected, in the following circumstances: - -1. When the column also contained a default or server_default value, -a positive value of ``None`` on the mapped attribute that expects to persist -JSON "null" would still result in the column-level default being triggered, -replacing the ``None`` value:: - - class MyObject(Base): - # ... - - json_value = Column(JSON(none_as_null=False), default="some default") - - - # would insert "some default" instead of "'null'", - # now will insert "'null'" - obj = MyObject(json_value=None) - session.add(obj) - session.commit() - -2. When the column *did not* contain a default or server_default value, a missing -value on a JSON column configured with none_as_null=False would still render -JSON NULL rather than falling back to not inserting any value, behaving -inconsistently vs. all other datatypes:: - - class MyObject(Base): - # ... - - some_other_value = Column(String(50)) - json_value = Column(JSON(none_as_null=False)) - - - # would result in NULL for some_other_value, - # but json "'null'" for json_value. Now results in NULL for both - # (the json_value is omitted from the INSERT) - obj = MyObject() - session.add(obj) - session.commit() - -This is a behavioral change that is backwards incompatible for an application -that was relying upon this to default a missing value as JSON null. This -essentially establishes that a **missing value is distinguished from a present -value of None**. See :ref:`behavior_change_3514` for further detail. - -3. When the :meth:`.Session.bulk_insert_mappings` method were used, ``None`` -would be ignored in all cases:: - - # would insert SQL NULL and/or trigger defaults, - # now inserts "'null'" - session.bulk_insert_mappings(MyObject, [{"json_value": None}]) - -The :class:`_types.JSON` type now implements the -:attr:`.TypeEngine.should_evaluate_none` flag, -indicating that ``None`` should not be ignored here; it is configured -automatically based on the value of :paramref:`.types.JSON.none_as_null`. -Thanks to :ticket:`3061`, we can differentiate when the value ``None`` is actively -set by the user versus when it was never set at all. - -The feature applies as well to the new base :class:`_types.JSON` type -and its descendant types. - -:ticket:`3514` - -.. _change_3514_jsonnull: - -New JSON.NULL Constant Added -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -To ensure that an application can always have full control at the value level -of whether a :class:`_types.JSON`, :class:`_postgresql.JSON`, :class:`.mysql.JSON`, -or :class:`_postgresql.JSONB` column -should receive a SQL NULL or JSON ``"null"`` value, the constant -:attr:`.types.JSON.NULL` has been added, which in conjunction with -:func:`.null` can be used to determine fully between SQL NULL and -JSON ``"null"``, regardless of what :paramref:`.types.JSON.none_as_null` is set -to:: - - from sqlalchemy import null - from sqlalchemy.dialects.postgresql import JSON - - obj1 = MyObject(json_value=null()) # will *always* insert SQL NULL - obj2 = MyObject(json_value=JSON.NULL) # will *always* insert JSON string "null" - - session.add_all([obj1, obj2]) - session.commit() - -The feature applies as well to the new base :class:`_types.JSON` type -and its descendant types. - -:ticket:`3514` - -.. _change_3516: - -Array support added to Core; new ANY and ALL operators ------------------------------------------------------- - -Along with the enhancements made to the PostgreSQL :class:`_postgresql.ARRAY` -type described in :ref:`change_3503`, the base class of :class:`_postgresql.ARRAY` -itself has been moved to Core in a new class :class:`_types.ARRAY`. - -Arrays are part of the SQL standard, as are several array-oriented functions -such as ``array_agg()`` and ``unnest()``. In support of these constructs -for not just PostgreSQL but also potentially for other array-capable backends -in the future such as DB2, the majority of array logic for SQL expressions -is now in Core. The :class:`_types.ARRAY` type still **only works on -PostgreSQL**, however it can be used directly, supporting special array -use cases such as indexed access, as well as support for the ANY and ALL:: - - mytable = Table("mytable", metadata, Column("data", ARRAY(Integer, dimensions=2))) - - expr = mytable.c.data[5][6] - - expr = mytable.c.data[5].any(12) - -In support of ANY and ALL, the :class:`_types.ARRAY` type retains the same -:meth:`.types.ARRAY.Comparator.any` and :meth:`.types.ARRAY.Comparator.all` methods -from the PostgreSQL type, but also exports these operations to new -standalone operator functions :func:`_expression.any_` and -:func:`_expression.all_`. These two functions work in more -of the traditional SQL way, allowing a right-side expression form such -as:: - - from sqlalchemy import any_, all_ - - select([mytable]).where(12 == any_(mytable.c.data[5])) - -For the PostgreSQL-specific operators "contains", "contained_by", and -"overlaps", one should continue to use the :class:`_postgresql.ARRAY` -type directly, which provides all functionality of the :class:`_types.ARRAY` -type as well. - -The :func:`_expression.any_` and :func:`_expression.all_` operators -are open-ended at the Core level, however their interpretation by backend -databases is limited. On the PostgreSQL backend, the two operators -**only accept array values**. Whereas on the MySQL backend, they -**only accept subquery values**. On MySQL, one can use an expression -such as:: - - from sqlalchemy import any_, all_ - - subq = select([mytable.c.value]) - select([mytable]).where(12 > any_(subq)) - -:ticket:`3516` - -.. _change_3132: - -New Function features, "WITHIN GROUP", array_agg and set aggregate functions ----------------------------------------------------------------------------- - -With the new :class:`_types.ARRAY` type we can also implement a pre-typed -function for the ``array_agg()`` SQL function that returns an array, -which is now available using :class:`_functions.array_agg`:: - - from sqlalchemy import func - - stmt = select([func.array_agg(table.c.value)]) - -A PostgreSQL element for an aggregate ORDER BY is also added via -:class:`_postgresql.aggregate_order_by`:: - - from sqlalchemy.dialects.postgresql import aggregate_order_by - - expr = func.array_agg(aggregate_order_by(table.c.a, table.c.b.desc())) - stmt = select([expr]) - -Producing: - -.. sourcecode:: sql - - SELECT array_agg(table1.a ORDER BY table1.b DESC) AS array_agg_1 FROM table1 - -The PG dialect itself also provides an :func:`_postgresql.array_agg` wrapper to -ensure the :class:`_postgresql.ARRAY` type:: - - from sqlalchemy.dialects.postgresql import array_agg - - stmt = select([array_agg(table.c.value).contains("foo")]) - -Additionally, functions like ``percentile_cont()``, ``percentile_disc()``, -``rank()``, ``dense_rank()`` and others that require an ordering via -``WITHIN GROUP (ORDER BY )`` are now available via the -:meth:`.FunctionElement.within_group` modifier:: - - from sqlalchemy import func - - stmt = select( - [ - department.c.id, - func.percentile_cont(0.5).within_group(department.c.salary.desc()), - ] - ) - -The above statement would produce SQL similar to: - -.. sourcecode:: sql - - SELECT department.id, percentile_cont(0.5) - WITHIN GROUP (ORDER BY department.salary DESC) - -Placeholders with correct return types are now provided for these functions, -and include :class:`.percentile_cont`, :class:`.percentile_disc`, -:class:`.rank`, :class:`.dense_rank`, :class:`.mode`, :class:`.percent_rank`, -and :class:`.cume_dist`. - -:ticket:`3132` :ticket:`1370` - -.. _change_2919: - -TypeDecorator now works with Enum, Boolean, "schema" types automatically ------------------------------------------------------------------------- - -The :class:`.SchemaType` types include types such as :class:`.Enum` -and :class:`.Boolean` which, in addition to corresponding to a database -type, also generate either a CHECK constraint or in the case of PostgreSQL -ENUM a new CREATE TYPE statement, will now work automatically with -:class:`.TypeDecorator` recipes. Previously, a :class:`.TypeDecorator` for -an :class:`_postgresql.ENUM` had to look like this:: - - # old way - class MyEnum(TypeDecorator, SchemaType): - impl = postgresql.ENUM("one", "two", "three", name="myenum") - - def _set_table(self, table): - self.impl._set_table(table) - -The :class:`.TypeDecorator` now propagates those additional events so it -can be done like any other type:: - - # new way - class MyEnum(TypeDecorator): - impl = postgresql.ENUM("one", "two", "three", name="myenum") - -:ticket:`2919` - -.. _change_2685: - -Multi-Tenancy Schema Translation for Table objects --------------------------------------------------- - -To support the use case of an application that uses the same set of -:class:`_schema.Table` objects in many schemas, such as schema-per-user, a new -execution option :paramref:`.Connection.execution_options.schema_translate_map` -is added. Using this mapping, a set of :class:`_schema.Table` -objects can be made on a per-connection basis to refer to any set of schemas -instead of the :paramref:`_schema.Table.schema` to which they were assigned. The -translation works for DDL and SQL generation, as well as with the ORM. - -For example, if the ``User`` class were assigned the schema "per_user":: - - class User(Base): - __tablename__ = "user" - id = Column(Integer, primary_key=True) - - __table_args__ = {"schema": "per_user"} - -On each request, the :class:`.Session` can be set up to refer to a -different schema each time:: - - session = Session() - session.connection( - execution_options={"schema_translate_map": {"per_user": "account_one"}} - ) - - # will query from the ``account_one.user`` table - session.query(User).get(5) - -.. seealso:: - - :ref:`schema_translating` - -:ticket:`2685` - -.. _change_3631: - -"Friendly" stringification of Core SQL constructs without a dialect -------------------------------------------------------------------- - -Calling ``str()`` on a Core SQL construct will now produce a string -in more cases than before, supporting various SQL constructs not normally -present in default SQL such as RETURNING, array indexes, and non-standard -datatypes: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy import table, column - t>>> t = table('x', column('a'), column('b')) - >>> print(t.insert().returning(t.c.a, t.c.b)) - {printsql}INSERT INTO x (a, b) VALUES (:a, :b) RETURNING x.a, x.b - -The ``str()`` function now calls upon an entirely separate dialect / compiler -intended just for plain string printing without a specific dialect set up, -so as more "just show me a string!" cases come up, these can be added -to this dialect/compiler without impacting behaviors on real dialects. - -.. seealso:: - - :ref:`change_3081` - -:ticket:`3631` - -.. _change_3531: - -The type_coerce function is now a persistent SQL element --------------------------------------------------------- - -The :func:`_expression.type_coerce` function previously would return -an object either of type :class:`.BindParameter` or :class:`.Label`, depending -on the input. An effect this would have was that in the case where expression -transformations were used, such as the conversion of an element from a -:class:`_schema.Column` to a :class:`.BindParameter` that's critical to ORM-level -lazy loading, the type coercion information would not be used since it would -have been lost already. - -To improve this behavior, the function now returns a persistent -:class:`.TypeCoerce` container around the given expression, which itself -remains unaffected; this construct is evaluated explicitly by the -SQL compiler. This allows for the coercion of the inner expression -to be maintained no matter how the statement is modified, including if -the contained element is replaced with a different one, as is common -within the ORM's lazy loading feature. - -The test case illustrating the effect makes use of a heterogeneous -primaryjoin condition in conjunction with custom types and lazy loading. -Given a custom type that applies a CAST as a "bind expression":: - - class StringAsInt(TypeDecorator): - impl = String - - def column_expression(self, col): - return cast(col, Integer) - - def bind_expression(self, value): - return cast(value, String) - -Then, a mapping where we are equating a string "id" column on one -table to an integer "id" column on the other:: - - class Person(Base): - __tablename__ = "person" - id = Column(StringAsInt, primary_key=True) - - pets = relationship( - "Pets", - primaryjoin=( - "foreign(Pets.person_id)" "==cast(type_coerce(Person.id, Integer), Integer)" - ), - ) - - - class Pets(Base): - __tablename__ = "pets" - id = Column("id", Integer, primary_key=True) - person_id = Column("person_id", Integer) - -Above, in the :paramref:`_orm.relationship.primaryjoin` expression, we are -using :func:`.type_coerce` to handle bound parameters passed via -lazyloading as integers, since we already know these will come from -our ``StringAsInt`` type which maintains the value as an integer in -Python. We are then using :func:`.cast` so that as a SQL expression, -the VARCHAR "id" column will be CAST to an integer for a regular non- -converted join as with :meth:`_query.Query.join` or :func:`_orm.joinedload`. -That is, a joinedload of ``.pets`` looks like: - -.. sourcecode:: sql - - SELECT person.id AS person_id, pets_1.id AS pets_1_id, - pets_1.person_id AS pets_1_person_id - FROM person - LEFT OUTER JOIN pets AS pets_1 - ON pets_1.person_id = CAST(person.id AS INTEGER) - -Without the CAST in the ON clause of the join, strongly-typed databases -such as PostgreSQL will refuse to implicitly compare the integer and fail. - -The lazyload case of ``.pets`` relies upon replacing -the ``Person.id`` column at load time with a bound parameter, which receives -a Python-loaded value. This replacement is specifically where the intent -of our :func:`.type_coerce` function would be lost. Prior to the change, -this lazy load comes out as: - -.. sourcecode:: sql - - SELECT pets.id AS pets_id, pets.person_id AS pets_person_id - FROM pets - WHERE pets.person_id = CAST(CAST(%(param_1)s AS VARCHAR) AS INTEGER) - -- {'param_1': 5} - -Where above, we see that our in-Python value of ``5`` is CAST first -to a VARCHAR, then back to an INTEGER in SQL; a double CAST which works, -but is nevertheless not what we asked for. - -With the change, the :func:`.type_coerce` function maintains a wrapper -even after the column is swapped out for a bound parameter, and the query now -looks like: - -.. sourcecode:: sql - - SELECT pets.id AS pets_id, pets.person_id AS pets_person_id - FROM pets - WHERE pets.person_id = CAST(%(param_1)s AS INTEGER) - -- {'param_1': 5} - -Where our outer CAST that's in our primaryjoin still takes effect, but the -needless CAST that's in part of the ``StringAsInt`` custom type is removed -as intended by the :func:`.type_coerce` function. - - -:ticket:`3531` - -Key Behavioral Changes - ORM -============================ - -.. _behavior_change_3514: - -JSON Columns will not insert JSON NULL if no value is supplied and no default is established --------------------------------------------------------------------------------------------- - -As detailed in :ref:`change_3514`, :class:`_types.JSON` will not render -a JSON "null" value if the value is missing entirely. To prevent SQL NULL, -a default should be set up. Given the following mapping:: - - class MyObject(Base): - # ... - - json_value = Column(JSON(none_as_null=False), nullable=False) - -The following flush operation will fail with an integrity error:: - - obj = MyObject() # note no json_value - session.add(obj) - session.commit() # will fail with integrity error - -If the default for the column should be JSON NULL, set this on the -Column:: - - class MyObject(Base): - # ... - - json_value = Column(JSON(none_as_null=False), nullable=False, default=JSON.NULL) - -Or, ensure the value is present on the object:: - - obj = MyObject(json_value=None) - session.add(obj) - session.commit() # will insert JSON NULL - -Note that setting ``None`` for the default is the same as omitting it entirely; -the :paramref:`.types.JSON.none_as_null` flag does not impact the value of ``None`` -passed to :paramref:`_schema.Column.default` or :paramref:`_schema.Column.server_default`:: - - # default=None is the same as omitting it entirely, does not apply JSON NULL - json_value = Column(JSON(none_as_null=False), nullable=False, default=None) - -.. seealso:: - - :ref:`change_3514` - -.. _change_3641: - -Columns no longer added redundantly with DISTINCT + ORDER BY ------------------------------------------------------------- - -A query such as the following will now augment only those columns -that are missing from the SELECT list, without duplicates:: - - q = ( - session.query(User.id, User.name.label("name")) - .distinct() - .order_by(User.id, User.name, User.fullname) - ) - -Produces: - -.. sourcecode:: sql - - SELECT DISTINCT user.id AS a_id, user.name AS name, - user.fullname AS a_fullname - FROM a ORDER BY user.id, user.name, user.fullname - -Previously, it would produce: - -.. sourcecode:: sql - - SELECT DISTINCT user.id AS a_id, user.name AS name, user.name AS a_name, - user.fullname AS a_fullname - FROM a ORDER BY user.id, user.name, user.fullname - -Where above, the ``user.name`` column is added unnecessarily. The results -would not be affected, as the additional columns are not included in the -result in any case, but the columns are unnecessary. - -Additionally, when the PostgreSQL DISTINCT ON format is used by passing -expressions to :meth:`_query.Query.distinct`, the above "column adding" logic -is disabled entirely. - -When the query is being bundled into a subquery for the purposes of -joined eager loading, the "augment column list" rules are necessarily -more aggressive so that the ORDER BY can still be satisfied, so this case -remains unchanged. - -:ticket:`3641` - -.. _change_3776: - -Same-named @validates decorators will now raise an exception ------------------------------------------------------------- - -The :func:`_orm.validates` decorator is only intended to be created once -per class for a particular attribute name. Creating more than one -now raises an error, whereas previously it would silently pick only the -last defined validator:: - - class A(Base): - __tablename__ = "a" - id = Column(Integer, primary_key=True) - - data = Column(String) - - @validates("data") - def _validate_data_one(self): - assert "x" in data - - @validates("data") - def _validate_data_two(self): - assert "y" in data - - - configure_mappers() - -Will raise: - -.. sourcecode:: text - - sqlalchemy.exc.InvalidRequestError: A validation function for mapped attribute 'data' - on mapper Mapper|A|a already exists. - -:ticket:`3776` - -Key Behavioral Changes - Core -============================= - -.. _behavior_change_3501: - -TextClause.columns() will match columns positionally, not by name, when passed positionally -------------------------------------------------------------------------------------------- - -The new behavior of the :meth:`_expression.TextClause.columns` method, which itself -was recently added as of the 0.9 series, is that when -columns are passed positionally without any additional keyword arguments, -they are linked to the ultimate result set -columns positionally, and no longer on name. It is hoped that the impact -of this change will be low due to the fact that the method has always been documented -illustrating the columns being passed in the same order as that of the -textual SQL statement, as would seem intuitive, even though the internals -weren't checking for this. - -An application that is using this method by passing :class:`_schema.Column` objects -to it positionally must ensure that the position of those :class:`_schema.Column` -objects matches the position in which these columns are stated in the -textual SQL. - -E.g., code like the following:: - - stmt = text("SELECT id, name, description FROM table") - - # no longer matches by name - stmt = stmt.columns(my_table.c.name, my_table.c.description, my_table.c.id) - -Would no longer work as expected; the order of the columns given is now -significant:: - - # correct version - stmt = stmt.columns(my_table.c.id, my_table.c.name, my_table.c.description) - -Possibly more likely, a statement that worked like this:: - - stmt = text("SELECT * FROM table") - stmt = stmt.columns(my_table.c.id, my_table.c.name, my_table.c.description) - -is now slightly risky, as the "*" specification will generally deliver columns -in the order in which they are present in the table itself. If the structure -of the table changes due to schema changes, this ordering may no longer be the same. -Therefore when using :meth:`_expression.TextClause.columns`, it's advised to list out -the desired columns explicitly in the textual SQL, though it's no longer -necessary to worry about the names themselves in the textual SQL. - -.. seealso:: - - :ref:`change_3501` - -.. _change_3809: - -String server_default now literal quoted ----------------------------------------- - -A server default passed to :paramref:`_schema.Column.server_default` as a plain -Python string that has quotes embedded is now -passed through the literal quoting system: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy.schema import MetaData, Table, Column, CreateTable - >>> from sqlalchemy.types import String - >>> t = Table("t", MetaData(), Column("x", String(), server_default="hi ' there")) - >>> print(CreateTable(t)) - {printsql}CREATE TABLE t ( - x VARCHAR DEFAULT 'hi '' there' - ) - -Previously the quote would render directly. This change may be backwards -incompatible for applications with such a use case who were working around -the issue. - - -:ticket:`3809` - -.. _change_2528: - -A UNION or similar of SELECTs with LIMIT/OFFSET/ORDER BY now parenthesizes the embedded selects ------------------------------------------------------------------------------------------------ - -An issue that, like others, was long driven by SQLite's lack of capabilities -has now been enhanced to work on all supporting backends. We refer to a query that -is a UNION of SELECT statements that themselves contain row-limiting or ordering -features which include LIMIT, OFFSET, and/or ORDER BY: - -.. sourcecode:: sql - - (SELECT x FROM table1 ORDER BY y LIMIT 1) UNION - (SELECT x FROM table2 ORDER BY y LIMIT 2) - -The above query requires parenthesis within each sub-select in order to -group the sub-results correctly. Production of the above statement in -SQLAlchemy Core looks like:: - - stmt1 = select([table1.c.x]).order_by(table1.c.y).limit(1) - stmt2 = select([table1.c.x]).order_by(table2.c.y).limit(2) - - stmt = union(stmt1, stmt2) - -Previously, the above construct would not produce parenthesization for the -inner SELECT statements, producing a query that fails on all backends. - -The above formats will **continue to fail on SQLite**; additionally, the format -that includes ORDER BY but no LIMIT/SELECT will **continue to fail on Oracle**. -This is not a backwards-incompatible change, because the queries fail without -the parentheses as well; with the fix, the queries at least work on all other -databases. - -In all cases, in order to produce a UNION of limited SELECT statements that -also works on SQLite and in all cases on Oracle, the -subqueries must be a SELECT of an ALIAS:: - - stmt1 = select([table1.c.x]).order_by(table1.c.y).limit(1).alias().select() - stmt2 = select([table2.c.x]).order_by(table2.c.y).limit(2).alias().select() - - stmt = union(stmt1, stmt2) - -This workaround works on all SQLAlchemy versions. In the ORM, it looks like:: - - stmt1 = session.query(Model1).order_by(Model1.y).limit(1).subquery().select() - stmt2 = session.query(Model2).order_by(Model2.y).limit(1).subquery().select() - - stmt = session.query(Model1).from_statement(stmt1.union(stmt2)) - -The behavior here has many parallels to the "join rewriting" behavior -introduced in SQLAlchemy 0.9 in :ref:`feature_joins_09`; however in this case -we have opted not to add new rewriting behavior to accommodate this -case for SQLite. -The existing rewriting behavior is very complicated already, and the case of -UNIONs with parenthesized SELECT statements is much less common than the -"right-nested-join" use case of that feature. - -:ticket:`2528` - - -Dialect Improvements and Changes - PostgreSQL -============================================= - -.. _change_3529: - -Support for INSERT..ON CONFLICT (DO UPDATE | DO NOTHING) --------------------------------------------------------- - -The ``ON CONFLICT`` clause of ``INSERT`` added to PostgreSQL as of -version 9.5 is now supported using a PostgreSQL-specific version of the -:class:`_expression.Insert` object, via :func:`sqlalchemy.dialects.postgresql.dml.insert`. -This :class:`_expression.Insert` subclass adds two new methods :meth:`_expression.Insert.on_conflict_do_update` -and :meth:`_expression.Insert.on_conflict_do_nothing` which implement the full syntax -supported by PostgreSQL 9.5 in this area:: - - from sqlalchemy.dialects.postgresql import insert - - insert_stmt = insert(my_table).values(id="some_id", data="some data to insert") - - do_update_stmt = insert_stmt.on_conflict_do_update( - index_elements=[my_table.c.id], set_=dict(data="some data to update") - ) - - conn.execute(do_update_stmt) - -The above will render: - -.. sourcecode:: sql - - INSERT INTO my_table (id, data) - VALUES (:id, :data) - ON CONFLICT id DO UPDATE SET data=:data_2 - -.. seealso:: - - :ref:`postgresql_insert_on_conflict` - -:ticket:`3529` - -.. _change_3499_postgresql: - -ARRAY and JSON types now correctly specify "unhashable" -------------------------------------------------------- - -As described in :ref:`change_3499`, the ORM relies upon being able to -produce a hash function for column values when a query's selected entities -mixes full ORM entities with column expressions. The ``hashable=False`` -flag is now correctly set on all of PG's "data structure" types, including -:class:`_postgresql.ARRAY` and :class:`_postgresql.JSON`. -The :class:`_postgresql.JSONB` and :class:`.HSTORE` -types already included this flag. For :class:`_postgresql.ARRAY`, -this is conditional based on the :paramref:`.postgresql.ARRAY.as_tuple` -flag, however it should no longer be necessary to set this flag -in order to have an array value present in a composed ORM row. - -.. seealso:: - - :ref:`change_3499` - - :ref:`change_3503` - -:ticket:`3499` - -.. _change_3503: - -Correct SQL Types are Established from Indexed Access of ARRAY, JSON, HSTORE ----------------------------------------------------------------------------- - -For all three of :class:`_postgresql.ARRAY`, :class:`_postgresql.JSON` and :class:`.HSTORE`, -the SQL type assigned to the expression returned by indexed access, e.g. -``col[someindex]``, should be correct in all cases. - -This includes: - -* The SQL type assigned to indexed access of an :class:`_postgresql.ARRAY` takes into - account the number of dimensions configured. An :class:`_postgresql.ARRAY` with three - dimensions will return a SQL expression with a type of :class:`_postgresql.ARRAY` of - one less dimension. Given a column with type ``ARRAY(Integer, dimensions=3)``, - we can now perform this expression:: - - int_expr = col[5][6][7] # returns an Integer expression object - - Previously, the indexed access to ``col[5]`` would return an expression of - type :class:`.Integer` where we could no longer perform indexed access - for the remaining dimensions, unless we used :func:`.cast` or :func:`.type_coerce`. - -* The :class:`_postgresql.JSON` and :class:`_postgresql.JSONB` types now mirror what PostgreSQL - itself does for indexed access. This means that all indexed access for - a :class:`_postgresql.JSON` or :class:`_postgresql.JSONB` type returns an expression that itself - is *always* :class:`_postgresql.JSON` or :class:`_postgresql.JSONB` itself, unless the - :attr:`~.postgresql.JSON.Comparator.astext` modifier is used. This means that whether - the indexed access of the JSON structure ultimately refers to a string, - list, number, or other JSON structure, PostgreSQL always considers it - to be JSON itself unless it is explicitly cast differently. Like - the :class:`_postgresql.ARRAY` type, this means that it is now straightforward - to produce JSON expressions with multiple levels of indexed access:: - - json_expr = json_col["key1"]["attr1"][5] - -* The "textual" type that is returned by indexed access of :class:`.HSTORE` - as well as the "textual" type that is returned by indexed access of - :class:`_postgresql.JSON` and :class:`_postgresql.JSONB` in conjunction with the - :attr:`~.postgresql.JSON.Comparator.astext` modifier is now configurable; it defaults - to :class:`_expression.TextClause` in both cases but can be set to a user-defined - type using the :paramref:`.postgresql.JSON.astext_type` or - :paramref:`.postgresql.HSTORE.text_type` parameters. - -.. seealso:: - - :ref:`change_3503_cast` - -:ticket:`3499` -:ticket:`3487` - -.. _change_3503_cast: - -The JSON cast() operation now requires ``.astext`` is called explicitly ------------------------------------------------------------------------ - -As part of the changes in :ref:`change_3503`, the workings of the -:meth:`_expression.ColumnElement.cast` operator on :class:`_postgresql.JSON` and -:class:`_postgresql.JSONB` no longer implicitly invoke the -:attr:`.postgresql.JSON.Comparator.astext` modifier; PostgreSQL's JSON/JSONB types -support CAST operations to each other without the "astext" aspect. - -This means that in most cases, an application that was doing this:: - - expr = json_col["somekey"].cast(Integer) - -Will now need to change to this:: - - expr = json_col["somekey"].astext.cast(Integer) - -.. _change_2729: - -ARRAY with ENUM will now emit CREATE TYPE for the ENUM ------------------------------------------------------- - -A table definition like the following will now emit CREATE TYPE -as expected:: - - enum = Enum( - "manager", - "place_admin", - "carwash_admin", - "parking_admin", - "service_admin", - "tire_admin", - "mechanic", - "carwasher", - "tire_mechanic", - name="work_place_roles", - ) - - - class WorkPlacement(Base): - __tablename__ = "work_placement" - id = Column(Integer, primary_key=True) - roles = Column(ARRAY(enum)) - - - e = create_engine("postgresql://scott:tiger@localhost/test", echo=True) - Base.metadata.create_all(e) - -emits: - -.. sourcecode:: sql - - CREATE TYPE work_place_roles AS ENUM ( - 'manager', 'place_admin', 'carwash_admin', 'parking_admin', - 'service_admin', 'tire_admin', 'mechanic', 'carwasher', - 'tire_mechanic') - - CREATE TABLE work_placement ( - id SERIAL NOT NULL, - roles work_place_roles[], - PRIMARY KEY (id) - ) - - -:ticket:`2729` - -Check constraints now reflect ------------------------------ - -The PostgreSQL dialect now supports reflection of CHECK constraints -both within the method :meth:`_reflection.Inspector.get_check_constraints` as well -as within :class:`_schema.Table` reflection within the :attr:`_schema.Table.constraints` -collection. - -"Plain" and "Materialized" views can be inspected separately ------------------------------------------------------------- - -The new argument :paramref:`.PGInspector.get_view_names.include` -allows specification of which sub-types of views should be returned:: - - from sqlalchemy import inspect - - insp = inspect(engine) - - plain_views = insp.get_view_names(include="plain") - all_views = insp.get_view_names(include=("plain", "materialized")) - -:ticket:`3588` - - -Added tablespace option to Index --------------------------------- - -The :class:`.Index` object now accepts the argument ``postgresql_tablespace`` -in order to specify TABLESPACE, the same way as accepted by the -:class:`_schema.Table` object. - -.. seealso:: - - :ref:`postgresql_index_storage` - -:ticket:`3720` - -Support for PyGreSQL --------------------- - -The `PyGreSQL `_ DBAPI is now supported. - - -The "postgres" module is removed --------------------------------- - -The ``sqlalchemy.dialects.postgres`` module, long deprecated, is -removed; this has emitted a warning for many years and projects -should be calling upon ``sqlalchemy.dialects.postgresql``. -Engine URLs of the form ``postgres://`` will still continue to function, -however. - -Support for FOR UPDATE SKIP LOCKED / FOR NO KEY UPDATE / FOR KEY SHARE ------------------------------------------------------------------------ - -The new parameters :paramref:`.GenerativeSelect.with_for_update.skip_locked` -and :paramref:`.GenerativeSelect.with_for_update.key_share` -in both Core and ORM apply a modification to a "SELECT...FOR UPDATE" -or "SELECT...FOR SHARE" query on the PostgreSQL backend: - -* SELECT FOR NO KEY UPDATE:: - - stmt = select([table]).with_for_update(key_share=True) - -* SELECT FOR UPDATE SKIP LOCKED:: - - stmt = select([table]).with_for_update(skip_locked=True) - -* SELECT FOR KEY SHARE:: - - stmt = select([table]).with_for_update(read=True, key_share=True) - -Dialect Improvements and Changes - MySQL -======================================== - -.. _change_3547: - -MySQL JSON Support ------------------- - -A new type :class:`.mysql.JSON` is added to the MySQL dialect supporting -the JSON type newly added to MySQL 5.7. This type provides both persistence -of JSON as well as rudimentary indexed-access using the ``JSON_EXTRACT`` -function internally. An indexable JSON column that works across MySQL -and PostgreSQL can be achieved by using the :class:`_types.JSON` datatype -common to both MySQL and PostgreSQL. - -.. seealso:: - - :ref:`change_3619` - -:ticket:`3547` - -.. _change_3332: - -Added support for AUTOCOMMIT "isolation level" ----------------------------------------------- - -The MySQL dialect now accepts the value "AUTOCOMMIT" for the -:paramref:`_sa.create_engine.isolation_level` and -:paramref:`.Connection.execution_options.isolation_level` -parameters:: - - connection = engine.connect() - connection = connection.execution_options(isolation_level="AUTOCOMMIT") - -The isolation level makes use of the various "autocommit" attributes -provided by most MySQL DBAPIs. - -:ticket:`3332` - -.. _change_mysql_3216: - -No more generation of an implicit KEY for composite primary key w/ AUTO_INCREMENT ---------------------------------------------------------------------------------- - -The MySQL dialect had the behavior such that if a composite primary key -on an InnoDB table featured AUTO_INCREMENT on one of its columns which was -not the first column, e.g.:: - - t = Table( - "some_table", - metadata, - Column("x", Integer, primary_key=True, autoincrement=False), - Column("y", Integer, primary_key=True, autoincrement=True), - mysql_engine="InnoDB", - ) - -DDL such as the following would be generated: - -.. sourcecode:: sql - - CREATE TABLE some_table ( - x INTEGER NOT NULL, - y INTEGER NOT NULL AUTO_INCREMENT, - PRIMARY KEY (x, y), - KEY idx_autoinc_y (y) - )ENGINE=InnoDB - -Note the above "KEY" with an auto-generated name; this is a change that -found its way into the dialect many years ago in response to the issue that -the AUTO_INCREMENT would otherwise fail on InnoDB without this additional KEY. - -This workaround has been removed and replaced with the much better system -of just stating the AUTO_INCREMENT column *first* within the primary key: - -.. sourcecode:: sql - - CREATE TABLE some_table ( - x INTEGER NOT NULL, - y INTEGER NOT NULL AUTO_INCREMENT, - PRIMARY KEY (y, x) - )ENGINE=InnoDB - -To maintain explicit control of the ordering of primary key columns, -use the :class:`.PrimaryKeyConstraint` construct explicitly (1.1.0b2) -(along with a KEY for the autoincrement column as required by MySQL), e.g.:: - - t = Table( - "some_table", - metadata, - Column("x", Integer, primary_key=True), - Column("y", Integer, primary_key=True, autoincrement=True), - PrimaryKeyConstraint("x", "y"), - UniqueConstraint("y"), - mysql_engine="InnoDB", - ) - -Along with the change :ref:`change_3216`, composite primary keys with -or without auto increment are now easier to specify; -:paramref:`_schema.Column.autoincrement` -now defaults to the value ``"auto"`` and the ``autoincrement=False`` -directives are no longer needed:: - - t = Table( - "some_table", - metadata, - Column("x", Integer, primary_key=True), - Column("y", Integer, primary_key=True, autoincrement=True), - mysql_engine="InnoDB", - ) - -Dialect Improvements and Changes - SQLite -========================================= - -.. _change_3634: - -Right-nested join workaround lifted for SQLite version 3.7.16 -------------------------------------------------------------- - -In version 0.9, the feature introduced by :ref:`feature_joins_09` went -through lots of effort to support rewriting of joins on SQLite to always -use subqueries in order to achieve a "right-nested-join" effect, as -SQLite has not supported this syntax for many years. Ironically, -the version of SQLite noted in that migration note, 3.7.15.2, was the *last* -version of SQLite to actually have this limitation! The next release was -3.7.16 and support for right nested joins was quietly added. In 1.1, the work -to identify the specific SQLite version and source commit where this change -was made was done (SQLite's changelog refers to it with the cryptic phrase "Enhance -the query optimizer to exploit transitive join constraints" without linking -to any issue number, change number, or further explanation), and the workarounds -present in this change are now lifted for SQLite when the DBAPI reports -that version 3.7.16 or greater is in effect. - -:ticket:`3634` - -.. _change_3633: - -Dotted column names workaround lifted for SQLite version 3.10.0 ---------------------------------------------------------------- - -The SQLite dialect has long had a workaround for an issue where the database -driver does not report the correct column names for some SQL result sets, in -particular when UNION is used. The workaround is detailed at -:ref:`sqlite_dotted_column_names`, and requires that SQLAlchemy assume that any -column name with a dot in it is actually a ``tablename.columnname`` combination -delivered via this buggy behavior, with an option to turn it off via the -``sqlite_raw_colnames`` execution option. - -As of SQLite version 3.10.0, the bug in UNION and other queries has been fixed; -like the change described in :ref:`change_3634`, SQLite's changelog only -identifies it cryptically as "Added the colUsed field to sqlite3_index_info for -use by the sqlite3_module.xBestIndex method", however SQLAlchemy's translation -of these dotted column names is no longer required with this version, so is -turned off when version 3.10.0 or greater is detected. - -Overall, the SQLAlchemy :class:`_engine.ResultProxy` as of the 1.0 series relies much -less on column names in result sets when delivering results for Core and ORM -SQL constructs, so the importance of this issue was already lessened in any -case. - -:ticket:`3633` - -.. _change_sqlite_schemas: - -Improved Support for Remote Schemas ------------------------------------ -The SQLite dialect now implements :meth:`_reflection.Inspector.get_schema_names` -and additionally has improved support for tables and indexes that are -created and reflected from a remote schema, which in SQLite is a -database that is assigned a name via the ``ATTACH`` statement; previously, -the``CREATE INDEX`` DDL didn't work correctly for a schema-bound table -and the :meth:`_reflection.Inspector.get_foreign_keys` method will now indicate the -given schema in the results. Cross-schema foreign keys aren't supported. - -.. _change_3629: - -Reflection of the name of PRIMARY KEY constraints -------------------------------------------------- - -The SQLite backend now takes advantage of the "sqlite_master" view -of SQLite in order to extract the name of the primary key constraint -of a table from the original DDL, in the same way that is achieved for -foreign key constraints in recent SQLAlchemy versions. - -:ticket:`3629` - -Check constraints now reflect ------------------------------ - -The SQLite dialect now supports reflection of CHECK constraints -both within the method :meth:`_reflection.Inspector.get_check_constraints` as well -as within :class:`_schema.Table` reflection within the :attr:`_schema.Table.constraints` -collection. - -ON DELETE and ON UPDATE foreign key phrases now reflect -------------------------------------------------------- - -The :class:`_reflection.Inspector` will now include ON DELETE and ON UPDATE -phrases from foreign key constraints on the SQLite dialect, and the -:class:`_schema.ForeignKeyConstraint` object as reflected as part of a -:class:`_schema.Table` will also indicate these phrases. - -Dialect Improvements and Changes - SQL Server -============================================= - -.. _change_3534: - -Added transaction isolation level support for SQL Server --------------------------------------------------------- - -All SQL Server dialects support transaction isolation level settings -via the :paramref:`_sa.create_engine.isolation_level` and -:paramref:`.Connection.execution_options.isolation_level` -parameters. The four standard levels are supported as well as -``SNAPSHOT``:: - - engine = create_engine( - "mssql+pyodbc://scott:tiger@ms_2008", isolation_level="REPEATABLE READ" - ) - -.. seealso:: - - :ref:`mssql_isolation_level` - -:ticket:`3534` - -.. _change_3504: - -String / varlength types no longer represent "max" explicitly on reflection ---------------------------------------------------------------------------- - -When reflecting a type such as :class:`.String`, :class:`_expression.TextClause`, etc. -which includes a length, an "un-lengthed" type under SQL Server would -copy the "length" parameter as the value ``"max"``:: - - >>> from sqlalchemy import create_engine, inspect - >>> engine = create_engine("mssql+pyodbc://scott:tiger@ms_2008", echo=True) - >>> engine.execute("create table s (x varchar(max), y varbinary(max))") - >>> insp = inspect(engine) - >>> for col in insp.get_columns("s"): - ... print(col["type"].__class__, col["type"].length) - max - max - -The "length" parameter in the base types is expected to be an integer value -or None only; None indicates unbounded length which the SQL Server dialect -interprets as "max". The fix then is so that these lengths come -out as None, so that the type objects work in non-SQL Server contexts:: - - >>> for col in insp.get_columns("s"): - ... print(col["type"].__class__, col["type"].length) - None - None - -Applications which may have been relying on a direct comparison of the "length" -value to the string "max" should consider the value of ``None`` to mean -the same thing. - -:ticket:`3504` - -Support for "non clustered" on primary key to allow clustered elsewhere ------------------------------------------------------------------------ - -The ``mssql_clustered`` flag available on :class:`.UniqueConstraint`, -:class:`.PrimaryKeyConstraint`, :class:`.Index` now defaults to ``None``, and -can be set to False which will render the NONCLUSTERED keyword in particular -for a primary key, allowing a different index to be used as "clustered". - -.. seealso:: - - :ref:`mssql_indexes` - -.. _change_3434: - -The legacy_schema_aliasing flag is now set to False ---------------------------------------------------- - -SQLAlchemy 1.0.5 introduced the ``legacy_schema_aliasing`` flag to the -MSSQL dialect, allowing so-called "legacy mode" aliasing to be turned off. -This aliasing attempts to turn schema-qualified tables into aliases; -given a table such as:: - - account_table = Table( - "account", - metadata, - Column("id", Integer, primary_key=True), - Column("info", String(100)), - schema="customer_schema", - ) - -The legacy mode of behavior will attempt to turn a schema-qualified table -name into an alias: - -.. sourcecode:: pycon+sql - - >>> eng = create_engine("mssql+pymssql://mydsn", legacy_schema_aliasing=True) - >>> print(account_table.select().compile(eng)) - {printsql}SELECT account_1.id, account_1.info - FROM customer_schema.account AS account_1 - -However, this aliasing has been shown to be unnecessary and in many cases -produces incorrect SQL. - -In SQLAlchemy 1.1, the ``legacy_schema_aliasing`` flag now defaults to -False, disabling this mode of behavior and allowing the MSSQL dialect to behave -normally with schema-qualified tables. For applications which may rely -on this behavior, set the flag back to True. - - -:ticket:`3434` - -Dialect Improvements and Changes - Oracle -========================================= - -Support for SKIP LOCKED ------------------------ - -The new parameter :paramref:`.GenerativeSelect.with_for_update.skip_locked` -in both Core and ORM will generate the "SKIP LOCKED" suffix for a -"SELECT...FOR UPDATE" or "SELECT.. FOR SHARE" query. diff --git a/doc/build/changelog/migration_12.rst b/doc/build/changelog/migration_12.rst deleted file mode 100644 index 454b17f12a5..00000000000 --- a/doc/build/changelog/migration_12.rst +++ /dev/null @@ -1,1851 +0,0 @@ -============================= -What's New in SQLAlchemy 1.2? -============================= - -.. admonition:: About this Document - - This document describes changes between SQLAlchemy version 1.1 - and SQLAlchemy version 1.2. - - -Introduction -============ - -This guide introduces what's new in SQLAlchemy version 1.2, -and also documents changes which affect users migrating -their applications from the 1.1 series of SQLAlchemy to 1.2. - -Please carefully review the sections on behavioral changes for -potentially backwards-incompatible changes in behavior. - -Platform Support -================ - -Targeting Python 2.7 and Up ---------------------------- - -SQLAlchemy 1.2 now moves the minimum Python version to 2.7, no longer -supporting 2.6. New language features are expected to be merged -into the 1.2 series that were not supported in Python 2.6. For Python 3 support, -SQLAlchemy is currently tested on versions 3.5 and 3.6. - - -New Features and Improvements - ORM -=================================== - -.. _change_3954: - -"Baked" loading now the default for lazy loads ----------------------------------------------- - -The :mod:`sqlalchemy.ext.baked` extension, first introduced in the 1.0 series, -allows for the construction of a so-called :class:`.BakedQuery` object, -which is an object that generates a :class:`_query.Query` object in conjunction -with a cache key representing the structure of the query; this cache key -is then linked to the resulting string SQL statement so that subsequent use -of another :class:`.BakedQuery` with the same structure will bypass all the -overhead of building the :class:`_query.Query` object, building the core -:func:`_expression.select` object within, as well as the compilation of the :func:`_expression.select` -into a string, cutting out well the majority of function call overhead normally -associated with constructing and emitting an ORM :class:`_query.Query` object. - -The :class:`.BakedQuery` is now used by default by the ORM when it generates -a "lazy" query for the lazy load of a :func:`_orm.relationship` construct, e.g. -that of the default ``lazy="select"`` relationship loader strategy. This -will allow for a significant reduction in function calls within the scope -of an application's use of lazy load queries to load collections and related -objects. Previously, this feature was available -in 1.0 and 1.1 through the use of a global API method or by using the -``baked_select`` strategy, it's now the only implementation for this behavior. -The feature has also been improved such that the caching can still take place -for objects that have additional loader options in effect subsequent -to the lazy load. - -The caching behavior can be disabled on a per-relationship basis using the -:paramref:`_orm.relationship.bake_queries` flag, which is available for -very unusual cases, such as a relationship that uses a custom -:class:`_query.Query` implementation that's not compatible with caching. - - -:ticket:`3954` - -.. _change_3944: - -New "selectin" eager loading, loads all collections at once using IN --------------------------------------------------------------------- - -A new eager loader called "selectin" loading is added, which in many ways -is similar to "subquery" loading, however produces a simpler SQL statement -that is cacheable as well as more efficient. - -Given a query as below:: - - q = ( - session.query(User) - .filter(User.name.like("%ed%")) - .options(subqueryload(User.addresses)) - ) - -The SQL produced would be the query against ``User`` followed by the -subqueryload for ``User.addresses`` (note the parameters are also listed): - -.. sourcecode:: sql - - SELECT users.id AS users_id, users.name AS users_name - FROM users - WHERE users.name LIKE ? - ('%ed%',) - - SELECT addresses.id AS addresses_id, - addresses.user_id AS addresses_user_id, - addresses.email_address AS addresses_email_address, - anon_1.users_id AS anon_1_users_id - FROM (SELECT users.id AS users_id - FROM users - WHERE users.name LIKE ?) AS anon_1 - JOIN addresses ON anon_1.users_id = addresses.user_id - ORDER BY anon_1.users_id - ('%ed%',) - -With "selectin" loading, we instead get a SELECT that refers to the -actual primary key values loaded in the parent query:: - - q = ( - session.query(User) - .filter(User.name.like("%ed%")) - .options(selectinload(User.addresses)) - ) - -Produces: - -.. sourcecode:: sql - - SELECT users.id AS users_id, users.name AS users_name - FROM users - WHERE users.name LIKE ? - ('%ed%',) - - SELECT users_1.id AS users_1_id, - addresses.id AS addresses_id, - addresses.user_id AS addresses_user_id, - addresses.email_address AS addresses_email_address - FROM users AS users_1 - JOIN addresses ON users_1.id = addresses.user_id - WHERE users_1.id IN (?, ?) - ORDER BY users_1.id - (1, 3) - -The above SELECT statement includes these advantages: - -* It doesn't use a subquery, just an INNER JOIN, meaning it will perform - much better on a database like MySQL that doesn't like subqueries - -* Its structure is independent of the original query; in conjunction with the - new :ref:`expanding IN parameter system ` we can in most cases - use the "baked" query to cache the string SQL, reducing per-query overhead - significantly - -* Because the query only fetches for a given list of primary key identifiers, - "selectin" loading is potentially compatible with :meth:`_query.Query.yield_per` to - operate on chunks of a SELECT result at a time, provided that the - database driver allows for multiple, simultaneous cursors (SQLite, PostgreSQL; - **not** MySQL drivers or SQL Server ODBC drivers). Neither joined eager - loading nor subquery eager loading are compatible with :meth:`_query.Query.yield_per`. - -The disadvantages of selectin eager loading are potentially large SQL -queries, with large lists of IN parameters. The list of IN parameters themselves -are chunked in groups of 500, so a result set of more than 500 lead objects -will have more additional "SELECT IN" queries following. Also, support -for composite primary keys depends on the database's ability to use -tuples with IN, e.g. -``(table.column_one, table_column_two) IN ((?, ?), (?, ?) (?, ?))``. -Currently, PostgreSQL and MySQL are known to be compatible with this syntax, -SQLite is not. - -.. seealso:: - - :ref:`selectin_eager_loading` - -:ticket:`3944` - -.. _change_3948: - -"selectin" polymorphic loading, loads subclasses using separate IN queries --------------------------------------------------------------------------- - -Along similar lines as the "selectin" relationship loading feature just -described at :ref:`change_3944` is "selectin" polymorphic loading. This -is a polymorphic loading feature tailored primarily towards joined eager -loading that allows the loading of the base entity to proceed with a simple -SELECT statement, but then the attributes of the additional subclasses -are loaded with additional SELECT statements: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy.orm import selectin_polymorphic - - >>> query = session.query(Employee).options( - ... selectin_polymorphic(Employee, [Manager, Engineer]) - ... ) - - >>> query.all() - {execsql}SELECT - employee.id AS employee_id, - employee.name AS employee_name, - employee.type AS employee_type - FROM employee - () - - SELECT - engineer.id AS engineer_id, - employee.id AS employee_id, - employee.type AS employee_type, - engineer.engineer_name AS engineer_engineer_name - FROM employee JOIN engineer ON employee.id = engineer.id - WHERE employee.id IN (?, ?) ORDER BY employee.id - (1, 2) - - SELECT - manager.id AS manager_id, - employee.id AS employee_id, - employee.type AS employee_type, - manager.manager_name AS manager_manager_name - FROM employee JOIN manager ON employee.id = manager.id - WHERE employee.id IN (?) ORDER BY employee.id - (3,) - -.. seealso:: - - :ref:`polymorphic_selectin` - -:ticket:`3948` - -.. _change_3058: - -ORM attributes that can receive ad-hoc SQL expressions ------------------------------------------------------- - -A new ORM attribute type :func:`_orm.query_expression` is added which -is similar to :func:`_orm.deferred`, except its SQL expression -is determined at query time using a new option :func:`_orm.with_expression`; -if not specified, the attribute defaults to ``None``:: - - from sqlalchemy.orm import query_expression - from sqlalchemy.orm import with_expression - - - class A(Base): - __tablename__ = "a" - id = Column(Integer, primary_key=True) - x = Column(Integer) - y = Column(Integer) - - # will be None normally... - expr = query_expression() - - - # but let's give it x + y - a1 = session.query(A).options(with_expression(A.expr, A.x + A.y)).first() - print(a1.expr) - -.. seealso:: - - :ref:`mapper_querytime_expression` - -:ticket:`3058` - -.. _change_orm_959: - -ORM Support of multiple-table deletes -------------------------------------- - -The ORM :meth:`_query.Query.delete` method supports multiple-table criteria -for DELETE, as introduced in :ref:`change_959`. The feature works -in the same manner as multiple-table criteria for UPDATE, first -introduced in 0.8 and described at :ref:`change_orm_2365`. - -Below, we emit a DELETE against ``SomeEntity``, adding -a FROM clause (or equivalent, depending on backend) -against ``SomeOtherEntity``:: - - query(SomeEntity).filter(SomeEntity.id == SomeOtherEntity.id).filter( - SomeOtherEntity.foo == "bar" - ).delete() - -.. seealso:: - - :ref:`change_959` - -:ticket:`959` - -.. _change_3229: - -Support for bulk updates of hybrids, composites ------------------------------------------------ - -Both hybrid attributes (e.g. :mod:`sqlalchemy.ext.hybrid`) as well as composite -attributes (:ref:`mapper_composite`) now support being used in the -SET clause of an UPDATE statement when using :meth:`_query.Query.update`. - -For hybrids, simple expressions can be used directly, or the new decorator -:meth:`.hybrid_property.update_expression` can be used to break a value -into multiple columns/expressions:: - - class Person(Base): - # ... - - first_name = Column(String(10)) - last_name = Column(String(10)) - - @hybrid.hybrid_property - def name(self): - return self.first_name + " " + self.last_name - - @name.expression - def name(cls): - return func.concat(cls.first_name, " ", cls.last_name) - - @name.update_expression - def name(cls, value): - f, l = value.split(" ", 1) - return [(cls.first_name, f), (cls.last_name, l)] - -Above, an UPDATE can be rendered using:: - - session.query(Person).filter(Person.id == 5).update({Person.name: "Dr. No"}) - -Similar functionality is available for composites, where composite values -will be broken out into their individual columns for bulk UPDATE:: - - session.query(Vertex).update({Edge.start: Point(3, 4)}) - -.. seealso:: - - :ref:`hybrid_bulk_update` - -.. _change_3911_3912: - -Hybrid attributes support reuse among subclasses, redefinition of @getter -------------------------------------------------------------------------- - -The :class:`sqlalchemy.ext.hybrid.hybrid_property` class now supports -calling mutators like ``@setter``, ``@expression`` etc. multiple times -across subclasses, and now provides a ``@getter`` mutator, so that -a particular hybrid can be repurposed across subclasses or other -classes. This now is similar to the behavior of ``@property`` in standard -Python:: - - class FirstNameOnly(Base): - # ... - - first_name = Column(String) - - @hybrid_property - def name(self): - return self.first_name - - @name.setter - def name(self, value): - self.first_name = value - - - class FirstNameLastName(FirstNameOnly): - # ... - - last_name = Column(String) - - @FirstNameOnly.name.getter - def name(self): - return self.first_name + " " + self.last_name - - @name.setter - def name(self, value): - self.first_name, self.last_name = value.split(" ", maxsplit=1) - - @name.expression - def name(cls): - return func.concat(cls.first_name, " ", cls.last_name) - -Above, the ``FirstNameOnly.name`` hybrid is referenced by the -``FirstNameLastName`` subclass in order to repurpose it specifically to the -new subclass. This is achieved by copying the hybrid object to a new one -within each call to ``@getter``, ``@setter``, as well as in all other -mutator methods like ``@expression``, leaving the previous hybrid's definition -intact. Previously, methods like ``@setter`` would modify the existing -hybrid in-place, interfering with the definition on the superclass. - -.. note:: Be sure to read the documentation at :ref:`hybrid_reuse_subclass` - for important notes regarding how to override - :meth:`.hybrid_property.expression` - and :meth:`.hybrid_property.comparator`, as a special qualifier - :attr:`.hybrid_property.overrides` may be necessary to avoid name - conflicts with :class:`.QueryableAttribute` in some cases. - -.. note:: This change in ``@hybrid_property`` implies that when adding setters and - other state to a ``@hybrid_property``, the **methods must retain the name - of the original hybrid**, else the new hybrid with the additional state will - be present on the class as the non-matching name. This is the same behavior - as that of the ``@property`` construct that is part of standard Python:: - - class FirstNameOnly(Base): - @hybrid_property - def name(self): - return self.first_name - - # WRONG - will raise AttributeError: can't set attribute when - # assigning to .name - @name.setter - def _set_name(self, value): - self.first_name = value - - - class FirstNameOnly(Base): - @hybrid_property - def name(self): - return self.first_name - - # CORRECT - note regular Python @property works the same way - @name.setter - def name(self, value): - self.first_name = value - -:ticket:`3911` - -:ticket:`3912` - -.. _change_3896_event: - -New bulk_replace event ----------------------- - -To suit the validation use case described in :ref:`change_3896_validates`, -a new :meth:`.AttributeEvents.bulk_replace` method is added, which is -called in conjunction with the :meth:`.AttributeEvents.append` and -:meth:`.AttributeEvents.remove` events. "bulk_replace" is called before -"append" and "remove" so that the collection can be modified ahead of comparison -to the existing collection. After that, individual items -are appended to a new target collection, firing off the "append" -event for items new to the collection, as was the previous behavior. -Below illustrates both "bulk_replace" and -"append" at the same time, including that "append" will receive an object -already handled by "bulk_replace" if collection assignment is used. -A new symbol :attr:`~.attributes.OP_BULK_REPLACE` may be used to determine -if this "append" event is the second part of a bulk replace:: - - from sqlalchemy.orm.attributes import OP_BULK_REPLACE - - - @event.listens_for(SomeObject.collection, "bulk_replace") - def process_collection(target, values, initiator): - values[:] = [_make_value(value) for value in values] - - - @event.listens_for(SomeObject.collection, "append", retval=True) - def process_collection(target, value, initiator): - # make sure bulk_replace didn't already do it - if initiator is None or initiator.op is not OP_BULK_REPLACE: - return _make_value(value) - else: - return value - -:ticket:`3896` - -.. _change_3303: - -New "modified" event handler for sqlalchemy.ext.mutable -------------------------------------------------------- - -A new event handler :meth:`.AttributeEvents.modified` is added, which is -triggered corresponding to calls to the :func:`.attributes.flag_modified` -method, which is normally called from the :mod:`sqlalchemy.ext.mutable` -extension:: - - from sqlalchemy.ext.declarative import declarative_base - from sqlalchemy.ext.mutable import MutableDict - from sqlalchemy import event - - Base = declarative_base() - - - class MyDataClass(Base): - __tablename__ = "my_data" - id = Column(Integer, primary_key=True) - data = Column(MutableDict.as_mutable(JSONEncodedDict)) - - - @event.listens_for(MyDataClass.data, "modified") - def modified_json(instance): - print("json value modified:", instance.data) - -Above, the event handler will be triggered when an in-place change to the -``.data`` dictionary occurs. - -:ticket:`3303` - -.. _change_3991: - -Added "for update" arguments to Session.refresh ------------------------------------------------- - -Added new argument :paramref:`.Session.refresh.with_for_update` to the -:meth:`.Session.refresh` method. When the :meth:`_query.Query.with_lockmode` -method were deprecated in favor of :meth:`_query.Query.with_for_update`, -the :meth:`.Session.refresh` method was never updated to reflect -the new option:: - - session.refresh(some_object, with_for_update=True) - -The :paramref:`.Session.refresh.with_for_update` argument accepts a dictionary -of options that will be passed as the same arguments which are sent to -:meth:`_query.Query.with_for_update`:: - - session.refresh(some_objects, with_for_update={"read": True}) - -The new parameter supersedes the :paramref:`.Session.refresh.lockmode` -parameter. - -:ticket:`3991` - -.. _change_3853: - -In-place mutation operators work for MutableSet, MutableList ------------------------------------------------------------- - -Implemented the in-place mutation operators ``__ior__``, ``__iand__``, -``__ixor__`` and ``__isub__`` for :class:`.mutable.MutableSet` and ``__iadd__`` -for :class:`.mutable.MutableList`. While these -methods would successfully update the collection previously, they would -not correctly fire off change events. The operators mutate the collection -as before but additionally emit the correct change event so that the change -becomes part of the next flush process:: - - model = session.query(MyModel).first() - model.json_set &= {1, 3} - -:ticket:`3853` - -.. _change_3769: - -AssociationProxy any(), has(), contains() work with chained association proxies -------------------------------------------------------------------------------- - -The :meth:`.AssociationProxy.any`, :meth:`.AssociationProxy.has` -and :meth:`.AssociationProxy.contains` comparison methods now support -linkage to an attribute that is -itself also an :class:`.AssociationProxy`, recursively. Below, ``A.b_values`` -is an association proxy that links to ``AtoB.bvalue``, which is -itself an association proxy onto ``B``:: - - class A(Base): - __tablename__ = "a" - id = Column(Integer, primary_key=True) - - b_values = association_proxy("atob", "b_value") - c_values = association_proxy("atob", "c_value") - - - class B(Base): - __tablename__ = "b" - id = Column(Integer, primary_key=True) - a_id = Column(ForeignKey("a.id")) - value = Column(String) - - c = relationship("C") - - - class C(Base): - __tablename__ = "c" - id = Column(Integer, primary_key=True) - b_id = Column(ForeignKey("b.id")) - value = Column(String) - - - class AtoB(Base): - __tablename__ = "atob" - - a_id = Column(ForeignKey("a.id"), primary_key=True) - b_id = Column(ForeignKey("b.id"), primary_key=True) - - a = relationship("A", backref="atob") - b = relationship("B", backref="atob") - - b_value = association_proxy("b", "value") - c_value = association_proxy("b", "c") - -We can query on ``A.b_values`` using :meth:`.AssociationProxy.contains` to -query across the two proxies ``A.b_values``, ``AtoB.b_value``: - -.. sourcecode:: pycon+sql - - >>> s.query(A).filter(A.b_values.contains("hi")).all() - {execsql}SELECT a.id AS a_id - FROM a - WHERE EXISTS (SELECT 1 - FROM atob - WHERE a.id = atob.a_id AND (EXISTS (SELECT 1 - FROM b - WHERE b.id = atob.b_id AND b.value = :value_1))) - -Similarly, we can query on ``A.c_values`` using :meth:`.AssociationProxy.any` -to query across the two proxies ``A.c_values``, ``AtoB.c_value``: - -.. sourcecode:: pycon+sql - - >>> s.query(A).filter(A.c_values.any(value="x")).all() - {execsql}SELECT a.id AS a_id - FROM a - WHERE EXISTS (SELECT 1 - FROM atob - WHERE a.id = atob.a_id AND (EXISTS (SELECT 1 - FROM b - WHERE b.id = atob.b_id AND (EXISTS (SELECT 1 - FROM c - WHERE b.id = c.b_id AND c.value = :value_1))))) - -:ticket:`3769` - -.. _change_4137: - -Identity key enhancements to support sharding ---------------------------------------------- - -The identity key structure used by the ORM now contains an additional -member, so that two identical primary keys that originate from different -contexts can co-exist within the same identity map. - -The example at :ref:`examples_sharding` has been updated to illustrate this -behavior. The example shows a sharded class ``WeatherLocation`` that -refers to a dependent ``WeatherReport`` object, where the ``WeatherReport`` -class is mapped to a table that stores a simple integer primary key. Two -``WeatherReport`` objects from different databases may have the same -primary key value. The example now illustrates that a new ``identity_token`` -field tracks this difference so that the two objects can co-exist in the -same identity map:: - - tokyo = WeatherLocation("Asia", "Tokyo") - newyork = WeatherLocation("North America", "New York") - - tokyo.reports.append(Report(80.0)) - newyork.reports.append(Report(75)) - - sess = create_session() - - sess.add_all([tokyo, newyork, quito]) - - sess.commit() - - # the Report class uses a simple integer primary key. So across two - # databases, a primary key will be repeated. The "identity_token" tracks - # in memory that these two identical primary keys are local to different - # databases. - - newyork_report = newyork.reports[0] - tokyo_report = tokyo.reports[0] - - assert inspect(newyork_report).identity_key == (Report, (1,), "north_america") - assert inspect(tokyo_report).identity_key == (Report, (1,), "asia") - - # the token representing the originating shard is also available directly - - assert inspect(newyork_report).identity_token == "north_america" - assert inspect(tokyo_report).identity_token == "asia" - -:ticket:`4137` - -New Features and Improvements - Core -==================================== - -.. _change_4102: - -Boolean datatype now enforces strict True/False/None values ------------------------------------------------------------ - -In version 1.1, the change described in :ref:`change_3730` produced an -unintended side effect of altering the way :class:`.Boolean` behaves when -presented with a non-integer value, such as a string. In particular, the -string value ``"0"``, which would previously result in the value ``False`` -being generated, would now produce ``True``. Making matters worse, the change -in behavior was only for some backends and not others, meaning code that sends -string ``"0"`` values to :class:`.Boolean` would break inconsistently across -backends. - -The ultimate solution to this problem is that **string values are not supported -with Boolean**, so in 1.2 a hard ``TypeError`` is raised if a non-integer / -True/False/None value is passed. Additionally, only the integer values -0 and 1 are accepted. - -To accommodate for applications that wish to have more liberal interpretation -of boolean values, the :class:`.TypeDecorator` should be used. Below -illustrates a recipe that will allow for the "liberal" behavior of the pre-1.1 -:class:`.Boolean` datatype:: - - from sqlalchemy import Boolean - from sqlalchemy import TypeDecorator - - - class LiberalBoolean(TypeDecorator): - impl = Boolean - - def process_bind_param(self, value, dialect): - if value is not None: - value = bool(int(value)) - return value - -:ticket:`4102` - -.. _change_3919: - -Pessimistic disconnection detection added to the connection pool ----------------------------------------------------------------- - -The connection pool documentation has long featured a recipe for using -the :meth:`_events.ConnectionEvents.engine_connect` engine event to emit a simple -statement on a checked-out connection to test it for liveness. The -functionality of this recipe has now been added into the connection pool -itself, when used in conjunction with an appropriate dialect. Using -the new parameter :paramref:`_sa.create_engine.pool_pre_ping`, each connection -checked out will be tested for freshness before being returned:: - - engine = create_engine("mysql+pymysql://", pool_pre_ping=True) - -While the "pre-ping" approach adds a small amount of latency to the connection -pool checkout, for a typical application that is transactionally-oriented -(which includes most ORM applications), this overhead is minimal, and -eliminates the problem of acquiring a stale connection that will raise -an error, requiring that the application either abandon or retry the operation. - -The feature does **not** accommodate for connections dropped within -an ongoing transaction or SQL operation. If an application must recover -from these as well, it would need to employ its own operation retry logic -to anticipate these errors. - - -.. seealso:: - - :ref:`pool_disconnects_pessimistic` - - -:ticket:`3919` - -.. _change_3907: - -The IN / NOT IN operator's empty collection behavior is now configurable; default expression simplified -------------------------------------------------------------------------------------------------------- - -An expression such as ``column.in_([])``, which is assumed to be false, -now produces the expression ``1 != 1`` -by default, instead of ``column != column``. This will **change the result** -of a query that is comparing a SQL expression or column that evaluates to -NULL when compared to an empty set, producing a boolean value false or true -(for NOT IN) rather than NULL. The warning that would emit under -this condition is also removed. The old behavior is available using the -:paramref:`_sa.create_engine.empty_in_strategy` parameter to -:func:`_sa.create_engine`. - -In SQL, the IN and NOT IN operators do not support comparison to a -collection of values that is explicitly empty; meaning, this syntax is -illegal: - -.. sourcecode:: sql - - mycolumn IN () - -To work around this, SQLAlchemy and other database libraries detect this -condition and render an alternative expression that evaluates to false, or -in the case of NOT IN, to true, based on the theory that "col IN ()" is always -false since nothing is in "the empty set". Typically, in order to -produce a false/true constant that is portable across databases and works -in the context of the WHERE clause, a simple tautology such as ``1 != 1`` is -used to evaluate to false and ``1 = 1`` to evaluate to true (a simple constant -"0" or "1" often does not work as the target of a WHERE clause). - -SQLAlchemy in its early days began with this approach as well, but soon it -was theorized that the SQL expression ``column IN ()`` would not evaluate to -false if the "column" were NULL; instead, the expression would produce NULL, -since "NULL" means "unknown", and comparisons to NULL in SQL usually produce -NULL. - -To simulate this result, SQLAlchemy changed from using ``1 != 1`` to -instead use th expression ``expr != expr`` for empty "IN" and ``expr = expr`` -for empty "NOT IN"; that is, instead of using a fixed value we use the -actual left-hand side of the expression. If the left-hand side of -the expression passed evaluates to NULL, then the comparison overall -also gets the NULL result instead of false or true. - -Unfortunately, users eventually complained that this expression had a very -severe performance impact on some query planners. At that point, a warning -was added when an empty IN expression was encountered, favoring that SQLAlchemy -continues to be "correct" and urging users to avoid code that generates empty -IN predicates in general, since typically they can be safely omitted. However, -this is of course burdensome in the case of queries that are built up dynamically -from input variables, where an incoming set of values might be empty. - -In recent months, the original assumptions of this decision have been -questioned. The notion that the expression "NULL IN ()" should return NULL was -only theoretical, and could not be tested since databases don't support that -syntax. However, as it turns out, you can in fact ask a relational database -what value it would return for "NULL IN ()" by simulating the empty set as -follows: - -.. sourcecode:: sql - - SELECT NULL IN (SELECT 1 WHERE 1 != 1) - -With the above test, we see that the databases themselves can't agree on -the answer. PostgreSQL, considered by most to be the most "correct" database, -returns False; because even though "NULL" represents "unknown", the "empty set" -means nothing is present, including all unknown values. On the -other hand, MySQL and MariaDB return NULL for the above expression, defaulting -to the more common behavior of "all comparisons to NULL return NULL". - -SQLAlchemy's SQL architecture is more sophisticated than it was when this -design decision was first made, so we can now allow either behavior to -be invoked at SQL string compilation time. Previously, the conversion to a -comparison expression were done at construction time, that is, the moment -the :meth:`.ColumnOperators.in_` or :meth:`.ColumnOperators.notin_` operators were invoked. -With the compilation-time behavior, the dialect itself can be instructed -to invoke either approach, that is, the "static" ``1 != 1`` comparison or the -"dynamic" ``expr != expr`` comparison. The default has been **changed** -to be the "static" comparison, since this agrees with the behavior that -PostgreSQL would have in any case and this is also what the vast majority -of users prefer. This will **change the result** of a query that is comparing -a null expression to the empty set, particularly one that is querying -for the negation ``where(~null_expr.in_([]))``, since this now evaluates to true -and not NULL. - -The behavior can now be controlled using the flag -:paramref:`_sa.create_engine.empty_in_strategy`, which defaults to the -``"static"`` setting, but may also be set to ``"dynamic"`` or -``"dynamic_warn"``, where the ``"dynamic_warn"`` setting is equivalent to the -previous behavior of emitting ``expr != expr`` as well as a performance -warning. However, it is anticipated that most users will appreciate the -"static" default. - -:ticket:`3907` - -.. _change_3953: - -Late-expanded IN parameter sets allow IN expressions with cached statements ---------------------------------------------------------------------------- - -Added a new kind of :func:`.bindparam` called "expanding". This is -for use in ``IN`` expressions where the list of elements is rendered -into individual bound parameters at statement execution time, rather -than at statement compilation time. This allows both a single bound -parameter name to be linked to an IN expression of multiple elements, -as well as allows query caching to be used with IN expressions. The -new feature allows the related features of "select in" loading and -"polymorphic in" loading to make use of the baked query extension -to reduce call overhead:: - - stmt = select([table]).where(table.c.col.in_(bindparam("foo", expanding=True))) - conn.execute(stmt, {"foo": [1, 2, 3]}) - -The feature should be regarded as **experimental** within the 1.2 series. - - -:ticket:`3953` - -.. _change_3999: - -Flattened operator precedence for comparison operators -------------------------------------------------------- - -The operator precedence for operators like IN, LIKE, equals, IS, MATCH, and -other comparison operators has been flattened into one level. This will -have the effect of more parenthesization being generated when comparison -operators are combined together, such as:: - - (column("q") == null()) != (column("y") == null()) - -Will now generate ``(q IS NULL) != (y IS NULL)`` rather than -``q IS NULL != y IS NULL``. - - -:ticket:`3999` - -.. _change_1546: - -Support for SQL Comments on Table, Column, includes DDL, reflection -------------------------------------------------------------------- - -The Core receives support for string comments associated with tables -and columns. These are specified via the :paramref:`_schema.Table.comment` and -:paramref:`_schema.Column.comment` arguments:: - - Table( - "my_table", - metadata, - Column("q", Integer, comment="the Q value"), - comment="my Q table", - ) - -Above, DDL will be rendered appropriately upon table create to associate -the above comments with the table/ column within the schema. When -the above table is autoloaded or inspected with :meth:`_reflection.Inspector.get_columns`, -the comments are included. The table comment is also available independently -using the :meth:`_reflection.Inspector.get_table_comment` method. - -Current backend support includes MySQL, PostgreSQL, and Oracle. - -:ticket:`1546` - -.. _change_959: - -Multiple-table criteria support for DELETE ------------------------------------------- - -The :class:`_expression.Delete` construct now supports multiple-table criteria, -implemented for those backends which support it, currently these are -PostgreSQL, MySQL and Microsoft SQL Server (support is also added to the -currently non-working Sybase dialect). The feature works in the same -was as that of multiple-table criteria for UPDATE, first introduced in -the 0.7 and 0.8 series. - -Given a statement as:: - - stmt = ( - users.delete() - .where(users.c.id == addresses.c.id) - .where(addresses.c.email_address.startswith("ed%")) - ) - conn.execute(stmt) - -The resulting SQL from the above statement on a PostgreSQL backend -would render as: - -.. sourcecode:: sql - - DELETE FROM users USING addresses - WHERE users.id = addresses.id - AND (addresses.email_address LIKE %(email_address_1)s || '%%') - -.. seealso:: - - :ref:`tutorial_multi_table_deletes` - -:ticket:`959` - -.. _change_2694: - -New "autoescape" option for startswith(), endswith() ----------------------------------------------------- - -The "autoescape" parameter is added to :meth:`.ColumnOperators.startswith`, -:meth:`.ColumnOperators.endswith`, :meth:`.ColumnOperators.contains`. -This parameter when set to ``True`` will automatically escape all occurrences -of ``%``, ``_`` with an escape character, which defaults to a forwards slash ``/``; -occurrences of the escape character itself are also escaped. The forwards slash -is used to avoid conflicts with settings like PostgreSQL's -``standard_confirming_strings``, whose default value changed as of PostgreSQL -9.1, and MySQL's ``NO_BACKSLASH_ESCAPES`` settings. The existing "escape" parameter -can now be used to change the autoescape character, if desired. - -.. note:: This feature has been changed as of 1.2.0 from its initial - implementation in 1.2.0b2 such that autoescape is now passed as a boolean - value, rather than a specific character to use as the escape character. - -An expression such as:: - - >>> column("x").startswith("total%score", autoescape=True) - -Renders as: - -.. sourcecode:: sql - - x LIKE :x_1 || '%' ESCAPE '/' - -Where the value of the parameter "x_1" is ``'total/%score'``. - -Similarly, an expression that has backslashes:: - - >>> column("x").startswith("total/score", autoescape=True) - -Will render the same way, with the value of the parameter "x_1" as -``'total//score'``. - - -:ticket:`2694` - -.. _change_floats_12: - -Stronger typing added to "float" datatypes ------------------------------------------- - -A series of changes allow for use of the :class:`.Float` datatype to more -strongly link itself to Python floating point values, instead of the more -generic :class:`.Numeric`. The changes are mostly related to ensuring -that Python floating point values are not erroneously coerced to -``Decimal()``, and are coerced to ``float`` if needed, on the result side, -if the application is working with plain floats. - -* A plain Python "float" value passed to a SQL expression will now be - pulled into a literal parameter with the type :class:`.Float`; previously, - the type was :class:`.Numeric`, with the default "asdecimal=True" flag, which - meant the result type would coerce to ``Decimal()``. In particular, - this would emit a confusing warning on SQLite:: - - - float_value = connection.scalar( - select([literal(4.56)]) # the "BindParameter" will now be - # Float, not Numeric(asdecimal=True) - ) - -* Math operations between :class:`.Numeric`, :class:`.Float`, and - :class:`.Integer` will now preserve the :class:`.Numeric` or :class:`.Float` - type in the resulting expression's type, including the ``asdecimal`` flag - as well as if the type should be :class:`.Float`:: - - # asdecimal flag is maintained - expr = column("a", Integer) * column("b", Numeric(asdecimal=False)) - assert expr.type.asdecimal == False - - # Float subclass of Numeric is maintained - expr = column("a", Integer) * column("b", Float()) - assert isinstance(expr.type, Float) - -* The :class:`.Float` datatype will apply the ``float()`` processor to - result values unconditionally if the DBAPI is known to support native - ``Decimal()`` mode. Some backends do not always guarantee that a floating - point number comes back as plain float and not precision numeric such - as MySQL. - -:ticket:`4017` - -:ticket:`4018` - -:ticket:`4020` - -.. change_3249: - -Support for GROUPING SETS, CUBE, ROLLUP ---------------------------------------- - -All three of GROUPING SETS, CUBE, ROLLUP are available via the -:attr:`.func` namespace. In the case of CUBE and ROLLUP, these functions -already work in previous versions, however for GROUPING SETS, a placeholder -is added to the compiler to allow for the space. All three functions -are named in the documentation now: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy import select, table, column, func, tuple_ - >>> t = table("t", column("value"), column("x"), column("y"), column("z"), column("q")) - >>> stmt = select([func.sum(t.c.value)]).group_by( - ... func.grouping_sets( - ... tuple_(t.c.x, t.c.y), - ... tuple_(t.c.z, t.c.q), - ... ) - ... ) - >>> print(stmt) - {printsql}SELECT sum(t.value) AS sum_1 - FROM t GROUP BY GROUPING SETS((t.x, t.y), (t.z, t.q)) - -:ticket:`3429` - -.. _change_4075: - -Parameter helper for multi-valued INSERT with contextual default generator --------------------------------------------------------------------------- - -A default generation function, e.g. that described at -:ref:`context_default_functions`, can look at the current parameters relevant -to the statement via the :attr:`.DefaultExecutionContext.current_parameters` -attribute. However, in the case of a :class:`_expression.Insert` construct that specifies -multiple VALUES clauses via the :meth:`_expression.Insert.values` method, the user-defined -function is called multiple times, once for each parameter set, however there -was no way to know which subset of keys in -:attr:`.DefaultExecutionContext.current_parameters` apply to that column. A -new function :meth:`.DefaultExecutionContext.get_current_parameters` is added, -which includes a keyword argument -:paramref:`.DefaultExecutionContext.get_current_parameters.isolate_multiinsert_groups` -defaulting to ``True``, which performs the extra work of delivering a sub-dictionary of -:attr:`.DefaultExecutionContext.current_parameters` which has the names -localized to the current VALUES clause being processed:: - - - def mydefault(context): - return context.get_current_parameters()["counter"] + 12 - - - mytable = Table( - "mytable", - metadata_obj, - Column("counter", Integer), - Column("counter_plus_twelve", Integer, default=mydefault, onupdate=mydefault), - ) - - stmt = mytable.insert().values([{"counter": 5}, {"counter": 18}, {"counter": 20}]) - - conn.execute(stmt) - -:ticket:`4075` - -Key Behavioral Changes - ORM -============================ - -.. _change_3934: - -The after_rollback() Session event now emits before the expiration of objects ------------------------------------------------------------------------------ - -The :meth:`.SessionEvents.after_rollback` event now has access to the attribute -state of objects before their state has been expired (e.g. the "snapshot -removal"). This allows the event to be consistent with the behavior -of the :meth:`.SessionEvents.after_commit` event which also emits before the -"snapshot" has been removed:: - - sess = Session() - - user = sess.query(User).filter_by(name="x").first() - - - @event.listens_for(sess, "after_rollback") - def after_rollback(session): - # 'user.name' is now present, assuming it was already - # loaded. previously this would raise upon trying - # to emit a lazy load. - print("user name: %s" % user.name) - - - @event.listens_for(sess, "after_commit") - def after_commit(session): - # 'user.name' is present, assuming it was already - # loaded. this is the existing behavior. - print("user name: %s" % user.name) - - - if should_rollback: - sess.rollback() - else: - sess.commit() - -Note that the :class:`.Session` will still disallow SQL from being emitted -within this event; meaning that unloaded attributes will still not be -able to load within the scope of the event. - -:ticket:`3934` - -.. _change_3891: - -Fixed issue involving single-table inheritance with ``select_from()`` ---------------------------------------------------------------------- - -The :meth:`_query.Query.select_from` method now honors the single-table inheritance -column discriminator when generating SQL; previously, only the expressions -in the query column list would be taken into account. - -Supposing ``Manager`` is a subclass of ``Employee``. A query like the following:: - - sess.query(Manager.id) - -Would generate SQL as: - -.. sourcecode:: sql - - SELECT employee.id FROM employee WHERE employee.type IN ('manager') - -However, if ``Manager`` were only specified by :meth:`_query.Query.select_from` -and not in the columns list, the discriminator would not be added:: - - sess.query(func.count(1)).select_from(Manager) - -would generate: - -.. sourcecode:: sql - - SELECT count(1) FROM employee - -With the fix, :meth:`_query.Query.select_from` now works correctly and we get: - -.. sourcecode:: sql - - SELECT count(1) FROM employee WHERE employee.type IN ('manager') - -Applications that may have been working around this by supplying the -WHERE clause manually may need to be adjusted. - -:ticket:`3891` - -.. _change_3913: - -Previous collection is no longer mutated upon replacement ---------------------------------------------------------- - -The ORM emits events whenever the members of a mapped collection change. -In the case of assigning a collection to an attribute that would replace -the previous collection, a side effect of this was that the collection -being replaced would also be mutated, which is misleading and unnecessary:: - - >>> a1, a2, a3 = Address("a1"), Address("a2"), Address("a3") - >>> user.addresses = [a1, a2] - - >>> previous_collection = user.addresses - - # replace the collection with a new one - >>> user.addresses = [a2, a3] - - >>> previous_collection - [Address('a1'), Address('a2')] - -Above, prior to the change, the ``previous_collection`` would have had the -"a1" member removed, corresponding to the member that's no longer in the -new collection. - -:ticket:`3913` - -.. _change_3896_validates: - -A @validates method receives all values on bulk-collection set before comparison --------------------------------------------------------------------------------- - -A method that uses ``@validates`` will now receive all members of a collection -during a "bulk set" operation, before comparison is applied against the -existing collection. - -Given a mapping as:: - - class A(Base): - __tablename__ = "a" - id = Column(Integer, primary_key=True) - bs = relationship("B") - - @validates("bs") - def convert_dict_to_b(self, key, value): - return B(data=value["data"]) - - - class B(Base): - __tablename__ = "b" - id = Column(Integer, primary_key=True) - a_id = Column(ForeignKey("a.id")) - data = Column(String) - -Above, we could use the validator as follows, to convert from an incoming -dictionary to an instance of ``B`` upon collection append:: - - a1 = A() - a1.bs.append({"data": "b1"}) - -However, a collection assignment would fail, since the ORM would assume -incoming objects are already instances of ``B`` as it attempts to compare them -to the existing members of the collection, before doing collection appends -which actually invoke the validator. This would make it impossible for bulk -set operations to accommodate non-ORM objects like dictionaries that needed -up-front modification:: - - a1 = A() - a1.bs = [{"data": "b1"}] - -The new logic uses the new :meth:`.AttributeEvents.bulk_replace` event to ensure -that all values are sent to the ``@validates`` function up front. - -As part of this change, this means that validators will now receive -**all** members of a collection upon bulk set, not just the members that -are new. Supposing a simple validator such as:: - - class A(Base): - # ... - - @validates("bs") - def validate_b(self, key, value): - assert value.data is not None - return value - -Above, if we began with a collection as:: - - a1 = A() - - b1, b2 = B(data="one"), B(data="two") - a1.bs = [b1, b2] - -And then, replaced the collection with one that overlaps the first:: - - b3 = B(data="three") - a1.bs = [b2, b3] - -Previously, the second assignment would trigger the ``A.validate_b`` -method only once, for the ``b3`` object. The ``b2`` object would be seen -as being already present in the collection and not validated. With the new -behavior, both ``b2`` and ``b3`` are passed to ``A.validate_b`` before passing -onto the collection. It is thus important that validation methods employ -idempotent behavior to suit such a case. - -.. seealso:: - - :ref:`change_3896_event` - -:ticket:`3896` - -.. _change_3753: - -Use flag_dirty() to mark an object as "dirty" without any attribute changing ----------------------------------------------------------------------------- - -An exception is now raised if the :func:`.attributes.flag_modified` function -is used to mark an attribute as modified that isn't actually loaded:: - - a1 = A(data="adf") - s.add(a1) - - s.flush() - - # expire, similarly as though we said s.commit() - s.expire(a1, "data") - - # will raise InvalidRequestError - attributes.flag_modified(a1, "data") - -This because the flush process will most likely fail in any case if the -attribute remains un-present by the time flush occurs. To mark an object -as "modified" without referring to any attribute specifically, so that it -is considered within the flush process for the purpose of custom event handlers -such as :meth:`.SessionEvents.before_flush`, use the new -:func:`.attributes.flag_dirty` function:: - - from sqlalchemy.orm import attributes - - attributes.flag_dirty(a1) - -:ticket:`3753` - -.. _change_3796: - -"scope" keyword removed from scoped_session -------------------------------------------- - -A very old and undocumented keyword argument ``scope`` has been removed:: - - from sqlalchemy.orm import scoped_session - - Session = scoped_session(sessionmaker()) - - session = Session(scope=None) - -The purpose of this keyword was an attempt to allow for variable -"scopes", where ``None`` indicated "no scope" and would therefore return -a new :class:`.Session`. The keyword has never been documented and will -now raise ``TypeError`` if encountered. It is not anticipated that this -keyword is in use, however if users report issues related to this during -beta testing, it can be restored with a deprecation. - -:ticket:`3796` - -.. _change_3471: - -Refinements to post_update in conjunction with onupdate -------------------------------------------------------- - -A relationship that uses the :paramref:`_orm.relationship.post_update` feature -will now interact better with a column that has an :paramref:`_schema.Column.onupdate` -value set. If an object is inserted with an explicit value for the column, -it is re-stated during the UPDATE so that the "onupdate" rule does not -overwrite it:: - - class A(Base): - __tablename__ = "a" - id = Column(Integer, primary_key=True) - favorite_b_id = Column(ForeignKey("b.id", name="favorite_b_fk")) - bs = relationship("B", primaryjoin="A.id == B.a_id") - favorite_b = relationship( - "B", primaryjoin="A.favorite_b_id == B.id", post_update=True - ) - updated = Column(Integer, onupdate=my_onupdate_function) - - - class B(Base): - __tablename__ = "b" - id = Column(Integer, primary_key=True) - a_id = Column(ForeignKey("a.id", name="a_fk")) - - - a1 = A() - b1 = B() - - a1.bs.append(b1) - a1.favorite_b = b1 - a1.updated = 5 - s.add(a1) - s.flush() - -Above, the previous behavior would be that an UPDATE would emit after the -INSERT, thus triggering the "onupdate" and overwriting the value -"5". The SQL now looks like: - -.. sourcecode:: sql - - INSERT INTO a (favorite_b_id, updated) VALUES (?, ?) - (None, 5) - INSERT INTO b (a_id) VALUES (?) - (1,) - UPDATE a SET favorite_b_id=?, updated=? WHERE a.id = ? - (1, 5, 1) - -Additionally, if the value of "updated" is *not* set, then we correctly -get back the newly generated value on ``a1.updated``; previously, the logic -that refreshes or expires the attribute to allow the generated value -to be present would not fire off for a post-update. The -:meth:`.InstanceEvents.refresh_flush` event is also emitted when a refresh -within flush occurs in this case. - -:ticket:`3471` - -:ticket:`3472` - -.. _change_3496: - -post_update integrates with ORM versioning ------------------------------------------- - -The post_update feature, documented at :ref:`post_update`, involves that an -UPDATE statement is emitted in response to changes to a particular -relationship-bound foreign key, in addition to the INSERT/UPDATE/DELETE that -would normally be emitted for the target row. This UPDATE statement -now participates in the versioning feature, documented at -:ref:`mapper_version_counter`. - -Given a mapping:: - - class Node(Base): - __tablename__ = "node" - id = Column(Integer, primary_key=True) - version_id = Column(Integer, default=0) - parent_id = Column(ForeignKey("node.id")) - favorite_node_id = Column(ForeignKey("node.id")) - - nodes = relationship("Node", primaryjoin=remote(parent_id) == id) - favorite_node = relationship( - "Node", primaryjoin=favorite_node_id == remote(id), post_update=True - ) - - __mapper_args__ = {"version_id_col": version_id} - -An UPDATE of a node that associates another node as "favorite" will -now increment the version counter as well as match the current version:: - - node = Node() - session.add(node) - session.commit() # node is now version #1 - - node = session.query(Node).get(node.id) - node.favorite_node = Node() - session.commit() # node is now version #2 - -Note that this means an object that receives an UPDATE in response to -other attributes changing, and a second UPDATE due to a post_update -relationship change, will now receive -**two version counter updates for one flush**. However, if the object -is subject to an INSERT within the current flush, the version counter -**will not** be incremented an additional time, unless a server-side -versioning scheme is in place. - -The reason post_update emits an UPDATE even for an UPDATE is now discussed at -:ref:`faq_post_update_update`. - -.. seealso:: - - :ref:`post_update` - - :ref:`faq_post_update_update` - - -:ticket:`3496` - -Key Behavioral Changes - Core -============================= - -.. _change_4063: - -The typing behavior of custom operators has been made consistent ----------------------------------------------------------------- - -User defined operators can be made on the fly using the -:meth:`.Operators.op` function. Previously, the typing behavior of -an expression against such an operator was inconsistent and also not -controllable. - -Whereas in 1.1, an expression such as the following would produce -a result with no return type (assume ``-%>`` is some special operator -supported by the database):: - - >>> column("x", types.DateTime).op("-%>")(None).type - NullType() - -Other types would use the default behavior of using the left-hand type -as the return type:: - - >>> column("x", types.String(50)).op("-%>")(None).type - String(length=50) - -These behaviors were mostly by accident, so the behavior has been made -consistent with the second form, that is the default return type is the -same as the left-hand expression:: - - >>> column("x", types.DateTime).op("-%>")(None).type - DateTime() - -As most user-defined operators tend to be "comparison" operators, often -one of the many special operators defined by PostgreSQL, the -:paramref:`.Operators.op.is_comparison` flag has been repaired to follow -its documented behavior of allowing the return type to be :class:`.Boolean` -in all cases, including for :class:`_types.ARRAY` and :class:`_types.JSON`:: - - >>> column("x", types.String(50)).op("-%>", is_comparison=True)(None).type - Boolean() - >>> column("x", types.ARRAY(types.Integer)).op("-%>", is_comparison=True)(None).type - Boolean() - >>> column("x", types.JSON()).op("-%>", is_comparison=True)(None).type - Boolean() - -To assist with boolean comparison operators, a new shorthand method -:meth:`.Operators.bool_op` has been added. This method should be preferred -for on-the-fly boolean operators: - -.. sourcecode:: pycon+sql - - >>> print(column("x", types.Integer).bool_op("-%>")(5)) - {printsql}x -%> :x_1 - - -.. _change_3740: - -Percent signs in literal_column() now conditionally escaped ------------------------------------------------------------ - -The :obj:`_expression.literal_column` construct now escapes percent sign characters -conditionally, based on whether or not the DBAPI in use makes use of a -percent-sign-sensitive paramstyle or not (e.g. 'format' or 'pyformat'). - -Previously, it was not possible to produce a :obj:`_expression.literal_column` -construct that stated a single percent sign: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy import literal_column - >>> print(literal_column("some%symbol")) - {printsql}some%%symbol - -The percent sign is now unaffected for dialects that are not set to -use the 'format' or 'pyformat' paramstyles; dialects such most MySQL -dialects which do state one of these paramstyles will continue to escape -as is appropriate: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy import literal_column - >>> print(literal_column("some%symbol")) - {printsql}some%symbol{stop} - >>> from sqlalchemy.dialects import mysql - >>> print(literal_column("some%symbol").compile(dialect=mysql.dialect())) - {printsql}some%%symbol{stop} - -As part of this change, the doubling that has been present when using -operators like :meth:`.ColumnOperators.contains`, -:meth:`.ColumnOperators.startswith` and :meth:`.ColumnOperators.endswith` -is also refined to only occur when appropriate. - -:ticket:`3740` - - -.. _change_3785: - -The column-level COLLATE keyword now quotes the collation name --------------------------------------------------------------- - -A bug in the :func:`_expression.collate` and :meth:`.ColumnOperators.collate` -functions, used to supply ad-hoc column collations at the statement level, -is fixed, where a case sensitive name would not be quoted:: - - stmt = select([mytable.c.x, mytable.c.y]).order_by( - mytable.c.somecolumn.collate("fr_FR") - ) - -now renders: - -.. sourcecode:: sql - - SELECT mytable.x, mytable.y, - FROM mytable ORDER BY mytable.somecolumn COLLATE "fr_FR" - -Previously, the case sensitive name `"fr_FR"` would not be quoted. Currently, -manual quoting of the "fr_FR" name is **not** detected, so applications that -are manually quoting the identifier should be adjusted. Note that this change -does not impact the use of collations at the type level (e.g. specified -on the datatype like :class:`.String` at the table level), where quoting -is already applied. - -:ticket:`3785` - -Dialect Improvements and Changes - PostgreSQL -============================================= - -.. _change_4109: - -Support for Batch Mode / Fast Execution Helpers ------------------------------------------------- - -The psycopg2 ``cursor.executemany()`` method has been identified as performing -poorly, particularly with INSERT statements. To alleviate this, psycopg2 -has added `Fast Execution Helpers `_ -which rework statements into fewer server round trips by sending multiple -DML statements in batch. SQLAlchemy 1.2 now includes support for these -helpers to be used transparently whenever the :class:`_engine.Engine` makes use -of ``cursor.executemany()`` to invoke a statement against multiple parameter -sets. The feature is off by default and can be enabled using the -``use_batch_mode`` argument on :func:`_sa.create_engine`:: - - engine = create_engine( - "postgresql+psycopg2://scott:tiger@host/dbname", use_batch_mode=True - ) - -The feature is considered to be experimental for the moment but may become -on by default in a future release. - -.. seealso:: - - :ref:`psycopg2_batch_mode` - -:ticket:`4109` - -.. _change_3959: - -Support for fields specification in INTERVAL, including full reflection ------------------------------------------------------------------------ - -The "fields" specifier in PostgreSQL's INTERVAL datatype allows specification -of which fields of the interval to store, including such values as "YEAR", -"MONTH", "YEAR TO MONTH", etc. The :class:`_postgresql.INTERVAL` datatype -now allows these values to be specified:: - - from sqlalchemy.dialects.postgresql import INTERVAL - - Table("my_table", metadata, Column("some_interval", INTERVAL(fields="DAY TO SECOND"))) - -Additionally, all INTERVAL datatypes can now be reflected independently -of the "fields" specifier present; the "fields" parameter in the datatype -itself will also be present:: - - >>> inspect(engine).get_columns("my_table") - [{'comment': None, - 'name': u'some_interval', 'nullable': True, - 'default': None, 'autoincrement': False, - 'type': INTERVAL(fields=u'day to second')}] - -:ticket:`3959` - -Dialect Improvements and Changes - MySQL -======================================== - -.. _change_4009: - -Support for INSERT..ON DUPLICATE KEY UPDATE -------------------------------------------- - -The ``ON DUPLICATE KEY UPDATE`` clause of ``INSERT`` supported by MySQL -is now supported using a MySQL-specific version of the -:class:`_expression.Insert` object, via :func:`sqlalchemy.dialects.mysql.dml.insert`. -This :class:`_expression.Insert` subclass adds a new method -:meth:`~.mysql.dml.Insert.on_duplicate_key_update` that implements MySQL's syntax:: - - from sqlalchemy.dialects.mysql import insert - - insert_stmt = insert(my_table).values(id="some_id", data="some data to insert") - - on_conflict_stmt = insert_stmt.on_duplicate_key_update( - data=insert_stmt.inserted.data, status="U" - ) - - conn.execute(on_conflict_stmt) - -The above will render: - -.. sourcecode:: sql - - INSERT INTO my_table (id, data) - VALUES (:id, :data) - ON DUPLICATE KEY UPDATE data=VALUES(data), status=:status_1 - -.. seealso:: - - :ref:`mysql_insert_on_duplicate_key_update` - -:ticket:`4009` - - -Dialect Improvements and Changes - Oracle -========================================= - -.. _change_cxoracle_12: - -Major Refactor to cx_Oracle Dialect, Typing System --------------------------------------------------- - -With the introduction of the 6.x series of the cx_Oracle DBAPI, SQLAlchemy's -cx_Oracle dialect has been reworked and simplified to take advantage of recent -improvements in cx_Oracle as well as dropping support for patterns that were -more relevant before the 5.x series of cx_Oracle. - -* The minimum cx_Oracle version supported is now 5.1.3; 5.3 or the most recent - 6.x series are recommended. - -* The handling of datatypes has been refactored. The ``cursor.setinputsizes()`` - method is no longer used for any datatype except LOB types, per advice from - cx_Oracle's developers. As a result, the parameters ``auto_setinputsizes`` - and ``exclude_setinputsizes`` are deprecated and no longer have any effect. - -* The ``coerce_to_decimal`` flag, when set to False to indicate that coercion - of numeric types with precision and scale to ``Decimal`` should not occur, - only impacts untyped (e.g. plain string with no :class:`.TypeEngine` objects) - statements. A Core expression that includes a :class:`.Numeric` type or - subtype will now follow the decimal coercion rules of that type. - -* The "two phase" transaction support in the dialect, already dropped for the - 6.x series of cx_Oracle, has now been removed entirely as this feature has - never worked correctly and is unlikely to have been in production use. - As a result, the ``allow_twophase`` dialect flag is deprecated and also has no - effect. - -* Fixed a bug involving the column keys present with RETURNING. Given - a statement as follows:: - - result = conn.execute(table.insert().values(x=5).returning(table.c.a, table.c.b)) - - Previously, the keys in each row of the result would be ``ret_0`` and ``ret_1``, - which are identifiers internal to the cx_Oracle RETURNING implementation. - The keys will now be ``a`` and ``b`` as is expected for other dialects. - -* cx_Oracle's LOB datatype represents return values as a ``cx_Oracle.LOB`` - object, which is a cursor-associated proxy that returns the ultimate data - value via a ``.read()`` method. Historically, if more rows were read before - these LOB objects were consumed (specifically, more rows than the value of - cursor.arraysize which causes a new batch of rows to be read), these LOB - objects would raise the error "LOB variable no longer valid after subsequent - fetch". SQLAlchemy worked around this by both automatically calling - ``.read()`` upon these LOBs within its typing system, as well as using a - special ``BufferedColumnResultSet`` which would ensure this data was buffered - in case a call like ``cursor.fetchmany()`` or ``cursor.fetchall()`` were - used. - - The dialect now makes use of a cx_Oracle outputtypehandler to handle these - ``.read()`` calls, so that they are always called up front regardless of how - many rows are being fetched, so that this error can no longer occur. As a - result, the use of the ``BufferedColumnResultSet``, as well as some other - internals to the Core ``ResultSet`` that were specific to this use case, - have been removed. The type objects are also simplified as they no longer - need to process a binary column result. - - Additionally, cx_Oracle 6.x has removed the conditions under which this error - occurs in any case, so the error is no longer possible. The error - can occur on SQLAlchemy in the case that the seldom (if ever) used - ``auto_convert_lobs=False`` option is in use, in conjunction with the - previous 5.x series of cx_Oracle, and more rows are read before the LOB - objects can be consumed. Upgrading to cx_Oracle 6.x will resolve that issue. - -.. _change_4003: - -Oracle Unique, Check constraints now reflected ----------------------------------------------- - -UNIQUE and CHECK constraints now reflect via -:meth:`_reflection.Inspector.get_unique_constraints` and -:meth:`_reflection.Inspector.get_check_constraints`. A :class:`_schema.Table` object that's -reflected will now include :class:`.CheckConstraint` objects as well. -See the notes at :ref:`oracle_constraint_reflection` for information -on behavioral quirks here, including that most :class:`_schema.Table` objects -will still not include any :class:`.UniqueConstraint` objects as these -usually represent via :class:`.Index`. - -.. seealso:: - - :ref:`oracle_constraint_reflection` - - -:ticket:`4003` - -.. _change_3276: - -Oracle foreign key constraint names are now "name normalized" -------------------------------------------------------------- - -The names of foreign key constraints as delivered to a -:class:`_schema.ForeignKeyConstraint` object during table reflection as well as -within the :meth:`_reflection.Inspector.get_foreign_keys` method will now be -"name normalized", that is, expressed as lower case for a case insensitive -name, rather than the raw UPPERCASE format that Oracle uses:: - - >>> insp.get_indexes("addresses") - [{'unique': False, 'column_names': [u'user_id'], - 'name': u'address_idx', 'dialect_options': {}}] - - >>> insp.get_pk_constraint("addresses") - {'name': u'pk_cons', 'constrained_columns': [u'id']} - - >>> insp.get_foreign_keys("addresses") - [{'referred_table': u'users', 'referred_columns': [u'id'], - 'referred_schema': None, 'name': u'user_id_fk', - 'constrained_columns': [u'user_id']}] - -Previously, the foreign keys result would look like:: - - [ - { - "referred_table": "users", - "referred_columns": ["id"], - "referred_schema": None, - "name": "USER_ID_FK", - "constrained_columns": ["user_id"], - } - ] - -Where the above could create problems particularly with Alembic autogenerate. - -:ticket:`3276` - - -Dialect Improvements and Changes - SQL Server -============================================= - -.. _change_2626: - -SQL Server schema names with embedded dots supported ----------------------------------------------------- - -The SQL Server dialect has a behavior such that a schema name with a dot inside -of it is assumed to be a "database"."owner" identifier pair, which is -necessarily split up into these separate components during table and component -reflection operations, as well as when rendering quoting for the schema name so -that the two symbols are quoted separately. The schema argument can -now be passed using brackets to manually specify where this split -occurs, allowing database and/or owner names that themselves contain one -or more dots:: - - Table("some_table", metadata, Column("q", String(50)), schema="[MyDataBase.dbo]") - -The above table will consider the "owner" to be ``MyDataBase.dbo``, which -will also be quoted upon render, and the "database" as None. To individually -refer to database name and owner, use two pairs of brackets:: - - Table( - "some_table", - metadata, - Column("q", String(50)), - schema="[MyDataBase.SomeDB].[MyDB.owner]", - ) - -Additionally, the :class:`.quoted_name` construct is now honored when -passed to "schema" by the SQL Server dialect; the given symbol will -not be split on the dot if the quote flag is True and will be interpreted -as the "owner". - -.. seealso:: - - :ref:`multipart_schema_names` - -:ticket:`2626` - -AUTOCOMMIT isolation level support ----------------------------------- - -Both the PyODBC and pymssql dialects now support the "AUTOCOMMIT" isolation -level as set by :meth:`_engine.Connection.execution_options` which will establish -the correct flags on the DBAPI connection object. diff --git a/doc/build/changelog/migration_13.rst b/doc/build/changelog/migration_13.rst index f7ce653f476..7249b3c21dc 100644 --- a/doc/build/changelog/migration_13.rst +++ b/doc/build/changelog/migration_13.rst @@ -1228,7 +1228,7 @@ the ON clause of the SQL join is expressed in terms of a SQL function. Expanding IN feature now supports empty lists --------------------------------------------- -The "expanding IN" feature introduced in version 1.2 at :ref:`change_3953` now +The "expanding IN" feature introduced in version 1.2 at ref_change_3953 now supports empty lists passed to the :meth:`.ColumnOperators.in_` operator. The implementation for an empty list will produce an "empty set" expression that is specific to a target backend, such as "SELECT CAST(NULL AS INTEGER) WHERE 1!=1" for PostgreSQL, @@ -1352,7 +1352,7 @@ Coercion of string SQL fragments to text() fully removed --------------------------------------------------------- The warnings that were first added in version 1.0, described at -:ref:`migration_2992`, have now been converted into exceptions. Continued +ref_migration_2992, have now been converted into exceptions. Continued concerns have been raised regarding the automatic coercion of string fragments passed to methods like :meth:`_query.Query.filter` and :meth:`_expression.Select.order_by` being converted to :func:`_expression.text` constructs, even though this has emitted a warning. @@ -1554,7 +1554,7 @@ can now be explicitly ordered by passing a list of 2-tuples:: .. seealso:: - :ref:`mysql_insert_on_duplicate_key_update` + ref_mysql_insert_on_duplicate_key_update Dialect Improvements and Changes - SQLite ============================================= @@ -1611,7 +1611,7 @@ The above table would render in a CREATE TABLE statement as: .. seealso:: - :ref:`sqlite_on_conflict_ddl` + ref_sqlite_on_conflict_ddl :ticket:`4360` @@ -1708,7 +1708,7 @@ Pass it via :func:`_sa.create_engine`:: .. seealso:: - :ref:`mssql_pyodbc_fastexecutemany` + ref_mssql_pyodbc_fastexecutemany :ticket:`4158` @@ -1758,7 +1758,7 @@ primary key column:: .. seealso:: - :ref:`mssql_identity` + ref_mssql_identity :ticket:`4362` diff --git a/doc/build/changelog/migration_14.rst b/doc/build/changelog/migration_14.rst index ac6cb6a5354..4e94c291993 100644 --- a/doc/build/changelog/migration_14.rst +++ b/doc/build/changelog/migration_14.rst @@ -174,7 +174,7 @@ construction routines that must be built up each time an ORM query seeks to run and construct ORM objects from result sets. To introduce the general idea of the feature, given code from the -:ref:`examples_performance` suite as follows, which will invoke +ref_examples_performance suite as follows, which will invoke a very simple query "n" times, for a default value of n=10000. The query returns only a single row, as the overhead we are looking to decrease is that of **many small queries**. The optimization is not as significant @@ -287,7 +287,7 @@ In addition, The :func:`_declarative.instrument_declarative` function is deprecated, superseded by :meth:`_orm.registry.map_declaratively`. The :class:`_declarative.ConcreteBase`, :class:`_declarative.AbstractConcreteBase`, and :class:`_declarative.DeferredReflection` classes remain as extensions in the -:ref:`declarative_toplevel` package. +ref_declarative_toplevel package. Mapping styles have now been organized such that they all extend from the :class:`_orm.registry` object, and fall into these categories: @@ -299,10 +299,10 @@ the :class:`_orm.registry` object, and fall into these categories: * Using :meth:`_orm.registry.mapped` Declarative Decorator * Declarative Table * Imperative Table (Hybrid) - * :ref:`orm_declarative_dataclasses` + * ref_orm_declarative_dataclasses * :ref:`Imperative (a.k.a. "classical" mapping) ` * Using :meth:`_orm.registry.map_imperatively` - * :ref:`orm_imperative_dataclasses` + * ref_orm_imperative_dataclasses The existing classical mapping function ``sqlalchemy.orm.mapper()`` remains, however it is deprecated to call upon ``sqlalchemy.orm.mapper()`` directly; the @@ -351,9 +351,9 @@ attribute systems can now interoperate with Declarative mappings as well. .. seealso:: - :ref:`orm_declarative_dataclasses` + ref_orm_declarative_dataclasses - :ref:`orm_imperative_dataclasses` + ref_orm_imperative_dataclasses :ticket:`5027` @@ -373,7 +373,7 @@ usage as well as :class:`_orm.Session` for ORM use, using the the initial releases of SQLAlchemy 1.4. This is super new stuff that uses some previously unfamiliar programming techniques. -The initial database API supported is the :ref:`dialect-postgresql-asyncpg` +The initial database API supported is the ref_dialect-postgresql-asyncpg asyncio driver for PostgreSQL. The internal features of SQLAlchemy are fully integrated by making use of @@ -423,7 +423,7 @@ tradition, the new API provides a **strictly optional feature** such that applications that wish to make use of such ORM features can opt to organize database-related code into functions which can then be run within greenlets using the :meth:`_asyncio.AsyncSession.run_sync` -method. See the ``greenlet_orm.py`` example at :ref:`examples_asyncio` +method. See the ``greenlet_orm.py`` example at ref_examples_asyncio for a demonstration. Support for asynchronous cursors is also provided using new methods @@ -438,9 +438,9 @@ in traditional SQLAlchemy. .. seealso:: - :ref:`asyncio_toplevel` + ref_asyncio_toplevel - :ref:`examples_asyncio` + ref_examples_asyncio @@ -553,7 +553,7 @@ is established as the implementation. :meth:`_sql.ColumnOperators.regexp_replace` - :ref:`pysqlite_regexp` - SQLite implementation notes + ref_pysqlite_regexp - SQLite implementation notes :ticket:`1390` @@ -1026,7 +1026,7 @@ structural specification, lists are used for data specification**. All IN expressions render parameters for each value in the list on the fly (e.g. expanding parameters) ------------------------------------------------------------------------------------------------------ -The "expanding IN" feature, first introduced in :ref:`change_3953`, has matured +The "expanding IN" feature, first introduced in ref_change_3953, has matured enough such that it is clearly superior to the previous method of rendering IN expressions. As the approach was improved to handle empty lists of values, it is now the only means that Core / ORM will use to render lists of IN @@ -1043,7 +1043,7 @@ their parameters, nor could the parameter dictionary be fully used for statements that included IN expressions generally. In order to service the "baked query" feature described at -:ref:`baked_toplevel`, a cacheable version of IN was needed, which is what +ref_baked_toplevel, a cacheable version of IN was needed, which is what brought about the "expanding IN" feature. In contrast to the existing behavior whereby the parameter list is expanded at statement construction time into individual :class:`.BindParameter` objects, the feature instead uses a single @@ -1114,7 +1114,7 @@ introduced in version 1.3 and is described at :ref:`change_4271`. The :paramref:`_sa.create_engine.empty_in_strategy` parameter, introduced in version 1.2 as a means for migrating for how this case was treated for the previous IN system, is now deprecated and this flag no longer has an effect; as described -in :ref:`change_3907`, this flag allowed a dialect to switch between the +in ref_change_3907, this flag allowed a dialect to switch between the original system of comparing a column against itself, which turned out to be a huge performance issue, and a newer system of comparing "1 != 1" in order to produce a "false" expression. The 1.3 introduced behavior which @@ -1941,7 +1941,7 @@ rather than invoking the same statement repeatedly, as psycopg2 lacks the abilit to PREPARE the statement ahead of time as would normally be expected for this approach to be performant. -SQLAlchemy includes a :ref:`performance suite ` within +SQLAlchemy includes a ref_examples_performance within its examples, where we can compare the times generated for the "batch_inserts" runner against 1.3 and 1.4, revealing a 3x-5x speedup for most flavors of batch insert: @@ -1993,7 +1993,7 @@ The ultimate INSERT statement can be seen by enabling statement logging on the P The feature batches rows into groups of 1000 by default which can be affected using the ``executemany_values_page_size`` argument documented at -:ref:`psycopg2_executemany_mode`. +ref_psycopg2_executemany_mode. :ticket:`5263` @@ -2572,7 +2572,7 @@ in the value of the ``name`` attribute. Since this is a SQL NULL, the ORM would skip including these values within an INSERT so that SQL-level defaults take place, if any, else the value defaults to NULL on the database side. -In version 1.0 as part of :ref:`migration_3061`, this behavior was refined so +In version 1.0 as part of ref_migration_3061, this behavior was refined so that the ``None`` value was no longer populated into ``__dict__``, only returned. Besides removing the mutating side effect of a getter operation, this change also made it possible to set columns that did have server defaults @@ -2969,7 +2969,7 @@ It also occurs beyond the cache boundary so that the INSERT statement may be cached before the VALUES are rendered. A quick test of the ``execute_values()`` approach using the -``bulk_inserts.py`` script in the :ref:`examples_performance` example +``bulk_inserts.py`` script in the ref_examples_performance example suite reveals an approximate **fivefold performance increase**: .. sourcecode:: text @@ -2983,7 +2983,7 @@ suite reveals an approximate **fivefold performance increase**: test_core_insert : A single Core INSERT construct inserting mappings in bulk. (100000 iterations); total time 0.944007 sec Support for the "batch" extension was added in version 1.2 in -:ref:`change_4109`, and enhanced to include support for the ``execute_values`` +ref_change_4109, and enhanced to include support for the ``execute_values`` extension in 1.3 in :ticket:`4623`. In 1.4 the ``execute_values`` extension is now being turned on by default for INSERT statements; the "batch" extension for UPDATE and DELETE remains off by default. @@ -3035,7 +3035,7 @@ with the following changes: .. seealso:: - :ref:`psycopg2_executemany_mode` + ref_psycopg2_executemany_mode :ticket:`5401` @@ -3051,7 +3051,7 @@ versions prior to 3.7.16, released in 2013. It is not expected that any modern Python versions rely upon this limitation. The behavior was first introduced in 0.9 and was part of the larger change of -allowing for right nested joins as described at :ref:`feature_joins_09`. +allowing for right nested joins as described at ref_feature_joins_09. However the SQLite workaround produced many regressions in the 2013-2014 period due to its complexity. In 2016, the dialect was modified so that the join rewriting logic would only occur for SQLite versions prior to 3.7.16 after @@ -3114,7 +3114,7 @@ was not created. .. seealso:: - :ref:`defaults_sequences` + ref_defaults_sequences :ticket:`4976` @@ -3139,7 +3139,7 @@ parameters should be used; see the MSSQL dialect documentation linked below. .. seealso:: - :ref:`mssql_identity` + ref_mssql_identity :ticket:`4235` diff --git a/doc/build/changelog/migration_20.rst b/doc/build/changelog/migration_20.rst index 1fe47b7ac17..522fb87b1c3 100644 --- a/doc/build/changelog/migration_20.rst +++ b/doc/build/changelog/migration_20.rst @@ -458,7 +458,7 @@ of the :class:`_orm.Mapped` generic container. Annotations which don't use :class:`_orm.Mapped` which link to constructs such as :func:`_orm.relationship` will raise errors in Python, as they suggest mis-configurations. -SQLAlchemy applications that use the :ref:`Mypy plugin ` with +SQLAlchemy applications that use the ref_mypy_toplevel with explicit annotations that don't use :class:`_orm.Mapped` in their annotations are subject to these errors, as would occur in the example below:: @@ -1298,8 +1298,8 @@ has always allowed this style using so-called remove the base class requirement, a first class :ref:`decorator ` form has been added. -As yet another separate but related enhancement, support for :ref:`Python -dataclasses ` is added as well to both +As yet another separate but related enhancement, support for Python +dataclasses is added as well to both declarative decorator and classical mapping forms. .. seealso:: @@ -1524,7 +1524,7 @@ following the table, and may include additional notes not summarized here. ) # or - + session.scalar( select(func.count(User.id)) ) @@ -2163,7 +2163,7 @@ and should be preferred. **Synopsis** The ``lazy="dynamic"`` relationship loader strategy, discussed at -:ref:`dynamic_relationship`, makes use of the :class:`_query.Query` object +ref_dynamic_relationship, makes use of the :class:`_query.Query` object which is legacy in 2.0. The "dynamic" relationship is not directly compatible with asyncio without workarounds, and additionally it does not fulfill its original purpose of preventing iteration of large collections as it has several @@ -2249,7 +2249,7 @@ new feature is as :ref:`change_7123`. :ref:`change_7123` - :ref:`write_only_relationship` + ref_write_only_relationship .. _migration_20_session_autocommit: diff --git a/doc/build/changelog/unreleased_11/README.txt b/doc/build/changelog/unreleased_11/README.txt deleted file mode 100644 index 068264c720b..00000000000 --- a/doc/build/changelog/unreleased_11/README.txt +++ /dev/null @@ -1,8 +0,0 @@ -See doc/build/changelog/README.txt for -information on the doc build system. - - -DO NOT REMOVE THIS FILE!!!! THIS DIRECTORY MUST BE PRESENT -FOR DOCS TO BUILD. - - diff --git a/doc/build/changelog/unreleased_12/README.txt b/doc/build/changelog/unreleased_12/README.txt deleted file mode 100644 index 068264c720b..00000000000 --- a/doc/build/changelog/unreleased_12/README.txt +++ /dev/null @@ -1,8 +0,0 @@ -See doc/build/changelog/README.txt for -information on the doc build system. - - -DO NOT REMOVE THIS FILE!!!! THIS DIRECTORY MUST BE PRESENT -FOR DOCS TO BUILD. - - diff --git a/doc/build/changelog/unreleased_13/6135.rst b/doc/build/changelog/unreleased_13/6135.rst deleted file mode 100644 index 942b04edf96..00000000000 --- a/doc/build/changelog/unreleased_13/6135.rst +++ /dev/null @@ -1,10 +0,0 @@ -.. change:: - :tags: schema, bug - :tickets: 6135 - :versions: 1.4.6 - - The :class:`_schema.Table` object now raises an informative error message if - it is instantiated without passing at least the :paramref:`_schema.Table.name` - and :paramref:`_schema.Table.metadata` arguments positionally. Previously, if - these were passed as keyword arguments, the object would silently fail to - initialize correctly. diff --git a/doc/build/changelog/unreleased_13/6182.rst b/doc/build/changelog/unreleased_13/6182.rst deleted file mode 100644 index 3228b4f2b41..00000000000 --- a/doc/build/changelog/unreleased_13/6182.rst +++ /dev/null @@ -1,13 +0,0 @@ -.. change:: - :tags: bug, postgresql, regression - :tickets: 6182 - :versions: 1.4.5 - - Fixed regression caused by :ticket:`6023` where the PostgreSQL cast - operator applied to elements within an :class:`_types.ARRAY` when using - psycopg2 would fail to use the correct type in the case that the datatype - were also embedded within an instance of the :class:`_types.Variant` - adapter. - - Additionally, repairs support for the correct CREATE TYPE to be emitted - when using a ``Variant(ARRAY(some_schema_type))``. diff --git a/doc/build/changelog/unreleased_13/6392.rst b/doc/build/changelog/unreleased_13/6392.rst deleted file mode 100644 index e7cda565a5e..00000000000 --- a/doc/build/changelog/unreleased_13/6392.rst +++ /dev/null @@ -1,9 +0,0 @@ -.. change:: - :tags: bug, orm - :tickets: 6392 - :versions: 1.4.12 - - Fixed issue in :meth:`_orm.Session.bulk_save_objects` when used with persistent - objects which would fail to track the primary key of mappings where the - column name of the primary key were different than the attribute name. - diff --git a/doc/build/changelog/unreleased_13/6589.rst b/doc/build/changelog/unreleased_13/6589.rst deleted file mode 100644 index b6f5fc60635..00000000000 --- a/doc/build/changelog/unreleased_13/6589.rst +++ /dev/null @@ -1,7 +0,0 @@ -.. change:: - :tags: bug, sqlite - :tickets: 6589 - :versions: 1.4.18 - - Add note regarding encryption-related pragmas for pysqlcipher passed in the - url. diff --git a/doc/build/changelog/unreleased_13/7115.rst b/doc/build/changelog/unreleased_13/7115.rst deleted file mode 100644 index 1f2c7fcf862..00000000000 --- a/doc/build/changelog/unreleased_13/7115.rst +++ /dev/null @@ -1,16 +0,0 @@ -.. change:: - :tags: bug, mysql, mariadb - :tickets: 7115, 7136 - :versions: 1.4.26 - - Fixes to accommodate for the MariaDB 10.6 series, including backwards - incompatible changes in both the mariadb-connector Python driver (supported - on SQLAlchemy 1.4 only) as well as the native 10.6 client libraries that - are used automatically by the mysqlclient DBAPI (applies to both 1.3 and - 1.4). The "utf8mb3" encoding symbol is now reported by these client - libraries when the encoding is stated as "utf8", leading to lookup and - encoding errors within the MySQL dialect that does not expect this symbol. - Updates to both the MySQL base library to accommodate for this utf8mb3 - symbol being reported as well as to the test suite. Thanks to Georg Richter - for support. - diff --git a/doc/build/changelog/unreleased_13/README.txt b/doc/build/changelog/unreleased_13/README.txt deleted file mode 100644 index 068264c720b..00000000000 --- a/doc/build/changelog/unreleased_13/README.txt +++ /dev/null @@ -1,8 +0,0 @@ -See doc/build/changelog/README.txt for -information on the doc build system. - - -DO NOT REMOVE THIS FILE!!!! THIS DIRECTORY MUST BE PRESENT -FOR DOCS TO BUILD. - - diff --git a/doc/build/changelog/whatsnew_20.rst b/doc/build/changelog/whatsnew_20.rst index 366c9c0ffb8..1a4519f5256 100644 --- a/doc/build/changelog/whatsnew_20.rst +++ b/doc/build/changelog/whatsnew_20.rst @@ -371,7 +371,7 @@ ORM Declarative Models ~~~~~~~~~~~~~~~~~~~~~~ SQLAlchemy 1.4 introduced the first SQLAlchemy-native ORM typing support -using a combination of sqlalchemy2-stubs_ and the :ref:`Mypy Plugin `. +using a combination of sqlalchemy2-stubs_ and the ref_mypy_toplevel. In SQLAlchemy 2.0, the Mypy plugin **remains available, and has been updated to work with SQLAlchemy 2.0's typing system**. However, it should now be considered **deprecated**, as applications now have a straightforward path to adopting the @@ -731,7 +731,7 @@ and :class:`_engine.Row` objects:: Using Legacy Mypy-Typed Models ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -SQLAlchemy applications that use the :ref:`Mypy plugin ` with +SQLAlchemy applications that use the ref_mypy_toplevel with explicit annotations that don't use :class:`_orm.Mapped` in their annotations are subject to errors under the new system, as such annotations are flagged as errors when using constructs such as :func:`_orm.relationship`. @@ -848,7 +848,7 @@ positional arguments as configured:: .. seealso:: - :ref:`orm_declarative_native_dataclasses` + ref_orm_declarative_native_dataclasses .. _change_6047: @@ -910,7 +910,7 @@ and are still improved by the "insertmanyvalues" approach. Benchmarks ~~~~~~~~~~ -SQLAlchemy includes a :ref:`Performance Suite ` within +SQLAlchemy includes a ref_examples_performance within the ``examples/`` directory, where we can make use of the ``bulk_insert`` suite to benchmark INSERTs of many rows using both Core and ORM in different ways. @@ -929,7 +929,7 @@ Operations that are improved by this feature include: which improves upon the experimental version of this feature first introduced in SQLAlchemy 1.4. * the :class:`_orm.Session` "bulk" operations described at - :ref:`bulk_operations`, which are superseded by the above mentioned + bulk_operations, which are superseded by the above mentioned ORM Bulk Insert feature. To get a sense of the scale of the operation, below are performance @@ -1271,11 +1271,11 @@ using :term:`2.0 style` to load the desired objects in an explicit way:: The :class:`_orm.WriteOnlyCollection` also integrates with the new :ref:`ORM bulk dml ` features, including support for bulk INSERT and UPDATE/DELETE with WHERE criteria, all including RETURNING support as -well. See the complete documentation at :ref:`write_only_relationship`. +well. See the complete documentation at ref_write_only_relationship. .. seealso:: - :ref:`write_only_relationship` + ref_write_only_relationship New pep-484 / type annotated mapping support for Dynamic Relationships ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -1323,7 +1323,7 @@ as ``AccountTransaction``. .. seealso:: - :ref:`dynamic_relationship` + ref_dynamic_relationship :ticket:`7123` @@ -1392,7 +1392,7 @@ Major Architectural, Performance and API Enhancements for Database Reflection ----------------------------------------------------------------------------- The internal system by which :class:`.Table` objects and their components are -:ref:`reflected ` has been completely rearchitected to +ref_metadata_reflection has been completely rearchitected to allow high performance bulk reflection of thousands of tables at once for participating dialects. Currently, the **PostgreSQL** and **Oracle** dialects participate in the new architecture, where the PostgreSQL dialect can now @@ -1504,7 +1504,7 @@ automatically. .. seealso:: - :ref:`postgresql_psycopg` + ref_postgresql_psycopg .. _ticket_8054: @@ -1517,7 +1517,7 @@ DBAPI, which is the renamed, new major release of the popular cx_Oracle driver. .. seealso:: - :ref:`oracledb` + ref_oracledb .. _ticket_7631: @@ -2195,7 +2195,7 @@ customizable via the :paramref:`_sa.create_engine.poolclass` parameter. .. seealso:: - :ref:`pysqlite_threading_pooling` + ref_pysqlite_threading_pooling :ticket:`7490` @@ -2286,12 +2286,12 @@ All PostgreSQL search functions and operators are available through use of :data:`.func` to generate PostgreSQL-specific functions and :meth:`.Operators.bool_op` (a boolean-typed version of :meth:`.Operators.op`) to generate arbitrary operators, in the same manner as they are available -in previous versions. See the examples at :ref:`postgresql_match`. +in previous versions. See the examples at ref_postgresql_match. Existing SQLAlchemy projects that make use of PG-specific directives within :meth:`.Operators.match` should make use of ``func.to_tsquery()`` directly. To render SQL in exactly the same form as would be present -in 1.4, see the version note at :ref:`postgresql_simple_match`. +in 1.4, see the version note at ref_postgresql_simple_match. diff --git a/doc/build/conf.py b/doc/build/conf.py index 8b1ec5efa17..4485a34acf7 100644 --- a/doc/build/conf.py +++ b/doc/build/conf.py @@ -35,12 +35,10 @@ extensions = [ "sphinx.ext.autodoc", - "zzzeeksphinx", "changelog", "sphinx_paramlinks", "sphinx_copybutton", ] -needs_extensions = {"zzzeeksphinx": "1.2.1"} # Add any paths that contain templates here, relative to this directory. # not sure why abspath() is needed here, some users @@ -293,7 +291,7 @@ # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. -html_theme = "zzzeeksphinx" +html_theme = "nature" # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the @@ -342,7 +340,7 @@ # Additional templates that should be rendered to pages, maps page names to # template names. -html_additional_pages = {"notfound": "notfound.html"} +# html_additional_pages = {"notfound": "notfound.html"} # If false, no module index is generated. html_domain_indices = False diff --git a/doc/build/core/api_basics.rst b/doc/build/core/api_basics.rst index a2b7060d3aa..640b4fd4461 100644 --- a/doc/build/core/api_basics.rst +++ b/doc/build/core/api_basics.rst @@ -6,6 +6,5 @@ Core API Basics :maxdepth: 2 event - inspection exceptions internals diff --git a/doc/build/core/connections.rst b/doc/build/core/connections.rst index 488c2dfad50..b47a0a9493d 100644 --- a/doc/build/core/connections.rst +++ b/doc/build/core/connections.rst @@ -411,15 +411,15 @@ reverted when a connection is returned to the connection pool. .. seealso:: - :ref:`SQLite Transaction Isolation ` + ref_sqlite_isolation_level - :ref:`PostgreSQL Transaction Isolation ` + ref_postgresql_isolation_level - :ref:`MySQL Transaction Isolation ` + ref_mysql_isolation_level - :ref:`SQL Server Transaction Isolation ` + ref_mssql_isolation_level - :ref:`Oracle Transaction Isolation ` + ref_oracle_isolation_level :ref:`session_transaction_isolation` - for the ORM @@ -852,8 +852,8 @@ as the schema name is passed to these methods explicitly. values into account for individual objects. To use a single :class:`_orm.Session` with multiple ``schema_translate_map`` - configurations, the :ref:`horizontal_sharding_toplevel` extension may - be used. See the example at :ref:`examples_sharding`. + configurations, the ref_horizontal_sharding_toplevel extension may + be used. See the example at ref_examples_sharding. .. _sql_caching: @@ -1351,12 +1351,6 @@ SELECTs with LIMIT/OFFSET are correctly rendered and cached. Using Lambdas to add significant speed gains to statement production ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -.. deepalchemy:: This technique is generally non-essential except in very performance - intensive scenarios, and intended for experienced Python programmers. - While fairly straightforward, it involves metaprogramming concepts that are - not appropriate for novice Python developers. The lambda approach can be - applied to at a later time to existing code with a minimal amount of effort. - Python functions, typically expressed as lambdas, may be used to generate SQL expressions which are cacheable based on the Python code location of the lambda function itself as well as the closure variables within the @@ -1765,7 +1759,7 @@ be used when the statement is invoked:: For a series of examples of "lambda" caching with performance comparisons, -see the "short_selects" test suite within the :ref:`examples_performance` +see the "short_selects" test suite within the ref_examples_performance performance example. .. _engine_insertmanyvalues: @@ -1820,7 +1814,7 @@ place when making use of the :meth:`_dml.Insert.returning` method of an execution, which occurs when passing a list of dictionaries to the :paramref:`_engine.Connection.execute.parameters` parameter of the :meth:`_engine.Connection.execute` or :meth:`_orm.Session.execute` methods (as -well as equivalent methods under :ref:`asyncio ` and +well as equivalent methods under ref_asyncio_toplevel and shorthand methods like :meth:`_orm.Session.scalars`). It also takes place within the ORM :term:`unit of work` process when using methods such as :meth:`_orm.Session.add` and :meth:`_orm.Session.add_all` to add rows. diff --git a/doc/build/core/constraints.rst b/doc/build/core/constraints.rst index 1efccdce5a4..de94e574bbe 100644 --- a/doc/build/core/constraints.rst +++ b/doc/build/core/constraints.rst @@ -412,13 +412,13 @@ Setting up Constraints when using the Declarative ORM Extension The :class:`_schema.Table` is the SQLAlchemy Core construct that allows one to define table metadata, which among other things can be used by the SQLAlchemy ORM -as a target to map a class. The :ref:`Declarative ` +as a target to map a class. The ref_declarative_toplevel extension allows the :class:`_schema.Table` object to be created automatically, given the contents of the table primarily as a mapping of :class:`_schema.Column` objects. To apply table-level constraint objects such as :class:`_schema.ForeignKeyConstraint` to a table defined using Declarative, use the ``__table_args__`` attribute, -described at :ref:`declarative_table_args`. +described at ref_declarative_table_args. .. _constraint_naming_conventions: diff --git a/doc/build/core/custom_types.rst b/doc/build/core/custom_types.rst index 44187ee4c15..da9178f865e 100644 --- a/doc/build/core/custom_types.rst +++ b/doc/build/core/custom_types.rst @@ -311,7 +311,7 @@ dictionary-oriented JSON structure, we can apply this as:: .. seealso:: - :ref:`mutable_toplevel` + ref_mutable_toplevel Dealing with Comparison Operations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -695,7 +695,7 @@ It is important to note that database types which are modified to have additional in-Python behaviors, including types based on :class:`.TypeDecorator` as well as other user-defined subclasses of datatypes, do not have any representation within a database schema. When using database -the introspection features described at :ref:`metadata_reflection`, SQLAlchemy +the introspection features described at ref_metadata_reflection, SQLAlchemy makes use of a fixed mapping which links the datatype information reported by a database server to a SQLAlchemy datatype object. For example, if we look inside of a PostgreSQL schema at the definition for a particular database @@ -774,7 +774,7 @@ additional Python behaviors of the custom datatypes, additional steps must be taken to allow this. The most straightforward is to override specific columns as described at -:ref:`reflection_overriding_columns`. In this technique, we simply +ref_reflection_overriding_columns. In this technique, we simply use reflection in combination with explicit :class:`_schema.Column` objects for those columns for which we want to use a custom or decorated datatype:: diff --git a/doc/build/core/defaults.rst b/doc/build/core/defaults.rst deleted file mode 100644 index ef5ad208159..00000000000 --- a/doc/build/core/defaults.rst +++ /dev/null @@ -1,838 +0,0 @@ -.. currentmodule:: sqlalchemy.schema - -.. _metadata_defaults_toplevel: - -.. _metadata_defaults: - -Column INSERT/UPDATE Defaults -============================= - -Column INSERT and UPDATE defaults refer to functions that create a **default -value** for a particular column in a row as an INSERT or UPDATE statement is -proceeding against that row, in the case where **no value was provided to the -INSERT or UPDATE statement for that column**. That is, if a table has a column -called "timestamp", and an INSERT statement proceeds which does not include a -value for this column, an INSERT default would create a new value, such as -the current time, that is used as the value to be INSERTed into the "timestamp" -column. If the statement *does* include a value for this column, then the -default does *not* take place. - -Column defaults can be server-side functions or constant values which are -defined in the database along with the schema in :term:`DDL`, or as SQL -expressions which are rendered directly within an INSERT or UPDATE statement -emitted by SQLAlchemy; they may also be client-side Python functions or -constant values which are invoked by SQLAlchemy before data is passed to the -database. - -.. note:: - - A column default handler should not be confused with a construct that - intercepts and modifies incoming values for INSERT and UPDATE statements - which *are* provided to the statement as it is invoked. This is known - as :term:`data marshalling`, where a column value is modified in some way - by the application before being sent to the database. SQLAlchemy provides - a few means of achieving this which include using :ref:`custom datatypes - `, :ref:`SQL execution events ` and - in the ORM :ref:`custom validators ` as well as - :ref:`attribute events `. Column defaults are only - invoked when there is **no value present** for a column in a SQL - :term:`DML` statement. - - -SQLAlchemy provides an array of features regarding default generation -functions which take place for non-present values during INSERT and UPDATE -statements. Options include: - -* Scalar values used as defaults during INSERT and UPDATE operations -* Python functions which execute upon INSERT and UPDATE operations -* SQL expressions which are embedded in INSERT statements (or in some cases execute beforehand) -* SQL expressions which are embedded in UPDATE statements -* Server side default values used during INSERT -* Markers for server-side triggers used during UPDATE - -The general rule for all insert/update defaults is that they only take effect -if no value for a particular column is passed as an ``execute()`` parameter; -otherwise, the given value is used. - -Scalar Defaults ---------------- - -The simplest kind of default is a scalar value used as the default value of a column:: - - Table("mytable", metadata_obj, Column("somecolumn", Integer, default=12)) - -Above, the value "12" will be bound as the column value during an INSERT if no -other value is supplied. - -A scalar value may also be associated with an UPDATE statement, though this is -not very common (as UPDATE statements are usually looking for dynamic -defaults):: - - Table("mytable", metadata_obj, Column("somecolumn", Integer, onupdate=25)) - -Python-Executed Functions -------------------------- - -The :paramref:`_schema.Column.default` and :paramref:`_schema.Column.onupdate` keyword arguments also accept Python -functions. These functions are invoked at the time of insert or update if no -other value for that column is supplied, and the value returned is used for -the column's value. Below illustrates a crude "sequence" that assigns an -incrementing counter to a primary key column:: - - # a function which counts upwards - i = 0 - - - def mydefault(): - global i - i += 1 - return i - - - t = Table( - "mytable", - metadata_obj, - Column("id", Integer, primary_key=True, default=mydefault), - ) - -It should be noted that for real "incrementing sequence" behavior, the -built-in capabilities of the database should normally be used, which may -include sequence objects or other autoincrementing capabilities. For primary -key columns, SQLAlchemy will in most cases use these capabilities -automatically. See the API documentation for -:class:`~sqlalchemy.schema.Column` including the :paramref:`_schema.Column.autoincrement` flag, as -well as the section on :class:`~sqlalchemy.schema.Sequence` later in this -chapter for background on standard primary key generation techniques. - -To illustrate onupdate, we assign the Python ``datetime`` function ``now`` to -the :paramref:`_schema.Column.onupdate` attribute:: - - import datetime - - t = Table( - "mytable", - metadata_obj, - Column("id", Integer, primary_key=True), - # define 'last_updated' to be populated with datetime.now() - Column("last_updated", DateTime, onupdate=datetime.datetime.now), - ) - -When an update statement executes and no value is passed for ``last_updated``, -the ``datetime.datetime.now()`` Python function is executed and its return -value used as the value for ``last_updated``. Notice that we provide ``now`` -as the function itself without calling it (i.e. there are no parenthesis -following) - SQLAlchemy will execute the function at the time the statement -executes. - -.. _context_default_functions: - -Context-Sensitive Default Functions -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The Python functions used by :paramref:`_schema.Column.default` and -:paramref:`_schema.Column.onupdate` may also make use of the current statement's -context in order to determine a value. The `context` of a statement is an -internal SQLAlchemy object which contains all information about the statement -being executed, including its source expression, the parameters associated with -it and the cursor. The typical use case for this context with regards to -default generation is to have access to the other values being inserted or -updated on the row. To access the context, provide a function that accepts a -single ``context`` argument:: - - def mydefault(context): - return context.get_current_parameters()["counter"] + 12 - - - t = Table( - "mytable", - metadata_obj, - Column("counter", Integer), - Column("counter_plus_twelve", Integer, default=mydefault, onupdate=mydefault), - ) - -The above default generation function is applied so that it will execute for -all INSERT and UPDATE statements where a value for ``counter_plus_twelve`` was -otherwise not provided, and the value will be that of whatever value is present -in the execution for the ``counter`` column, plus the number 12. - -For a single statement that is being executed using "executemany" style, e.g. -with multiple parameter sets passed to :meth:`_engine.Connection.execute`, the -user-defined function is called once for each set of parameters. For the use case of -a multi-valued :class:`_expression.Insert` construct (e.g. with more than one VALUES -clause set up via the :meth:`_expression.Insert.values` method), the user-defined function -is also called once for each set of parameters. - -When the function is invoked, the special method -:meth:`.DefaultExecutionContext.get_current_parameters` is available from -the context object (an subclass of :class:`.DefaultExecutionContext`). This -method returns a dictionary of column-key to values that represents the -full set of values for the INSERT or UPDATE statement. In the case of a -multi-valued INSERT construct, the subset of parameters that corresponds to -the individual VALUES clause is isolated from the full parameter dictionary -and returned alone. - -.. versionadded:: 1.2 - - Added :meth:`.DefaultExecutionContext.get_current_parameters` method, - which improves upon the still-present - :attr:`.DefaultExecutionContext.current_parameters` attribute - by offering the service of organizing multiple VALUES clauses - into individual parameter dictionaries. - -.. _defaults_client_invoked_sql: - -Client-Invoked SQL Expressions ------------------------------- - -The :paramref:`_schema.Column.default` and :paramref:`_schema.Column.onupdate` keywords may -also be passed SQL expressions, which are in most cases rendered inline within the -INSERT or UPDATE statement:: - - t = Table( - "mytable", - metadata_obj, - Column("id", Integer, primary_key=True), - # define 'create_date' to default to now() - Column("create_date", DateTime, default=func.now()), - # define 'key' to pull its default from the 'keyvalues' table - Column( - "key", - String(20), - default=select(keyvalues.c.key).where(keyvalues.c.type="type1"), - ), - # define 'last_modified' to use the current_timestamp SQL function on update - Column("last_modified", DateTime, onupdate=func.utc_timestamp()), - ) - -Above, the ``create_date`` column will be populated with the result of the -``now()`` SQL function (which, depending on backend, compiles into ``NOW()`` -or ``CURRENT_TIMESTAMP`` in most cases) during an INSERT statement, and the -``key`` column with the result of a SELECT subquery from another table. The -``last_modified`` column will be populated with the value of -the SQL ``UTC_TIMESTAMP()`` MySQL function when an UPDATE statement is -emitted for this table. - -.. note:: - - When using SQL functions with the :attr:`.func` construct, we "call" the - named function, e.g. with parenthesis as in ``func.now()``. This differs - from when we specify a Python callable as a default such as - ``datetime.datetime``, where we pass the function itself, but we don't - invoke it ourselves. In the case of a SQL function, invoking - ``func.now()`` returns the SQL expression object that will render the - "NOW" function into the SQL being emitted. - -Default and update SQL expressions specified by :paramref:`_schema.Column.default` and -:paramref:`_schema.Column.onupdate` are invoked explicitly by SQLAlchemy when an -INSERT or UPDATE statement occurs, typically rendered inline within the DML -statement except in certain cases listed below. This is different than a -"server side" default, which is part of the table's DDL definition, e.g. as -part of the "CREATE TABLE" statement, which are likely more common. For -server side defaults, see the next section :ref:`server_defaults`. - -When a SQL expression indicated by :paramref:`_schema.Column.default` is used with -primary key columns, there are some cases where SQLAlchemy must "pre-execute" -the default generation SQL function, meaning it is invoked in a separate SELECT -statement, and the resulting value is passed as a parameter to the INSERT. -This only occurs for primary key columns for an INSERT statement that is being -asked to return this primary key value, where RETURNING or ``cursor.lastrowid`` -may not be used. An :class:`_expression.Insert` construct that specifies the -:paramref:`~.expression.insert.inline` flag will always render default expressions -inline. - -When the statement is executed with a single set of parameters (that is, it is -not an "executemany" style execution), the returned -:class:`~sqlalchemy.engine.CursorResult` will contain a collection accessible -via :meth:`_engine.CursorResult.postfetch_cols` which contains a list of all -:class:`~sqlalchemy.schema.Column` objects which had an inline-executed -default. Similarly, all parameters which were bound to the statement, including -all Python and SQL expressions which were pre-executed, are present in the -:meth:`_engine.CursorResult.last_inserted_params` or -:meth:`_engine.CursorResult.last_updated_params` collections on -:class:`~sqlalchemy.engine.CursorResult`. The -:attr:`_engine.CursorResult.inserted_primary_key` collection contains a list of primary -key values for the row inserted (a list so that single-column and -composite-column primary keys are represented in the same format). - -.. _server_defaults: - -Server-invoked DDL-Explicit Default Expressions ------------------------------------------------ - -A variant on the SQL expression default is the :paramref:`_schema.Column.server_default`, which gets -placed in the CREATE TABLE statement during a :meth:`_schema.Table.create` operation: - -.. sourcecode:: python+sql - - t = Table( - "test", - metadata_obj, - Column("abc", String(20), server_default="abc"), - Column("created_at", DateTime, server_default=func.sysdate()), - Column("index_value", Integer, server_default=text("0")), - ) - -A create call for the above table will produce: - -.. sourcecode:: sql - - CREATE TABLE test ( - abc varchar(20) default 'abc', - created_at datetime default sysdate, - index_value integer default 0 - ) - -The above example illustrates the two typical use cases for :paramref:`_schema.Column.server_default`, -that of the SQL function (SYSDATE in the above example) as well as a server-side constant -value (the integer "0" in the above example). It is advisable to use the -:func:`_expression.text` construct for any literal SQL values as opposed to passing the -raw value, as SQLAlchemy does not typically perform any quoting or escaping on -these values. - -Like client-generated expressions, :paramref:`_schema.Column.server_default` can accommodate -SQL expressions in general, however it is expected that these will usually be simple -functions and expressions, and not the more complex cases like an embedded SELECT. - - -.. _triggered_columns: - -Marking Implicitly Generated Values, timestamps, and Triggered Columns ----------------------------------------------------------------------- - -Columns which generate a new value on INSERT or UPDATE based on other -server-side database mechanisms, such as database-specific auto-generating -behaviors such as seen with TIMESTAMP columns on some platforms, as well as -custom triggers that invoke upon INSERT or UPDATE to generate a new value, -may be called out using :class:`.FetchedValue` as a marker:: - - from sqlalchemy.schema import FetchedValue - - t = Table( - "test", - metadata_obj, - Column("id", Integer, primary_key=True), - Column("abc", TIMESTAMP, server_default=FetchedValue()), - Column("def", String(20), server_onupdate=FetchedValue()), - ) - -The :class:`.FetchedValue` indicator does not affect the rendered DDL for the -CREATE TABLE. Instead, it marks the column as one that will have a new value -populated by the database during the process of an INSERT or UPDATE statement, -and for supporting databases may be used to indicate that the column should be -part of a RETURNING or OUTPUT clause for the statement. Tools such as the -SQLAlchemy ORM then make use of this marker in order to know how to get at the -value of the column after such an operation. In particular, the -:meth:`.ValuesBase.return_defaults` method can be used with an :class:`_expression.Insert` -or :class:`_expression.Update` construct to indicate that these values should be -returned. - -For details on using :class:`.FetchedValue` with the ORM, see -:ref:`orm_server_defaults`. - -.. warning:: The :paramref:`_schema.Column.server_onupdate` directive - **does not** currently produce MySQL's - "ON UPDATE CURRENT_TIMESTAMP()" clause. See - :ref:`mysql_timestamp_onupdate` for background on how to produce - this clause. - - -.. seealso:: - - :ref:`orm_server_defaults` - -.. _defaults_sequences: - -Defining Sequences ------------------- - -SQLAlchemy represents database sequences using the -:class:`~sqlalchemy.schema.Sequence` object, which is considered to be a -special case of "column default". It only has an effect on databases which have -explicit support for sequences, which among SQLAlchemy's included dialects -includes PostgreSQL, Oracle, MS SQL Server, and MariaDB. The -:class:`~sqlalchemy.schema.Sequence` object is otherwise ignored. - -.. tip:: - - In newer database engines, the :class:`.Identity` construct should likely - be preferred vs. :class:`.Sequence` for generation of integer primary key - values. See the section :ref:`identity_ddl` for background on this - construct. - -The :class:`~sqlalchemy.schema.Sequence` may be placed on any column as a -"default" generator to be used during INSERT operations, and can also be -configured to fire off during UPDATE operations if desired. It is most -commonly used in conjunction with a single integer primary key column:: - - table = Table( - "cartitems", - metadata_obj, - Column( - "cart_id", - Integer, - Sequence("cart_id_seq", start=1), - primary_key=True, - ), - Column("description", String(40)), - Column("createdate", DateTime()), - ) - -Where above, the table ``cartitems`` is associated with a sequence named -``cart_id_seq``. Emitting :meth:`.MetaData.create_all` for the above -table will include: - -.. sourcecode:: sql - - CREATE SEQUENCE cart_id_seq START WITH 1 - - CREATE TABLE cartitems ( - cart_id INTEGER NOT NULL, - description VARCHAR(40), - createdate TIMESTAMP WITHOUT TIME ZONE, - PRIMARY KEY (cart_id) - ) - -.. tip:: - - When using tables with explicit schema names (detailed at - :ref:`schema_table_schema_name`), the configured schema of the :class:`.Table` - is **not** automatically shared by an embedded :class:`.Sequence`, instead, - specify :paramref:`.Sequence.schema`:: - - Sequence("cart_id_seq", start=1, schema="some_schema") - - The :class:`.Sequence` may also be made to automatically make use of the - :paramref:`.MetaData.schema` setting on the :class:`.MetaData` in use; - see :ref:`sequence_metadata` for background. - -When :class:`_dml.Insert` DML constructs are invoked against the ``cartitems`` -table, without an explicit value passed for the ``cart_id`` column, the -``cart_id_seq`` sequence will be used to generate a value on participating -backends. Typically, the sequence function is embedded in the INSERT statement, -which is combined with RETURNING so that the newly generated value can be -returned to the Python process: - -.. sourcecode:: sql - - INSERT INTO cartitems (cart_id, description, createdate) - VALUES (next_val(cart_id_seq), 'some description', '2015-10-15 12:00:15') - RETURNING cart_id - -When using :meth:`.Connection.execute` to invoke an :class:`_dml.Insert` construct, -newly generated primary key identifiers, including but not limited to those -generated using :class:`.Sequence`, are available from the :class:`.CursorResult` -construct using the :attr:`.CursorResult.inserted_primary_key` attribute. - -When the :class:`~sqlalchemy.schema.Sequence` is associated with a -:class:`_schema.Column` as its **Python-side** default generator, the -:class:`.Sequence` will also be subject to "CREATE SEQUENCE" and "DROP -SEQUENCE" DDL when similar DDL is emitted for the owning :class:`_schema.Table`, -such as when using :meth:`.MetaData.create_all` to generate DDL for a series -of tables. - -The :class:`.Sequence` may also be associated with a -:class:`.MetaData` construct directly. This allows the :class:`.Sequence` -to be used in more than one :class:`.Table` at a time and also allows the -:paramref:`.MetaData.schema` parameter to be inherited. See the section -:ref:`sequence_metadata` for background. - -Associating a Sequence on a SERIAL column -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -PostgreSQL's SERIAL datatype is an auto-incrementing type that implies -the implicit creation of a PostgreSQL sequence when CREATE TABLE is emitted. -The :class:`.Sequence` construct, when indicated for a :class:`_schema.Column`, -may indicate that it should not be used in this specific case by specifying -a value of ``True`` for the :paramref:`.Sequence.optional` parameter. -This allows the given :class:`.Sequence` to be used for backends that have no -alternative primary key generation system but to ignore it for backends -such as PostgreSQL which will automatically generate a sequence for a particular -column:: - - table = Table( - "cartitems", - metadata_obj, - Column( - "cart_id", - Integer, - # use an explicit Sequence where available, but not on - # PostgreSQL where SERIAL will be used - Sequence("cart_id_seq", start=1, optional=True), - primary_key=True, - ), - Column("description", String(40)), - Column("createdate", DateTime()), - ) - -In the above example, ``CREATE TABLE`` for PostgreSQL will make use of the -``SERIAL`` datatype for the ``cart_id`` column, and the ``cart_id_seq`` -sequence will be ignored. However on Oracle, the ``cart_id_seq`` sequence -will be created explicitly. - -.. tip:: - - This particular interaction of SERIAL and SEQUENCE is fairly legacy, and - as in other cases, using :class:`.Identity` instead will simplify the - operation to simply use ``IDENTITY`` on all supported backends. - - -Executing a Sequence Standalone -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -A SEQUENCE is a first class schema object in SQL and can be used to generate -values independently in the database. If you have a :class:`.Sequence` -object, it can be invoked with its "next value" instruction by -passing it directly to a SQL execution method:: - - with my_engine.connect() as conn: - seq = Sequence("some_sequence", start=1) - nextid = conn.execute(seq) - -In order to embed the "next value" function of a :class:`.Sequence` -inside of a SQL statement like a SELECT or INSERT, use the :meth:`.Sequence.next_value` -method, which will render at statement compilation time a SQL function that is -appropriate for the target backend: - -.. sourcecode:: pycon+sql - - >>> my_seq = Sequence("some_sequence", start=1) - >>> stmt = select(my_seq.next_value()) - >>> print(stmt.compile(dialect=postgresql.dialect())) - {printsql}SELECT nextval('some_sequence') AS next_value_1 - -.. _sequence_metadata: - -Associating a Sequence with the MetaData -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -For a :class:`.Sequence` that is to be associated with arbitrary -:class:`.Table` objects, the :class:`.Sequence` may be associated with -a particular :class:`_schema.MetaData`, using the -:paramref:`.Sequence.metadata` parameter:: - - seq = Sequence("my_general_seq", metadata=metadata_obj, start=1) - -Such a sequence can then be associated with columns in the usual way:: - - table = Table( - "cartitems", - metadata_obj, - seq, - Column("description", String(40)), - Column("createdate", DateTime()), - ) - -In the above example, the :class:`.Sequence` object is treated as an -independent schema construct that can exist on its own or be shared among -tables. - -Explicitly associating the :class:`.Sequence` with :class:`_schema.MetaData` -allows for the following behaviors: - -* The :class:`.Sequence` will inherit the :paramref:`_schema.MetaData.schema` - parameter specified to the target :class:`_schema.MetaData`, which - affects the production of CREATE / DROP DDL as well as how the - :meth:`.Sequence.next_value` function is rendered in SQL statements. - -* The :meth:`_schema.MetaData.create_all` and :meth:`_schema.MetaData.drop_all` - methods will emit CREATE / DROP for this :class:`.Sequence`, - even if the :class:`.Sequence` is not associated with any - :class:`_schema.Table` / :class:`_schema.Column` that's a member of this - :class:`_schema.MetaData`. - -Associating a Sequence as the Server Side Default -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. note:: The following technique is known to work only with the PostgreSQL - database. It does not work with Oracle. - -The preceding sections illustrate how to associate a :class:`.Sequence` with a -:class:`_schema.Column` as the **Python side default generator**:: - - Column( - "cart_id", - Integer, - Sequence("cart_id_seq", metadata=metadata_obj, start=1), - primary_key=True, - ) - -In the above case, the :class:`.Sequence` will automatically be subject -to CREATE SEQUENCE / DROP SEQUENCE DDL when the related :class:`_schema.Table` -is subject to CREATE / DROP. However, the sequence will **not** be present -as the server-side default for the column when CREATE TABLE is emitted. - -If we want the sequence to be used as a server-side default, -meaning it takes place even if we emit INSERT commands to the table from -the SQL command line, we can use the :paramref:`_schema.Column.server_default` -parameter in conjunction with the value-generation function of the -sequence, available from the :meth:`.Sequence.next_value` method. Below -we illustrate the same :class:`.Sequence` being associated with the -:class:`_schema.Column` both as the Python-side default generator as well as -the server-side default generator:: - - cart_id_seq = Sequence("cart_id_seq", metadata=metadata_obj, start=1) - table = Table( - "cartitems", - metadata_obj, - Column( - "cart_id", - Integer, - cart_id_seq, - server_default=cart_id_seq.next_value(), - primary_key=True, - ), - Column("description", String(40)), - Column("createdate", DateTime()), - ) - -or with the ORM:: - - class CartItem(Base): - __tablename__ = "cartitems" - - cart_id_seq = Sequence("cart_id_seq", metadata=Base.metadata, start=1) - cart_id = Column( - Integer, cart_id_seq, server_default=cart_id_seq.next_value(), primary_key=True - ) - description = Column(String(40)) - createdate = Column(DateTime) - -When the "CREATE TABLE" statement is emitted, on PostgreSQL it would be -emitted as: - -.. sourcecode:: sql - - CREATE TABLE cartitems ( - cart_id INTEGER DEFAULT nextval('cart_id_seq') NOT NULL, - description VARCHAR(40), - createdate TIMESTAMP WITHOUT TIME ZONE, - PRIMARY KEY (cart_id) - ) - -Placement of the :class:`.Sequence` in both the Python-side and server-side -default generation contexts ensures that the "primary key fetch" logic -works in all cases. Typically, sequence-enabled databases also support -RETURNING for INSERT statements, which is used automatically by SQLAlchemy -when emitting this statement. However if RETURNING is not used for a particular -insert, then SQLAlchemy would prefer to "pre-execute" the sequence outside -of the INSERT statement itself, which only works if the sequence is -included as the Python-side default generator function. - -The example also associates the :class:`.Sequence` with the enclosing -:class:`_schema.MetaData` directly, which again ensures that the :class:`.Sequence` -is fully associated with the parameters of the :class:`_schema.MetaData` collection -including the default schema, if any. - -.. seealso:: - - :ref:`postgresql_sequences` - in the PostgreSQL dialect documentation - - :ref:`oracle_returning` - in the Oracle dialect documentation - -.. _computed_ddl: - -Computed Columns (GENERATED ALWAYS AS) --------------------------------------- - -.. versionadded:: 1.3.11 - -The :class:`.Computed` construct allows a :class:`_schema.Column` to be declared in -DDL as a "GENERATED ALWAYS AS" column, that is, one which has a value that is -computed by the database server. The construct accepts a SQL expression -typically declared textually using a string or the :func:`_expression.text` construct, in -a similar manner as that of :class:`.CheckConstraint`. The SQL expression is -then interpreted by the database server in order to determine the value for the -column within a row. - -Example:: - - from sqlalchemy import Table, Column, MetaData, Integer, Computed - - metadata_obj = MetaData() - - square = Table( - "square", - metadata_obj, - Column("id", Integer, primary_key=True), - Column("side", Integer), - Column("area", Integer, Computed("side * side")), - Column("perimeter", Integer, Computed("4 * side")), - ) - -The DDL for the ``square`` table when run on a PostgreSQL 12 backend will look -like: - -.. sourcecode:: sql - - CREATE TABLE square ( - id SERIAL NOT NULL, - side INTEGER, - area INTEGER GENERATED ALWAYS AS (side * side) STORED, - perimeter INTEGER GENERATED ALWAYS AS (4 * side) STORED, - PRIMARY KEY (id) - ) - -Whether the value is persisted upon INSERT and UPDATE, or if it is calculated -on fetch, is an implementation detail of the database; the former is known as -"stored" and the latter is known as "virtual". Some database implementations -support both, but some only support one or the other. The optional -:paramref:`.Computed.persisted` flag may be specified as ``True`` or ``False`` -to indicate if the "STORED" or "VIRTUAL" keyword should be rendered in DDL, -however this will raise an error if the keyword is not supported by the target -backend; leaving it unset will use a working default for the target backend. - -The :class:`.Computed` construct is a subclass of the :class:`.FetchedValue` -object, and will set itself up as both the "server default" and "server -onupdate" generator for the target :class:`_schema.Column`, meaning it will be treated -as a default generating column when INSERT and UPDATE statements are generated, -as well as that it will be fetched as a generating column when using the ORM. -This includes that it will be part of the RETURNING clause of the database -for databases which support RETURNING and the generated values are to be -eagerly fetched. - -.. note:: A :class:`_schema.Column` that is defined with the :class:`.Computed` - construct may not store any value outside of that which the server applies - to it; SQLAlchemy's behavior when a value is passed for such a column - to be written in INSERT or UPDATE is currently that the value will be - ignored. - -"GENERATED ALWAYS AS" is currently known to be supported by: - -* MySQL version 5.7 and onwards - -* MariaDB 10.x series and onwards - -* PostgreSQL as of version 12 - -* Oracle - with the caveat that RETURNING does not work correctly with UPDATE - (a warning will be emitted to this effect when the UPDATE..RETURNING that - includes a computed column is rendered) - -* Microsoft SQL Server - -* SQLite as of version 3.31 - -When :class:`.Computed` is used with an unsupported backend, if the target -dialect does not support it, a :class:`.CompileError` is raised when attempting -to render the construct. Otherwise, if the dialect supports it but the -particular database server version in use does not, then a subclass of -:class:`.DBAPIError`, usually :class:`.OperationalError`, is raised when the -DDL is emitted to the database. - -.. seealso:: - - :class:`.Computed` - -.. _identity_ddl: - -Identity Columns (GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY) ------------------------------------------------------------------ - -.. versionadded:: 1.4 - -The :class:`.Identity` construct allows a :class:`_schema.Column` to be declared -as an identity column and rendered in DDL as "GENERATED { ALWAYS | BY DEFAULT } -AS IDENTITY". An identity column has its value automatically generated by the -database server using an incrementing (or decrementing) sequence. The construct -shares most of its option to control the database behaviour with -:class:`.Sequence`. - -Example:: - - from sqlalchemy import Table, Column, MetaData, Integer, Identity, String - - metadata_obj = MetaData() - - data = Table( - "data", - metadata_obj, - Column("id", Integer, Identity(start=42, cycle=True), primary_key=True), - Column("data", String), - ) - -The DDL for the ``data`` table when run on a PostgreSQL 12 backend will look -like: - -.. sourcecode:: sql - - CREATE TABLE data ( - id INTEGER GENERATED BY DEFAULT AS IDENTITY (START WITH 42 CYCLE) NOT NULL, - data VARCHAR, - PRIMARY KEY (id) - ) - -The database will generate a value for the ``id`` column upon insert, -starting from ``42``, if the statement did not already contain a value for -the ``id`` column. -An identity column can also require that the database generates the value -of the column, ignoring the value passed with the statement or raising an -error, depending on the backend. To activate this mode, set the parameter -:paramref:`_schema.Identity.always` to ``True`` in the -:class:`.Identity` construct. Updating the previous -example to include this parameter will generate the following DDL: - -.. sourcecode:: sql - - CREATE TABLE data ( - id INTEGER GENERATED ALWAYS AS IDENTITY (START WITH 42 CYCLE) NOT NULL, - data VARCHAR, - PRIMARY KEY (id) - ) - -The :class:`.Identity` construct is a subclass of the :class:`.FetchedValue` -object, and will set itself up as the "server default" generator for the -target :class:`_schema.Column`, meaning it will be treated -as a default generating column when INSERT statements are generated, -as well as that it will be fetched as a generating column when using the ORM. -This includes that it will be part of the RETURNING clause of the database -for databases which support RETURNING and the generated values are to be -eagerly fetched. - -The :class:`.Identity` construct is currently known to be supported by: - -* PostgreSQL as of version 10. - -* Oracle as of version 12. It also supports passing ``always=None`` to - enable the default generated mode and the parameter ``on_null=True`` to - specify "ON NULL" in conjunction with a "BY DEFAULT" identity column. - -* Microsoft SQL Server. MSSQL uses a custom syntax that only supports the - ``start`` and ``increment`` parameters, and ignores all other. - -When :class:`.Identity` is used with an unsupported backend, it is ignored, -and the default SQLAlchemy logic for autoincrementing columns is used. - -An error is raised when a :class:`_schema.Column` specifies both an -:class:`.Identity` and also sets :paramref:`_schema.Column.autoincrement` -to ``False``. - -.. seealso:: - - :class:`.Identity` - - -Default Objects API -------------------- - -.. autoclass:: Computed - :members: - - -.. autoclass:: ColumnDefault - - -.. autoclass:: DefaultClause - - -.. autoclass:: DefaultGenerator - - -.. autoclass:: FetchedValue - - -.. autoclass:: Sequence - :members: - - -.. autoclass:: Identity - :members: diff --git a/doc/build/core/engines.rst b/doc/build/core/engines.rst index 696f058e6e9..5521978d4b1 100644 --- a/doc/build/core/engines.rst +++ b/doc/build/core/engines.rst @@ -192,7 +192,7 @@ MySQL DBAPIs available, including PyMySQL:: # PyMySQL engine = create_engine("mysql+pymysql://scott:tiger@localhost/foo") -More notes on connecting to MySQL at :ref:`mysql_toplevel`. +More notes on connecting to MySQL at ref_mysql_toplevel. Oracle ^^^^^^^^^^ @@ -203,7 +203,7 @@ The Oracle dialect uses cx_oracle as the default DBAPI:: engine = create_engine("oracle+cx_oracle://scott:tiger@tnsname") -More notes on connecting to Oracle at :ref:`oracle_toplevel`. +More notes on connecting to Oracle at ref_oracle_toplevel. Microsoft SQL Server ^^^^^^^^^^^^^^^^^^^^ @@ -217,7 +217,7 @@ also available:: # pymssql engine = create_engine("mssql+pymssql://scott:tiger@hostname:port/dbname") -More notes on connecting to SQL Server at :ref:`mssql_toplevel`. +More notes on connecting to SQL Server at ref_mssql_toplevel. SQLite ^^^^^^^ @@ -248,7 +248,7 @@ To use a SQLite ``:memory:`` database, specify an empty URL:: engine = create_engine("sqlite://") -More notes on connecting to SQLite at :ref:`sqlite_toplevel`. +More notes on connecting to SQLite at ref_sqlite_toplevel. Others ^^^^^^ @@ -291,7 +291,7 @@ application, rather than creating a new one for each connection. .. note:: :class:`.QueuePool` is not used by default for SQLite engines. See - :ref:`sqlite_toplevel` for details on SQLite connection pool usage. + ref_sqlite_toplevel for details on SQLite connection pool usage. For more information on connection pooling, see :ref:`pooling_toplevel`. @@ -424,7 +424,7 @@ parameter, this could be implemented as:: .. seealso:: - :ref:`mssql_pyodbc_access_tokens` - a more concrete example involving + ref_mssql_pyodbc_access_tokens - a more concrete example involving SQL Server Modifying the DBAPI connection after connect, or running commands after connect diff --git a/doc/build/core/expression_api.rst b/doc/build/core/expression_api.rst index 8000735a11e..8802e15473c 100644 --- a/doc/build/core/expression_api.rst +++ b/doc/build/core/expression_api.rst @@ -17,8 +17,5 @@ in the :ref:`unified_tutorial`. operators selectable dml - functions compiler - serializer foundation - visitors diff --git a/doc/build/core/functions.rst b/doc/build/core/functions.rst deleted file mode 100644 index 6fcee6edaa2..00000000000 --- a/doc/build/core/functions.rst +++ /dev/null @@ -1,143 +0,0 @@ -.. _functions_toplevel: -.. _generic_functions: - -========================= -SQL and Generic Functions -========================= - -.. currentmodule:: sqlalchemy.sql.functions - -SQL functions are invoked by using the :data:`_sql.func` namespace. -See the tutorial at :ref:`tutorial_functions` for background on how to -use the :data:`_sql.func` object to render SQL functions in statements. - -.. seealso:: - - :ref:`tutorial_functions` - in the :ref:`unified_tutorial` - -Function API ------------- - -The base API for SQL functions, which provides for the :data:`_sql.func` -namespace as well as classes that may be used for extensibility. - -.. autoclass:: AnsiFunction - :exclude-members: inherit_cache, __new__ - -.. autoclass:: Function - -.. autoclass:: FunctionElement - :members: - :exclude-members: inherit_cache, __new__ - -.. autoclass:: GenericFunction - :exclude-members: inherit_cache, __new__ - -.. autofunction:: register_function - - -Selected "Known" Functions --------------------------- - -These are :class:`.GenericFunction` implementations for a selected set of -common SQL functions that set up the expected return type for each function -automatically. The are invoked in the same way as any other member of the -:data:`_sql.func` namespace:: - - select(func.count("*")).select_from(some_table) - -Note that any name not known to :data:`_sql.func` generates the function name -as is - there is no restriction on what SQL functions can be called, known or -unknown to SQLAlchemy, built-in or user defined. The section here only -describes those functions where SQLAlchemy already knows what argument and -return types are in use. - -.. autoclass:: array_agg - :no-members: - -.. autoclass:: char_length - :no-members: - -.. autoclass:: coalesce - :no-members: - -.. autoclass:: concat - :no-members: - -.. autoclass:: count - :no-members: - -.. autoclass:: cube - :no-members: - -.. autoclass:: cume_dist - :no-members: - -.. autoclass:: current_date - :no-members: - -.. autoclass:: current_time - :no-members: - -.. autoclass:: current_timestamp - :no-members: - -.. autoclass:: current_user - :no-members: - -.. autoclass:: dense_rank - :no-members: - -.. autoclass:: grouping_sets - :no-members: - -.. autoclass:: localtime - :no-members: - -.. autoclass:: localtimestamp - :no-members: - -.. autoclass:: max - :no-members: - -.. autoclass:: min - :no-members: - -.. autoclass:: mode - :no-members: - -.. autoclass:: next_value - :no-members: - -.. autoclass:: now - :no-members: - -.. autoclass:: percent_rank - :no-members: - -.. autoclass:: percentile_cont - :no-members: - -.. autoclass:: percentile_disc - :no-members: - -.. autoclass:: random - :no-members: - -.. autoclass:: rank - :no-members: - -.. autoclass:: rollup - :no-members: - -.. autoclass:: session_user - :no-members: - -.. autoclass:: sum - :no-members: - -.. autoclass:: sysdate - :no-members: - -.. autoclass:: user - :no-members: diff --git a/doc/build/core/future.rst b/doc/build/core/future.rst deleted file mode 100644 index 9c171b9db58..00000000000 --- a/doc/build/core/future.rst +++ /dev/null @@ -1,17 +0,0 @@ -:orphan: - -SQLAlchemy 2.0 Future (Core) -============================ - -.. admonition:: We're 2.0! - - This page described the "future" mode provided in SQLAlchemy 1.4 - for the purposes of 1.4 -> 2.0 transition. For 2.0, the "future" - parameter on :func:`_sa.create_engine` and :class:`_orm.Session` - continues to remain available for backwards-compatibility support, however - if specified must be left at the value of ``True``. - - .. seealso:: - - :ref:`migration_20_toplevel` - Introduction to the 2.0 series of SQLAlchemy - diff --git a/doc/build/core/index.rst b/doc/build/core/index.rst index 764247ab566..4629bd82e8e 100644 --- a/doc/build/core/index.rst +++ b/doc/build/core/index.rst @@ -12,8 +12,8 @@ Language provides a schema-centric usage paradigm. :maxdepth: 2 expression_api + engines_connections schema types - engines_connections api_basics diff --git a/doc/build/core/inspection.rst b/doc/build/core/inspection.rst deleted file mode 100644 index 7816cd3fd8c..00000000000 --- a/doc/build/core/inspection.rst +++ /dev/null @@ -1,49 +0,0 @@ -.. _core_inspection_toplevel: -.. _inspection_toplevel: - -Runtime Inspection API -====================== - -.. automodule:: sqlalchemy.inspection - -.. autofunction:: sqlalchemy.inspect - - -Available Inspection Targets ----------------------------- - -Below is a listing of many of the most common inspection targets. - -* :class:`.Connectable` (i.e. :class:`_engine.Engine`, - :class:`_engine.Connection`) - returns an :class:`_reflection.Inspector` object. -* :class:`_expression.ClauseElement` - all SQL expression components, including - :class:`_schema.Table`, :class:`_schema.Column`, serve as their own inspection objects, - meaning any of these objects passed to :func:`_sa.inspect` return themselves. -* ``object`` - an object given will be checked by the ORM for a mapping - - if so, an :class:`.InstanceState` is returned representing the mapped - state of the object. The :class:`.InstanceState` also provides access - to per attribute state via the :class:`.AttributeState` interface as well - as the per-flush "history" of any attribute via the :class:`.History` - object. - - .. seealso:: - - :ref:`orm_mapper_inspection_instancestate` - -* ``type`` (i.e. a class) - a class given will be checked by the ORM for a - mapping - if so, a :class:`_orm.Mapper` for that class is returned. - - .. seealso:: - - :ref:`orm_mapper_inspection_mapper` - -* mapped attribute - passing a mapped attribute to :func:`_sa.inspect`, such - as ``inspect(MyClass.some_attribute)``, returns a :class:`.QueryableAttribute` - object, which is the :term:`descriptor` associated with a mapped class. - This descriptor refers to a :class:`.MapperProperty`, which is usually - an instance of :class:`.ColumnProperty` - or :class:`.RelationshipProperty`, via its :attr:`.QueryableAttribute.property` - attribute. -* :class:`.AliasedClass` - returns an :class:`.AliasedInsp` object. - - diff --git a/doc/build/core/metadata.rst b/doc/build/core/metadata.rst index 49669705628..ac7a740bf52 100644 --- a/doc/build/core/metadata.rst +++ b/doc/build/core/metadata.rst @@ -401,10 +401,10 @@ at once, such as:: .. seealso:: - :ref:`multipart_schema_names` - describes use of dotted schema names + ref_multipart_schema_names - describes use of dotted schema names with the SQL Server dialect. - :ref:`metadata_reflection_schemas` + ref_metadata_reflection_schemas .. _schema_metadata_schema_name: @@ -543,7 +543,7 @@ for specific information regarding how default schemas are configured. .. seealso:: - :ref:`postgresql_alternate_search_path` - in the :ref:`postgresql_toplevel` dialect documentation. + ref_postgresql_alternate_search_path - in the :ref:`postgresql_toplevel` dialect documentation. @@ -552,8 +552,8 @@ Schemas and Reflection ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The schema feature of SQLAlchemy interacts with the table reflection -feature introduced at :ref:`metadata_reflection_toplevel`. See the section -:ref:`metadata_reflection_schemas` for additional details on how this works. +feature introduced at ref_metadata_reflection_toplevel. See the section +ref_metadata_reflection_schemas for additional details on how this works. Backend-Specific Options diff --git a/doc/build/core/pooling.rst b/doc/build/core/pooling.rst index 8e7f1592d02..49993a792f3 100644 --- a/doc/build/core/pooling.rst +++ b/doc/build/core/pooling.rst @@ -214,8 +214,8 @@ consistent with the state of the transaction:: .. seealso:: - * :ref:`mssql_reset_on_return` - in the :ref:`mssql_toplevel` documentation - * :ref:`postgresql_reset_on_return` in the :ref:`postgresql_toplevel` documentation + * ref_mssql_reset_on_return - in the ref_mssql_toplevel documentation + * ref_postgresql_reset_on_return in the :ref:`postgresql_toplevel` documentation diff --git a/doc/build/core/reflection.rst b/doc/build/core/reflection.rst deleted file mode 100644 index 1ff517510ec..00000000000 --- a/doc/build/core/reflection.rst +++ /dev/null @@ -1,556 +0,0 @@ -.. currentmodule:: sqlalchemy.schema - -.. _metadata_reflection_toplevel: -.. _metadata_reflection: - - -Reflecting Database Objects -=========================== - -A :class:`~sqlalchemy.schema.Table` object can be instructed to load -information about itself from the corresponding database schema object already -existing within the database. This process is called *reflection*. In the -most simple case you need only specify the table name, a :class:`~sqlalchemy.schema.MetaData` -object, and the ``autoload_with`` argument:: - - >>> messages = Table("messages", metadata_obj, autoload_with=engine) - >>> [c.name for c in messages.columns] - ['message_id', 'message_name', 'date'] - -The above operation will use the given engine to query the database for -information about the ``messages`` table, and will then generate -:class:`~sqlalchemy.schema.Column`, :class:`~sqlalchemy.schema.ForeignKey`, -and other objects corresponding to this information as though the -:class:`~sqlalchemy.schema.Table` object were hand-constructed in Python. - -When tables are reflected, if a given table references another one via foreign -key, a second :class:`~sqlalchemy.schema.Table` object is created within the -:class:`~sqlalchemy.schema.MetaData` object representing the connection. -Below, assume the table ``shopping_cart_items`` references a table named -``shopping_carts``. Reflecting the ``shopping_cart_items`` table has the -effect such that the ``shopping_carts`` table will also be loaded:: - - >>> shopping_cart_items = Table("shopping_cart_items", metadata_obj, autoload_with=engine) - >>> "shopping_carts" in metadata_obj.tables - True - -The :class:`~sqlalchemy.schema.MetaData` has an interesting "singleton-like" -behavior such that if you requested both tables individually, -:class:`~sqlalchemy.schema.MetaData` will ensure that exactly one -:class:`~sqlalchemy.schema.Table` object is created for each distinct table -name. The :class:`~sqlalchemy.schema.Table` constructor actually returns to -you the already-existing :class:`~sqlalchemy.schema.Table` object if one -already exists with the given name. Such as below, we can access the already -generated ``shopping_carts`` table just by naming it:: - - shopping_carts = Table("shopping_carts", metadata_obj) - -Of course, it's a good idea to use ``autoload_with=engine`` with the above table -regardless. This is so that the table's attributes will be loaded if they have -not been already. The autoload operation only occurs for the table if it -hasn't already been loaded; once loaded, new calls to -:class:`~sqlalchemy.schema.Table` with the same name will not re-issue any -reflection queries. - -.. _reflection_overriding_columns: - -Overriding Reflected Columns ----------------------------- - -Individual columns can be overridden with explicit values when reflecting -tables; this is handy for specifying custom datatypes, constraints such as -primary keys that may not be configured within the database, etc.:: - - >>> mytable = Table( - ... "mytable", - ... metadata_obj, - ... Column( - ... "id", Integer, primary_key=True - ... ), # override reflected 'id' to have primary key - ... Column("mydata", Unicode(50)), # override reflected 'mydata' to be Unicode - ... # additional Column objects which require no change are reflected normally - ... autoload_with=some_engine, - ... ) - -.. seealso:: - - :ref:`custom_and_decorated_types_reflection` - illustrates how the above - column override technique applies to the use of custom datatypes with - table reflection. - -Reflecting Views ----------------- - -The reflection system can also reflect views. Basic usage is the same as that -of a table:: - - my_view = Table("some_view", metadata, autoload_with=engine) - -Above, ``my_view`` is a :class:`~sqlalchemy.schema.Table` object with -:class:`~sqlalchemy.schema.Column` objects representing the names and types of -each column within the view "some_view". - -Usually, it's desired to have at least a primary key constraint when -reflecting a view, if not foreign keys as well. View reflection doesn't -extrapolate these constraints. - -Use the "override" technique for this, specifying explicitly those columns -which are part of the primary key or have foreign key constraints:: - - my_view = Table( - "some_view", - metadata, - Column("view_id", Integer, primary_key=True), - Column("related_thing", Integer, ForeignKey("othertable.thing_id")), - autoload_with=engine, - ) - -Reflecting All Tables at Once ------------------------------ - -The :class:`~sqlalchemy.schema.MetaData` object can also get a listing of -tables and reflect the full set. This is achieved by using the -:func:`~sqlalchemy.schema.MetaData.reflect` method. After calling it, all -located tables are present within the :class:`~sqlalchemy.schema.MetaData` -object's dictionary of tables:: - - metadata_obj = MetaData() - metadata_obj.reflect(bind=someengine) - users_table = metadata_obj.tables["users"] - addresses_table = metadata_obj.tables["addresses"] - -``metadata.reflect()`` also provides a handy way to clear or delete all the rows in a database:: - - metadata_obj = MetaData() - metadata_obj.reflect(bind=someengine) - for table in reversed(metadata_obj.sorted_tables): - someengine.execute(table.delete()) - -.. _metadata_reflection_schemas: - -Reflecting Tables from Other Schemas ------------------------------------- - -The section :ref:`schema_table_schema_name` introduces the concept of table -schemas, which are namespaces within a database that contain tables and other -objects, and which can be specified explicitly. The "schema" for a -:class:`_schema.Table` object, as well as for other objects like views, indexes and -sequences, can be set up using the :paramref:`_schema.Table.schema` parameter, -and also as the default schema for a :class:`_schema.MetaData` object using the -:paramref:`_schema.MetaData.schema` parameter. - -The use of this schema parameter directly affects where the table reflection -feature will look when it is asked to reflect objects. For example, given -a :class:`_schema.MetaData` object configured with a default schema name -"project" via its :paramref:`_schema.MetaData.schema` parameter:: - - >>> metadata_obj = MetaData(schema="project") - -The :meth:`.MetaData.reflect` will then utilize that configured ``.schema`` -for reflection:: - - >>> # uses `schema` configured in metadata_obj - >>> metadata_obj.reflect(someengine) - -The end result is that :class:`_schema.Table` objects from the "project" -schema will be reflected, and they will be populated as schema-qualified -with that name:: - - >>> metadata_obj.tables["project.messages"] - Table('messages', MetaData(), Column('message_id', INTEGER(), table=), schema='project') - -Similarly, an individual :class:`_schema.Table` object that includes the -:paramref:`_schema.Table.schema` parameter will also be reflected from that -database schema, overriding any default schema that may have been configured on the -owning :class:`_schema.MetaData` collection:: - - >>> messages = Table("messages", metadata_obj, schema="project", autoload_with=someengine) - >>> messages - Table('messages', MetaData(), Column('message_id', INTEGER(), table=), schema='project') - -Finally, the :meth:`_schema.MetaData.reflect` method itself also allows a -:paramref:`_schema.MetaData.reflect.schema` parameter to be passed, so we -could also load tables from the "project" schema for a default configured -:class:`_schema.MetaData` object:: - - >>> metadata_obj = MetaData() - >>> metadata_obj.reflect(someengine, schema="project") - -We can call :meth:`_schema.MetaData.reflect` any number of times with different -:paramref:`_schema.MetaData.schema` arguments (or none at all) to continue -populating the :class:`_schema.MetaData` object with more objects:: - - >>> # add tables from the "customer" schema - >>> metadata_obj.reflect(someengine, schema="customer") - >>> # add tables from the default schema - >>> metadata_obj.reflect(someengine) - -.. _reflection_schema_qualified_interaction: - -Interaction of Schema-qualified Reflection with the Default Schema -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. admonition:: Section Best Practices Summarized - - In this section, we discuss SQLAlchemy's reflection behavior regarding - tables that are visible in the "default schema" of a database session, - and how these interact with SQLAlchemy directives that include the schema - explicitly. As a best practice, ensure the "default" schema for a database - is just a single name, and not a list of names; for tables that are - part of this "default" schema and can be named without schema qualification - in DDL and SQL, leave corresponding :paramref:`_schema.Table.schema` and - similar schema parameters set to their default of ``None``. - -As described at :ref:`schema_metadata_schema_name`, databases that have -the concept of schemas usually also include the concept of a "default" schema. -The reason for this is naturally that when one refers to table objects without -a schema as is common, a schema-capable database will still consider that -table to be in a "schema" somewhere. Some databases such as PostgreSQL -take this concept further into the notion of a -`schema search path -`_ -where *multiple* schema names can be considered in a particular database -session to be "implicit"; referring to a table name that it's any of those -schemas will not require that the schema name be present (while at the same time -it's also perfectly fine if the schema name *is* present). - -Since most relational databases therefore have the concept of a particular -table object which can be referred towards both in a schema-qualified way, as -well as an "implicit" way where no schema is present, this presents a -complexity for SQLAlchemy's reflection -feature. Reflecting a table in -a schema-qualified manner will always populate its :attr:`_schema.Table.schema` -attribute and additionally affect how this :class:`_schema.Table` is organized -into the :attr:`_schema.MetaData.tables` collection, that is, in a schema -qualified manner. Conversely, reflecting the **same** table in a non-schema -qualified manner will organize it into the :attr:`_schema.MetaData.tables` -collection **without** being schema qualified. The end result is that there -would be two separate :class:`_schema.Table` objects in the single -:class:`_schema.MetaData` collection representing the same table in the -actual database. - -To illustrate the ramifications of this issue, consider tables from the -"project" schema in the previous example, and suppose also that the "project" -schema is the default schema of our database connection, or if using a database -such as PostgreSQL suppose the "project" schema is set up in the PostgreSQL -``search_path``. This would mean that the database accepts the following -two SQL statements as equivalent: - -.. sourcecode:: sql - - -- schema qualified - SELECT message_id FROM project.messages - - -- non-schema qualified - SELECT message_id FROM messages - -This is not a problem as the table can be found in both ways. However -in SQLAlchemy, it's the **identity** of the :class:`_schema.Table` object -that determines its semantic role within a SQL statement. Based on the current -decisions within SQLAlchemy, this means that if we reflect the same "messages" table in -both a schema-qualified as well as a non-schema qualified manner, we get -**two** :class:`_schema.Table` objects that will **not** be treated as -semantically equivalent:: - - >>> # reflect in non-schema qualified fashion - >>> messages_table_1 = Table("messages", metadata_obj, autoload_with=someengine) - >>> # reflect in schema qualified fashion - >>> messages_table_2 = Table( - ... "messages", metadata_obj, schema="project", autoload_with=someengine - ... ) - >>> # two different objects - >>> messages_table_1 is messages_table_2 - False - >>> # stored in two different ways - >>> metadata.tables["messages"] is messages_table_1 - True - >>> metadata.tables["project.messages"] is messages_table_2 - True - -The above issue becomes more complicated when the tables being reflected contain -foreign key references to other tables. Suppose "messages" has a "project_id" -column which refers to rows in another schema-local table "projects", meaning -there is a :class:`_schema.ForeignKeyConstraint` object that is part of the -definition of the "messages" table. - -We can find ourselves in a situation where one :class:`_schema.MetaData` -collection may contain as many as four :class:`_schema.Table` objects -representing these two database tables, where one or two of the additional -tables were generated by the reflection process; this is because when -the reflection process encounters a foreign key constraint on a table -being reflected, it branches out to reflect that referenced table as well. -The decision making it uses to assign the schema to this referenced -table is that SQLAlchemy will **omit a default schema** from the reflected -:class:`_schema.ForeignKeyConstraint` object if the owning -:class:`_schema.Table` also omits its schema name and also that these two objects -are in the same schema, but will **include** it if -it were not omitted. - -The common scenario is when the reflection of a table in a schema qualified -fashion then loads a related table that will also be performed in a schema -qualified fashion:: - - >>> # reflect "messages" in a schema qualified fashion - >>> messages_table_1 = Table( - ... "messages", metadata_obj, schema="project", autoload_with=someengine - ... ) - -The above ``messages_table_1`` will refer to ``projects`` also in a schema -qualified fashion. This "projects" table will be reflected automatically by -the fact that "messages" refers to it:: - - >>> messages_table_1.c.project_id - Column('project_id', INTEGER(), ForeignKey('project.projects.project_id'), table=) - -if some other part of the code reflects "projects" in a non-schema qualified -fashion, there are now two projects tables that are not the same: - - >>> # reflect "projects" in a non-schema qualified fashion - >>> projects_table_1 = Table("projects", metadata_obj, autoload_with=someengine) - - >>> # messages does not refer to projects_table_1 above - >>> messages_table_1.c.project_id.references(projects_table_1.c.project_id) - False - - >>> # it refers to this one - >>> projects_table_2 = metadata_obj.tables["project.projects"] - >>> messages_table_1.c.project_id.references(projects_table_2.c.project_id) - True - - >>> # they're different, as one non-schema qualified and the other one is - >>> projects_table_1 is projects_table_2 - False - -The above confusion can cause problems within applications that use table -reflection to load up application-level :class:`_schema.Table` objects, as -well as within migration scenarios, in particular such as when using Alembic -Migrations to detect new tables and foreign key constraints. - -The above behavior can be remedied by sticking to one simple practice: - -* Don't include the :paramref:`_schema.Table.schema` parameter for any - :class:`_schema.Table` that expects to be located in the **default** schema - of the database. - -For PostgreSQL and other databases that support a "search" path for schemas, -add the following additional practice: - -* Keep the "search path" narrowed down to **one schema only, which is the - default schema**. - - -.. seealso:: - - :ref:`postgresql_schema_reflection` - additional details of this behavior - as regards the PostgreSQL database. - - -.. _metadata_reflection_inspector: - -Fine Grained Reflection with Inspector --------------------------------------- - -A low level interface which provides a backend-agnostic system of loading -lists of schema, table, column, and constraint descriptions from a given -database is also available. This is known as the "Inspector":: - - from sqlalchemy import create_engine - from sqlalchemy import inspect - - engine = create_engine("...") - insp = inspect(engine) - print(insp.get_table_names()) - -.. autoclass:: sqlalchemy.engine.reflection.Inspector - :members: - :undoc-members: - -.. autoclass:: sqlalchemy.engine.interfaces.ReflectedColumn - :members: - :inherited-members: dict - -.. autoclass:: sqlalchemy.engine.interfaces.ReflectedComputed - :members: - :inherited-members: dict - -.. autoclass:: sqlalchemy.engine.interfaces.ReflectedCheckConstraint - :members: - :inherited-members: dict - -.. autoclass:: sqlalchemy.engine.interfaces.ReflectedForeignKeyConstraint - :members: - :inherited-members: dict - -.. autoclass:: sqlalchemy.engine.interfaces.ReflectedIdentity - :members: - :inherited-members: dict - -.. autoclass:: sqlalchemy.engine.interfaces.ReflectedIndex - :members: - :inherited-members: dict - -.. autoclass:: sqlalchemy.engine.interfaces.ReflectedPrimaryKeyConstraint - :members: - :inherited-members: dict - -.. autoclass:: sqlalchemy.engine.interfaces.ReflectedUniqueConstraint - :members: - :inherited-members: dict - -.. autoclass:: sqlalchemy.engine.interfaces.ReflectedTableComment - :members: - :inherited-members: dict - - -.. _metadata_reflection_dbagnostic_types: - -Reflecting with Database-Agnostic Types ---------------------------------------- - -When the columns of a table are reflected, using either the -:paramref:`_schema.Table.autoload_with` parameter of :class:`_schema.Table` or -the :meth:`_reflection.Inspector.get_columns` method of -:class:`_reflection.Inspector`, the datatypes will be as specific as possible -to the target database. This means that if an "integer" datatype is reflected -from a MySQL database, the type will be represented by the -:class:`sqlalchemy.dialects.mysql.INTEGER` class, which includes MySQL-specific -attributes such as "display_width". Or on PostgreSQL, a PostgreSQL-specific -datatype such as :class:`sqlalchemy.dialects.postgresql.INTERVAL` or -:class:`sqlalchemy.dialects.postgresql.ENUM` may be returned. - -There is a use case for reflection which is that a given :class:`_schema.Table` -is to be transferred to a different vendor database. To suit this use case, -there is a technique by which these vendor-specific datatypes can be converted -on the fly to be instance of SQLAlchemy backend-agnostic datatypes, for -the examples above types such as :class:`_types.Integer`, :class:`_types.Interval` -and :class:`_types.Enum`. This may be achieved by intercepting the -column reflection using the :meth:`_events.DDLEvents.column_reflect` event -in conjunction with the :meth:`_types.TypeEngine.as_generic` method. - -Given a table in MySQL (chosen because MySQL has a lot of vendor-specific -datatypes and options): - -.. sourcecode:: sql - - CREATE TABLE IF NOT EXISTS my_table ( - id INTEGER PRIMARY KEY AUTO_INCREMENT, - data1 VARCHAR(50) CHARACTER SET latin1, - data2 MEDIUMINT(4), - data3 TINYINT(2) - ) - -The above table includes MySQL-only integer types ``MEDIUMINT`` and -``TINYINT`` as well as a ``VARCHAR`` that includes the MySQL-only ``CHARACTER -SET`` option. If we reflect this table normally, it produces a -:class:`_schema.Table` object that will contain those MySQL-specific datatypes -and options: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy import MetaData, Table, create_engine - >>> mysql_engine = create_engine("mysql+mysqldb://scott:tiger@localhost/test") - >>> metadata_obj = MetaData() - >>> my_mysql_table = Table("my_table", metadata_obj, autoload_with=mysql_engine) - -The above example reflects the above table schema into a new :class:`_schema.Table` -object. We can then, for demonstration purposes, print out the MySQL-specific -"CREATE TABLE" statement using the :class:`_schema.CreateTable` construct: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy.schema import CreateTable - >>> print(CreateTable(my_mysql_table).compile(mysql_engine)) - {printsql}CREATE TABLE my_table ( - id INTEGER(11) NOT NULL AUTO_INCREMENT, - data1 VARCHAR(50) CHARACTER SET latin1, - data2 MEDIUMINT(4), - data3 TINYINT(2), - PRIMARY KEY (id) - )ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 - - -Above, the MySQL-specific datatypes and options were maintained. If we wanted -a :class:`_schema.Table` that we could instead transfer cleanly to another -database vendor, replacing the special datatypes -:class:`sqlalchemy.dialects.mysql.MEDIUMINT` and -:class:`sqlalchemy.dialects.mysql.TINYINT` with :class:`_types.Integer`, we can -choose instead to "genericize" the datatypes on this table, or otherwise change -them in any way we'd like, by establishing a handler using the -:meth:`_events.DDLEvents.column_reflect` event. The custom handler will make use -of the :meth:`_types.TypeEngine.as_generic` method to convert the above -MySQL-specific type objects into generic ones, by replacing the ``"type"`` -entry within the column dictionary entry that is passed to the event handler. -The format of this dictionary is described at :meth:`_reflection.Inspector.get_columns`: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy import event - >>> metadata_obj = MetaData() - - >>> @event.listens_for(metadata_obj, "column_reflect") - ... def genericize_datatypes(inspector, tablename, column_dict): - ... column_dict["type"] = column_dict["type"].as_generic() - - >>> my_generic_table = Table("my_table", metadata_obj, autoload_with=mysql_engine) - -We now get a new :class:`_schema.Table` that is generic and uses -:class:`_types.Integer` for those datatypes. We can now emit a -"CREATE TABLE" statement for example on a PostgreSQL database: - -.. sourcecode:: pycon+sql - - >>> pg_engine = create_engine("postgresql+psycopg2://scott:tiger@localhost/test", echo=True) - >>> my_generic_table.create(pg_engine) - {execsql}CREATE TABLE my_table ( - id SERIAL NOT NULL, - data1 VARCHAR(50), - data2 INTEGER, - data3 INTEGER, - PRIMARY KEY (id) - ) - -Noting above also that SQLAlchemy will usually make a decent guess for other -behaviors, such as that the MySQL ``AUTO_INCREMENT`` directive is represented -in PostgreSQL most closely using the ``SERIAL`` auto-incrementing datatype. - -.. versionadded:: 1.4 Added the :meth:`_types.TypeEngine.as_generic` method - and additionally improved the use of the :meth:`_events.DDLEvents.column_reflect` - event such that it may be applied to a :class:`_schema.MetaData` object - for convenience. - - -Limitations of Reflection -------------------------- - -It's important to note that the reflection process recreates :class:`_schema.Table` -metadata using only information which is represented in the relational database. -This process by definition cannot restore aspects of a schema that aren't -actually stored in the database. State which is not available from reflection -includes but is not limited to: - -* Client side defaults, either Python functions or SQL expressions defined using - the ``default`` keyword of :class:`_schema.Column` (note this is separate from ``server_default``, - which specifically is what's available via reflection). - -* Column information, e.g. data that might have been placed into the - :attr:`_schema.Column.info` dictionary - -* The value of the ``.quote`` setting for :class:`_schema.Column` or :class:`_schema.Table` - -* The association of a particular :class:`.Sequence` with a given :class:`_schema.Column` - -The relational database also in many cases reports on table metadata in a -different format than what was specified in SQLAlchemy. The :class:`_schema.Table` -objects returned from reflection cannot be always relied upon to produce the identical -DDL as the original Python-defined :class:`_schema.Table` objects. Areas where -this occurs includes server defaults, column-associated sequences and various -idiosyncrasies regarding constraints and datatypes. Server side defaults may -be returned with cast directives (typically PostgreSQL will include a ``::`` -cast) or different quoting patterns than originally specified. - -Another category of limitation includes schema structures for which reflection -is only partially or not yet defined. Recent improvements to reflection allow -things like views, indexes and foreign key options to be reflected. As of this -writing, structures like CHECK constraints, table comments, and triggers are -not reflected. - diff --git a/doc/build/core/schema.rst b/doc/build/core/schema.rst index 5a4f939bf7e..670e4726837 100644 --- a/doc/build/core/schema.rst +++ b/doc/build/core/schema.rst @@ -36,8 +36,6 @@ in creating real schema generation scripts. :maxdepth: 3 metadata - reflection - defaults constraints ddl diff --git a/doc/build/core/serializer.rst b/doc/build/core/serializer.rst deleted file mode 100644 index 5423306a92d..00000000000 --- a/doc/build/core/serializer.rst +++ /dev/null @@ -1,6 +0,0 @@ -Expression Serializer Extension -=============================== - -.. automodule:: sqlalchemy.ext.serializer - :members: - :undoc-members: diff --git a/doc/build/core/visitors.rst b/doc/build/core/visitors.rst deleted file mode 100644 index 06d839d54cb..00000000000 --- a/doc/build/core/visitors.rst +++ /dev/null @@ -1,27 +0,0 @@ -Visitor and Traversal Utilities -================================ - -The :mod:`sqlalchemy.sql.visitors` module consists of classes and functions -that serve the purpose of generically **traversing** a Core SQL expression -structure. This is not unlike the Python ``ast`` module in that is presents -a system by which a program can operate upon each component of a SQL -expression. Common purposes this serves are locating various kinds of -elements such as :class:`_schema.Table` or :class:`.BindParameter` objects, -as well as altering the state of the structure such as replacing certain FROM -clauses with others. - -.. note:: the :mod:`sqlalchemy.sql.visitors` module is an internal API and - is not fully public. It is subject to change and may additionally not - function as expected for use patterns that aren't considered within - SQLAlchemy's own internals. - -The :mod:`sqlalchemy.sql.visitors` module is part of the **internals** of -SQLAlchemy and it is not usually used by calling application code. It is -however used in certain edge cases such as when constructing caching routines -as well as when building out custom SQL expressions using the -:ref:`Custom SQL Constructs and Compilation Extension `. - -.. automodule:: sqlalchemy.sql.visitors - :members: - :private-members: - diff --git a/doc/build/dialects/index.rst b/doc/build/dialects/index.rst index 06b41326402..9a4badee238 100644 --- a/doc/build/dialects/index.rst +++ b/doc/build/dialects/index.rst @@ -19,18 +19,7 @@ Included Dialects :glob: postgresql - mysql - sqlite - oracle - mssql -Support Levels for Included Dialects -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The following table summarizes the support level for each included dialect. - -.. dialect-table:: **Supported database versions for included dialects** - :header-rows: 1 Support Definitions ^^^^^^^^^^^^^^^^^^^ diff --git a/doc/build/dialects/mssql.rst b/doc/build/dialects/mssql.rst deleted file mode 100644 index 3aff1008738..00000000000 --- a/doc/build/dialects/mssql.rst +++ /dev/null @@ -1,163 +0,0 @@ -.. _mssql_toplevel: - -Microsoft SQL Server -==================== - -.. automodule:: sqlalchemy.dialects.mssql.base - -SQL Server SQL Constructs -------------------------- - -.. currentmodule:: sqlalchemy.dialects.mssql - -.. autofunction:: try_cast - -SQL Server Data Types ---------------------- - -As with all SQLAlchemy dialects, all UPPERCASE types that are known to be -valid with SQL server are importable from the top level dialect, whether -they originate from :mod:`sqlalchemy.types` or from the local dialect:: - - from sqlalchemy.dialects.mssql import ( - BIGINT, - BINARY, - BIT, - CHAR, - DATE, - DATETIME, - DATETIME2, - DATETIMEOFFSET, - DECIMAL, - FLOAT, - IMAGE, - INTEGER, - JSON, - MONEY, - NCHAR, - NTEXT, - NUMERIC, - NVARCHAR, - REAL, - SMALLDATETIME, - SMALLINT, - SMALLMONEY, - SQL_VARIANT, - TEXT, - TIME, - TIMESTAMP, - TINYINT, - UNIQUEIDENTIFIER, - VARBINARY, - VARCHAR, - ) - -Types which are specific to SQL Server, or have SQL Server-specific -construction arguments, are as follows: - -.. note: where :noindex: is used, indicates a type that is not redefined - in the dialect module, just imported from sqltypes. this avoids warnings - in the sphinx build - -.. currentmodule:: sqlalchemy.dialects.mssql - -.. autoclass:: BIT - :members: __init__ - - -.. autoclass:: CHAR - :members: __init__ - :noindex: - - -.. autoclass:: DATETIME2 - :members: __init__ - - -.. autoclass:: DATETIMEOFFSET - :members: __init__ - - -.. autoclass:: IMAGE - :members: __init__ - - -.. autoclass:: JSON - :members: __init__ - - -.. autoclass:: MONEY - :members: __init__ - - -.. autoclass:: NCHAR - :members: __init__ - :noindex: - - -.. autoclass:: NTEXT - :members: __init__ - - -.. autoclass:: NVARCHAR - :members: __init__ - :noindex: - -.. autoclass:: REAL - :members: __init__ - -.. autoclass:: ROWVERSION - :members: __init__ - -.. autoclass:: SMALLDATETIME - :members: __init__ - - -.. autoclass:: SMALLMONEY - :members: __init__ - - -.. autoclass:: SQL_VARIANT - :members: __init__ - - -.. autoclass:: TEXT - :members: __init__ - :noindex: - -.. autoclass:: TIME - :members: __init__ - - -.. autoclass:: TIMESTAMP - :members: __init__ - -.. autoclass:: TINYINT - :members: __init__ - - -.. autoclass:: UNIQUEIDENTIFIER - :members: __init__ - - -.. autoclass:: VARBINARY - :members: __init__ - :noindex: - -.. autoclass:: VARCHAR - :members: __init__ - :noindex: - - -.. autoclass:: XML - :members: __init__ - - -PyODBC ------- -.. automodule:: sqlalchemy.dialects.mssql.pyodbc - -pymssql -------- -.. automodule:: sqlalchemy.dialects.mssql.pymssql - diff --git a/doc/build/dialects/mysql.rst b/doc/build/dialects/mysql.rst deleted file mode 100644 index a46bf721e21..00000000000 --- a/doc/build/dialects/mysql.rst +++ /dev/null @@ -1,260 +0,0 @@ -.. _mysql_toplevel: - -MySQL and MariaDB -================= - -.. automodule:: sqlalchemy.dialects.mysql.base - -MySQL SQL Constructs --------------------- - -.. currentmodule:: sqlalchemy.dialects.mysql - -.. autoclass:: match - :members: - -MySQL Data Types ----------------- - -As with all SQLAlchemy dialects, all UPPERCASE types that are known to be -valid with MySQL are importable from the top level dialect:: - - from sqlalchemy.dialects.mysql import ( - BIGINT, - BINARY, - BIT, - BLOB, - BOOLEAN, - CHAR, - DATE, - DATETIME, - DECIMAL, - DECIMAL, - DOUBLE, - ENUM, - FLOAT, - INTEGER, - LONGBLOB, - LONGTEXT, - MEDIUMBLOB, - MEDIUMINT, - MEDIUMTEXT, - NCHAR, - NUMERIC, - NVARCHAR, - REAL, - SET, - SMALLINT, - TEXT, - TIME, - TIMESTAMP, - TINYBLOB, - TINYINT, - TINYTEXT, - VARBINARY, - VARCHAR, - YEAR, - ) - -Types which are specific to MySQL, or have MySQL-specific -construction arguments, are as follows: - -.. note: where :noindex: is used, indicates a type that is not redefined - in the dialect module, just imported from sqltypes. this avoids warnings - in the sphinx build - -.. currentmodule:: sqlalchemy.dialects.mysql - -.. autoclass:: BIGINT - :members: __init__ - - -.. autoclass:: BINARY - :noindex: - :members: __init__ - - -.. autoclass:: BIT - :members: __init__ - - -.. autoclass:: BLOB - :members: __init__ - :noindex: - - -.. autoclass:: BOOLEAN - :members: __init__ - :noindex: - - -.. autoclass:: CHAR - :members: __init__ - - -.. autoclass:: DATE - :members: __init__ - :noindex: - - -.. autoclass:: DATETIME - :members: __init__ - - -.. autoclass:: DECIMAL - :members: __init__ - - -.. autoclass:: DOUBLE - :members: __init__ - :noindex: - -.. autoclass:: ENUM - :members: __init__ - - -.. autoclass:: FLOAT - :members: __init__ - - -.. autoclass:: INTEGER - :members: __init__ - -.. autoclass:: JSON - :members: - -.. autoclass:: LONGBLOB - :members: __init__ - - -.. autoclass:: LONGTEXT - :members: __init__ - - -.. autoclass:: MEDIUMBLOB - :members: __init__ - - -.. autoclass:: MEDIUMINT - :members: __init__ - - -.. autoclass:: MEDIUMTEXT - :members: __init__ - - -.. autoclass:: NCHAR - :members: __init__ - - -.. autoclass:: NUMERIC - :members: __init__ - - -.. autoclass:: NVARCHAR - :members: __init__ - - -.. autoclass:: REAL - :members: __init__ - - -.. autoclass:: SET - :members: __init__ - - -.. autoclass:: SMALLINT - :members: __init__ - - -.. autoclass:: TEXT - :members: __init__ - :noindex: - - -.. autoclass:: TIME - :members: __init__ - - -.. autoclass:: TIMESTAMP - :members: __init__ - - -.. autoclass:: TINYBLOB - :members: __init__ - - -.. autoclass:: TINYINT - :members: __init__ - - -.. autoclass:: TINYTEXT - :members: __init__ - - -.. autoclass:: VARBINARY - :members: __init__ - :noindex: - - -.. autoclass:: VARCHAR - :members: __init__ - - -.. autoclass:: YEAR - :members: __init__ - -MySQL DML Constructs -------------------------- - -.. autofunction:: sqlalchemy.dialects.mysql.insert - -.. autoclass:: sqlalchemy.dialects.mysql.Insert - :members: - - - -mysqlclient (fork of MySQL-Python) ----------------------------------- - -.. automodule:: sqlalchemy.dialects.mysql.mysqldb - -PyMySQL -------- - -.. automodule:: sqlalchemy.dialects.mysql.pymysql - -MariaDB-Connector ------------------- - -.. automodule:: sqlalchemy.dialects.mysql.mariadbconnector - -MySQL-Connector ---------------- - -.. automodule:: sqlalchemy.dialects.mysql.mysqlconnector - -.. _asyncmy: - -asyncmy -------- - -.. automodule:: sqlalchemy.dialects.mysql.asyncmy - - -.. _aiomysql: - -aiomysql --------- - -.. automodule:: sqlalchemy.dialects.mysql.aiomysql - -cymysql -------- - -.. automodule:: sqlalchemy.dialects.mysql.cymysql - -pyodbc ------- - -.. automodule:: sqlalchemy.dialects.mysql.pyodbc diff --git a/doc/build/dialects/oracle.rst b/doc/build/dialects/oracle.rst deleted file mode 100644 index 02f51221416..00000000000 --- a/doc/build/dialects/oracle.rst +++ /dev/null @@ -1,84 +0,0 @@ -.. _oracle_toplevel: - -Oracle -====== - -.. automodule:: sqlalchemy.dialects.oracle.base - -Oracle Data Types ------------------ - -As with all SQLAlchemy dialects, all UPPERCASE types that are known to be -valid with Oracle are importable from the top level dialect, whether -they originate from :mod:`sqlalchemy.types` or from the local dialect:: - - from sqlalchemy.dialects.oracle import ( - BFILE, - BLOB, - CHAR, - CLOB, - DATE, - DOUBLE_PRECISION, - FLOAT, - INTERVAL, - LONG, - NCLOB, - NCHAR, - NUMBER, - NVARCHAR, - NVARCHAR2, - RAW, - TIMESTAMP, - VARCHAR, - VARCHAR2, - ) - -.. versionadded:: 1.2.19 Added :class:`_types.NCHAR` to the list of datatypes - exported by the Oracle dialect. - -Types which are specific to Oracle, or have Oracle-specific -construction arguments, are as follows: - -.. currentmodule:: sqlalchemy.dialects.oracle - -.. autoclass:: BFILE - :members: __init__ - -.. autoclass:: DATE - :members: __init__ - -.. autoclass:: FLOAT - :members: __init__ - -.. autoclass:: INTERVAL - :members: __init__ - -.. autoclass:: NCLOB - :members: __init__ - -.. autoclass:: NUMBER - :members: __init__ - -.. autoclass:: LONG - :members: __init__ - -.. autoclass:: RAW - :members: __init__ - -.. autoclass:: TIMESTAMP - :members: __init__ - -.. _cx_oracle: - -cx_Oracle ---------- - -.. automodule:: sqlalchemy.dialects.oracle.cx_oracle - -.. _oracledb: - -python-oracledb ---------------- - -.. automodule:: sqlalchemy.dialects.oracle.oracledb - diff --git a/doc/build/dialects/postgresql.rst b/doc/build/dialects/postgresql.rst index fce0e4610e8..c12cc83735a 100644 --- a/doc/build/dialects/postgresql.rst +++ b/doc/build/dialects/postgresql.rst @@ -3,7 +3,9 @@ PostgreSQL ========== -.. automodule:: sqlalchemy.dialects.postgresql.base +The PG module + +.. .. automodule:: sqlalchemy.dialects.postgresql.base ARRAY Types ----------- @@ -283,7 +285,7 @@ Illustrating insertion and selecting of a record:: as handled by the ORM will not automatically detect in-place changes to a particular list value; to update list values with the ORM, either re-assign a new list to the attribute, or use the :class:`.MutableList` - type modifier. See the section :ref:`mutable_toplevel` for background. + type modifier. See the section ref_mutable_toplevel for background. The available multirange datatypes are as follows: @@ -533,31 +535,3 @@ PostgreSQL DML Constructs .. _postgresql_psycopg2: -psycopg2 --------- - -.. automodule:: sqlalchemy.dialects.postgresql.psycopg2 - -.. _postgresql_psycopg: - -psycopg --------- - -.. automodule:: sqlalchemy.dialects.postgresql.psycopg - -pg8000 ------- - -.. automodule:: sqlalchemy.dialects.postgresql.pg8000 - -.. _dialect-postgresql-asyncpg: - -asyncpg -------- - -.. automodule:: sqlalchemy.dialects.postgresql.asyncpg - -psycopg2cffi ------------- - -.. automodule:: sqlalchemy.dialects.postgresql.psycopg2cffi diff --git a/doc/build/dialects/sqlite.rst b/doc/build/dialects/sqlite.rst deleted file mode 100644 index d25301fa53f..00000000000 --- a/doc/build/dialects/sqlite.rst +++ /dev/null @@ -1,71 +0,0 @@ -.. _sqlite_toplevel: - -SQLite -====== - -.. automodule:: sqlalchemy.dialects.sqlite.base - -SQLite Data Types ------------------ - -As with all SQLAlchemy dialects, all UPPERCASE types that are known to be -valid with SQLite are importable from the top level dialect, whether -they originate from :mod:`sqlalchemy.types` or from the local dialect:: - - from sqlalchemy.dialects.sqlite import ( - BLOB, - BOOLEAN, - CHAR, - DATE, - DATETIME, - DECIMAL, - FLOAT, - INTEGER, - NUMERIC, - JSON, - SMALLINT, - TEXT, - TIME, - TIMESTAMP, - VARCHAR, - ) - -.. module:: sqlalchemy.dialects.sqlite - -.. autoclass:: DATETIME - -.. autoclass:: DATE - -.. autoclass:: JSON - -.. autoclass:: TIME - -SQLite DML Constructs -------------------------- - -.. autofunction:: sqlalchemy.dialects.sqlite.insert - -.. autoclass:: sqlalchemy.dialects.sqlite.Insert - :members: - -.. _pysqlite: - -Pysqlite --------- - -.. automodule:: sqlalchemy.dialects.sqlite.pysqlite - -.. _aiosqlite: - -Aiosqlite ---------- - -.. automodule:: sqlalchemy.dialects.sqlite.aiosqlite - - -.. _pysqlcipher: - -Pysqlcipher ------------ - -.. automodule:: sqlalchemy.dialects.sqlite.pysqlcipher diff --git a/doc/build/errors.rst b/doc/build/errors.rst index ea76a991408..cee35f3265d 100644 --- a/doc/build/errors.rst +++ b/doc/build/errors.rst @@ -907,7 +907,7 @@ Mitigation of this error is via these techniques: :ref:`session_committing` - background on session commit - :ref:`session_expire` - background on attribute expiry + ref_session_expire - background on attribute expiry .. _error_7s2a: @@ -1327,7 +1327,7 @@ annotations within class definitions at runtime. A requirement of this form is that all ORM annotations must make use of a generic container called :class:`_orm.Mapped` to be properly annotated. Legacy SQLAlchemy mappings which include explicit :pep:`484` typing annotations, such as those which use the -:ref:`legacy Mypy extension ` for typing support, may include +ref_mypy_toplevel for typing support, may include directives such as those for :func:`_orm.relationship` that don't include this generic. @@ -1346,7 +1346,7 @@ When transforming to a dataclass, attribute(s) originate from superclass < ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This warning occurs when using the SQLAlchemy ORM Mapped Dataclasses feature -described at :ref:`orm_declarative_native_dataclasses` in conjunction with +described at ref_orm_declarative_native_dataclasses in conjunction with any mixin class or abstract base that is not itself declared as a dataclass, such as in the example below:: @@ -1463,7 +1463,7 @@ attention to the rules applied to `inheritance `_. .. seealso:: - :ref:`orm_declarative_native_dataclasses` - SQLAlchemy dataclasses documentation + ref_orm_declarative_native_dataclasses - SQLAlchemy dataclasses documentation `Python dataclasses `_ - on the python.org website @@ -1491,7 +1491,7 @@ with a non compatible :term:`DBAPI`. .. seealso:: - :ref:`asyncio_toplevel` + ref_asyncio_toplevel .. _error_xd2s: @@ -1506,7 +1506,7 @@ attempt, which is unsupported when using SQLAlchemy with AsyncIO dialects. .. seealso:: - :ref:`asyncio_orm_avoid_lazyloads` - covers most ORM scenarios where + ref_asyncio_orm_avoid_lazyloads - covers most ORM scenarios where this problem can occur and how to mitigate. .. _error_xd3s: @@ -1534,7 +1534,7 @@ that performs the desired operations:: .. seealso:: - :ref:`asyncio_inspector` - additional examples of using :func:`_sa.inspect` + ref_asyncio_inspector - additional examples of using :func:`_sa.inspect` with the asyncio extension. diff --git a/doc/build/faq/connections.rst b/doc/build/faq/connections.rst index b0c354c00c3..1af2b583cf3 100644 --- a/doc/build/faq/connections.rst +++ b/doc/build/faq/connections.rst @@ -402,7 +402,7 @@ current SQLAlchemy versions. .. seealso:: - :ref:`pysqlite_threading_pooling` - info on PySQLite's behavior. + ref_pysqlite_threading_pooling - info on PySQLite's behavior. .. _faq_dbapi_connection: diff --git a/doc/build/faq/installation.rst b/doc/build/faq/installation.rst index 72b4fc15915..7e77a3eb0f8 100644 --- a/doc/build/faq/installation.rst +++ b/doc/build/faq/installation.rst @@ -21,11 +21,11 @@ to the ``pip install`` command: pip install sqlalchemy[asyncio] -For more background, see :ref:`asyncio_install`. +For more background, see ref_asyncio_install. .. seealso:: - :ref:`asyncio_install` + ref_asyncio_install diff --git a/doc/build/faq/performance.rst b/doc/build/faq/performance.rst index aa8d4e314f1..ad0195cd38d 100644 --- a/doc/build/faq/performance.rst +++ b/doc/build/faq/performance.rst @@ -439,7 +439,7 @@ Common strategies to mitigate this include: for user, address in session.execute(select(u_b, a_b).join(User.addresses)): ... -* Use result caching - see :ref:`examples_caching` for an in-depth example +* Use result caching - see ref_examples_caching for an in-depth example of this. * Consider a faster interpreter like that of PyPy. @@ -449,7 +449,7 @@ practice they are very easy to read. .. seealso:: - :ref:`examples_performance` - a suite of performance demonstrations + ref_examples_performance - a suite of performance demonstrations with bundled profiling capabilities. I'm inserting 400,000 rows with the ORM and it's really slow! diff --git a/doc/build/faq/sessions.rst b/doc/build/faq/sessions.rst index a2c61c0a41d..80d587aee8e 100644 --- a/doc/build/faq/sessions.rst +++ b/doc/build/faq/sessions.rst @@ -64,7 +64,7 @@ Three ways, from most common to least: 2. We tell our :class:`.Session` to re-read rows that it has already read, either when we next query for them using :meth:`.Session.expire_all` or :meth:`.Session.expire`, or immediately on an object using - :class:`.Session.refresh`. See :ref:`session_expire` for detail on this. + :class:`.Session.refresh`. See ref_session_expire for detail on this. 3. We can run whole queries while setting them to definitely overwrite already-loaded objects as they read rows by using "populate existing". diff --git a/doc/build/glossary.rst b/doc/build/glossary.rst index 21c8a6ebfb6..c7db2439d93 100644 --- a/doc/build/glossary.rst +++ b/doc/build/glossary.rst @@ -88,7 +88,7 @@ Glossary .. seealso:: - :ref:`metadata_reflection_toplevel` - complete background on + ref_metadata_reflection_toplevel - complete background on database reflection. :ref:`orm_declarative_reflected` - background on integrating @@ -461,9 +461,9 @@ Glossary .. seealso:: - :ref:`mapper_version_counter` - SQLAlchemy's built-in version id feature. + ref_mapper_version_counter - SQLAlchemy's built-in version id feature. - :ref:`examples_versioning` - other examples of mappings that version rows + ref_examples_versioning - other examples of mappings that version rows temporally. registry @@ -613,8 +613,8 @@ Glossary :doc:`orm/queryguide/relationships` - includes information on lazy loading of ORM related objects - :ref:`asyncio_orm_avoid_lazyloads` - tips on avoiding lazy loading - when using the :ref:`asyncio_toplevel` extension + ref_asyncio_orm_avoid_lazyloads - tips on avoiding lazy loading + when using the ref_asyncio_toplevel extension eager load eager loads @@ -824,7 +824,7 @@ Glossary .. seealso:: - :ref:`session_expire` + ref_session_expire Session The container or scope for ORM database operations. Sessions @@ -1623,7 +1623,7 @@ Glossary .. seealso:: - :ref:`session_object_states` + ref_session_object_states pending This describes one of the major object states which @@ -1635,7 +1635,7 @@ Glossary .. seealso:: - :ref:`session_object_states` + ref_session_object_states deleted This describes one of the major object states which @@ -1650,7 +1650,7 @@ Glossary .. seealso:: - :ref:`session_object_states` + ref_session_object_states persistent This describes one of the major object states which @@ -1665,7 +1665,7 @@ Glossary .. seealso:: - :ref:`session_object_states` + ref_session_object_states detached This describes one of the major object states which @@ -1681,7 +1681,7 @@ Glossary .. seealso:: - :ref:`session_object_states` + ref_session_object_states attached Indicates an ORM object that is presently associated with a specific @@ -1689,5 +1689,5 @@ Glossary .. seealso:: - :ref:`session_object_states` + ref_session_object_states diff --git a/doc/build/index.rst b/doc/build/index.rst index 37b807723f3..fa7a609b45d 100644 --- a/doc/build/index.rst +++ b/doc/build/index.rst @@ -80,20 +80,15 @@ SQLAlchemy Documentation * **Using the ORM:** :doc:`Using the ORM Session ` | :doc:`ORM Querying Guide ` | - :doc:`Using AsyncIO ` * **Configuration Extensions:** - :doc:`Association Proxy ` | - :doc:`Hybrid Attributes ` | - :doc:`Mutable Scalars ` | - :doc:`Automap ` | - :doc:`All extensions ` + none... * **Extending the ORM:** :doc:`ORM Events and Internals ` * **Other:** - :doc:`Introduction to Examples ` + none .. container:: core @@ -102,14 +97,13 @@ SQLAlchemy Documentation * **Engines, Connections, Pools:** :doc:`Engine Configuration ` | :doc:`Connections, Transactions, Results ` | - :doc:`AsyncIO Support ` | :doc:`Connection Pooling ` * **Schema Definition:** :doc:`Overview ` | :ref:`Tables and Columns ` | - :ref:`Database Introspection (Reflection) ` | - :ref:`Insert/Update Defaults ` | + ref_metadata_reflection_toplevel | + ref_metadata_defaults_toplevel | :ref:`Constraints and Indexes ` | :ref:`Using Data Definition Language (DDL) ` @@ -118,7 +112,6 @@ SQLAlchemy Documentation :doc:`Operator Reference ` | :doc:`SELECT and related constructs ` | :doc:`INSERT, UPDATE, DELETE ` | - :doc:`SQL Functions ` | :doc:`Table of Contents ` @@ -130,7 +123,6 @@ SQLAlchemy Documentation * **Core Basics:** :doc:`Overview ` | - :doc:`Runtime Inspection API ` | :doc:`Event System ` | :doc:`Core Event Interfaces ` | :doc:`Creating Custom SQL Constructs ` @@ -149,10 +141,6 @@ SQLAlchemy Documentation This section describes notes, options, and usage patterns regarding individual dialects. :doc:`PostgreSQL ` | - :doc:`MySQL ` | - :doc:`SQLite ` | - :doc:`Oracle ` | - :doc:`Microsoft SQL Server ` :doc:`More Dialects ... ` diff --git a/doc/build/intro.rst b/doc/build/intro.rst index cac103ed831..ef469ca6811 100644 --- a/doc/build/intro.rst +++ b/doc/build/intro.rst @@ -80,7 +80,7 @@ Code Examples Working code examples, mostly regarding the ORM, are included in the SQLAlchemy distribution. A description of all the included example -applications is at :ref:`examples_toplevel`. +applications is at ref_examples_toplevel. There is also a wide variety of examples involving both core SQLAlchemy constructs as well as the ORM on the wiki. See @@ -109,7 +109,7 @@ SQLAlchemy's ``asyncio`` support depends upon the `greenlet `_ project. This dependency will be installed by default on common machine platforms, however is not supported on every architecture and also may not install by default on -less common architectures. See the section :ref:`asyncio_install` for +less common architectures. See the section ref_asyncio_install for additional details on ensuring asyncio support is present. Supported Installation Methods diff --git a/doc/build/orm/backref.rst b/doc/build/orm/backref.rst deleted file mode 100644 index 01f4c90736d..00000000000 --- a/doc/build/orm/backref.rst +++ /dev/null @@ -1,190 +0,0 @@ -.. _relationships_backref: - -Using the legacy 'backref' relationship parameter --------------------------------------------------- - -.. note:: The :paramref:`_orm.relationship.backref` keyword should be considered - legacy, and use of :paramref:`_orm.relationship.back_populates` with explicit - :func:`_orm.relationship` constructs should be preferred. Using - individual :func:`_orm.relationship` constructs provides advantages - including that both ORM mapped classes will include their attributes - up front as the class is constructed, rather than as a deferred step, - and configuration is more straightforward as all arguments are explicit. - New :pep:`484` features in SQLAlchemy 2.0 also take advantage of - attributes being explicitly present in source code rather than - using dynamic attribute generation. - -.. seealso:: - - For general information about bidirectional relationships, see the - following sections: - - :ref:`tutorial_orm_related_objects` - in the :ref:`unified_tutorial`, - presents an overview of bi-directional relationship configuration - and behaviors using :paramref:`_orm.relationship.back_populates` - - :ref:`back_populates_cascade` - notes on bi-directional :func:`_orm.relationship` - behavior regarding :class:`_orm.Session` cascade behaviors. - - :paramref:`_orm.relationship.back_populates` - - -The :paramref:`_orm.relationship.backref` keyword argument on the -:func:`_orm.relationship` construct allows the -automatic generation of a new :func:`_orm.relationship` that will be automatically -be added to the ORM mapping for the related class. It will then be -placed into a :paramref:`_orm.relationship.back_populates` configuration -against the current :func:`_orm.relationship` being configured, with both -:func:`_orm.relationship` constructs referring to each other. - -Starting with the following example:: - - from sqlalchemy import Column, ForeignKey, Integer, String - from sqlalchemy.orm import DeclarativeBase, relationship - - - class Base(DeclarativeBase): - pass - - - class User(Base): - __tablename__ = "user" - id = mapped_column(Integer, primary_key=True) - name = mapped_column(String) - - addresses = relationship("Address", backref="user") - - - class Address(Base): - __tablename__ = "address" - id = mapped_column(Integer, primary_key=True) - email = mapped_column(String) - user_id = mapped_column(Integer, ForeignKey("user.id")) - -The above configuration establishes a collection of ``Address`` objects on ``User`` called -``User.addresses``. It also establishes a ``.user`` attribute on ``Address`` which will -refer to the parent ``User`` object. Using :paramref:`_orm.relationship.back_populates` -it's equivalent to the following:: - - from sqlalchemy import Column, ForeignKey, Integer, String - from sqlalchemy.orm import DeclarativeBase, relationship - - - class Base(DeclarativeBase): - pass - - - class User(Base): - __tablename__ = "user" - id = mapped_column(Integer, primary_key=True) - name = mapped_column(String) - - addresses = relationship("Address", back_populates="user") - - - class Address(Base): - __tablename__ = "address" - id = mapped_column(Integer, primary_key=True) - email = mapped_column(String) - user_id = mapped_column(Integer, ForeignKey("user.id")) - - user = relationship("User", back_populates="addresses") - -The behavior of the ``User.addresses`` and ``Address.user`` relationships -is that they now behave in a **bi-directional** way, indicating that -changes on one side of the relationship impact the other. An example -and discussion of this behavior is in the :ref:`unified_tutorial` -at :ref:`tutorial_orm_related_objects`. - - -Backref Default Arguments -~~~~~~~~~~~~~~~~~~~~~~~~~ - -Since :paramref:`_orm.relationship.backref` generates a whole new -:func:`_orm.relationship`, the generation process by default -will attempt to include corresponding arguments in the new -:func:`_orm.relationship` that correspond to the original arguments. -As an example, below is a :func:`_orm.relationship` that includes a -:ref:`custom join condition ` -which also includes the :paramref:`_orm.relationship.backref` keyword:: - - from sqlalchemy import Column, ForeignKey, Integer, String - from sqlalchemy.orm import DeclarativeBase, relationship - - - class Base(DeclarativeBase): - pass - - - class User(Base): - __tablename__ = "user" - id = mapped_column(Integer, primary_key=True) - name = mapped_column(String) - - addresses = relationship( - "Address", - primaryjoin=( - "and_(User.id==Address.user_id, Address.email.startswith('tony'))" - ), - backref="user", - ) - - - class Address(Base): - __tablename__ = "address" - id = mapped_column(Integer, primary_key=True) - email = mapped_column(String) - user_id = mapped_column(Integer, ForeignKey("user.id")) - -When the "backref" is generated, the :paramref:`_orm.relationship.primaryjoin` -condition is copied to the new :func:`_orm.relationship` as well:: - - >>> print(User.addresses.property.primaryjoin) - "user".id = address.user_id AND address.email LIKE :email_1 || '%%' - >>> - >>> print(Address.user.property.primaryjoin) - "user".id = address.user_id AND address.email LIKE :email_1 || '%%' - >>> - -Other arguments that are transferrable include the -:paramref:`_orm.relationship.secondary` parameter that refers to a -many-to-many association table, as well as the "join" arguments -:paramref:`_orm.relationship.primaryjoin` and -:paramref:`_orm.relationship.secondaryjoin`; "backref" is smart enough to know -that these two arguments should also be "reversed" when generating -the opposite side. - -Specifying Backref Arguments -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Lots of other arguments for a "backref" are not implicit, and -include arguments like -:paramref:`_orm.relationship.lazy`, -:paramref:`_orm.relationship.remote_side`, -:paramref:`_orm.relationship.cascade` and -:paramref:`_orm.relationship.cascade_backrefs`. For this case we use -the :func:`.backref` function in place of a string; this will store -a specific set of arguments that will be transferred to the new -:func:`_orm.relationship` when generated:: - - # - from sqlalchemy.orm import backref - - - class User(Base): - __tablename__ = "user" - id = mapped_column(Integer, primary_key=True) - name = mapped_column(String) - - addresses = relationship( - "Address", - backref=backref("user", lazy="joined"), - ) - -Where above, we placed a ``lazy="joined"`` directive only on the ``Address.user`` -side, indicating that when a query against ``Address`` is made, a join to the ``User`` -entity should be made automatically which will populate the ``.user`` attribute of each -returned ``Address``. The :func:`.backref` function formatted the arguments we gave -it into a form that is interpreted by the receiving :func:`_orm.relationship` as additional -arguments to be applied to the new relationship it creates. - diff --git a/doc/build/orm/basic_relationships.rst b/doc/build/orm/basic_relationships.rst index 91353681ed2..6be7f8bfe11 100644 --- a/doc/build/orm/basic_relationships.rst +++ b/doc/build/orm/basic_relationships.rst @@ -716,14 +716,14 @@ association object:: To enhance the association object pattern such that direct access to the ``Association`` object is optional, SQLAlchemy -provides the :ref:`associationproxy_toplevel` extension. This +provides the ref_associationproxy_toplevel extension. This extension allows the configuration of attributes which will access two "hops" with a single access, one "hop" to the associated object, and a second to a target attribute. .. seealso:: - :ref:`associationproxy_toplevel` - allows direct "many to many" style + ref_associationproxy_toplevel - allows direct "many to many" style access between parent and child for a three-class association object mapping. .. warning:: @@ -731,7 +731,7 @@ associated object, and a second to a target attribute. Avoid mixing the association object pattern with the :ref:`many-to-many ` pattern directly, as this produces conditions where data may be read and written in an inconsistent fashion without special steps; - the :ref:`association proxy ` is typically + the ref_associationproxy_toplevel is typically used to provide more succinct access. For more detailed background on the caveats introduced by this combination, see the next section :ref:`association_pattern_w_m2m`. @@ -899,11 +899,11 @@ A popular alternative to the above pattern is one where the direct many-to-many ``Parent.children`` and ``Child.parents`` relationships are replaced with an extension that will transparently proxy through the ``Association`` class, while keeping everything consistent from the ORM's point of -view. This extension is known as the :ref:`Association Proxy `. +view. This extension is known as the ref_associationproxy_toplevel. .. seealso:: - :ref:`associationproxy_toplevel` - allows direct "many to many" style + ref_associationproxy_toplevel - allows direct "many to many" style access between parent and child for a three-class association object mapping. .. _orm_declarative_relationship_eval: diff --git a/doc/build/orm/cascades.rst b/doc/build/orm/cascades.rst index 02d68669eee..458dd5f7b6a 100644 --- a/doc/build/orm/cascades.rst +++ b/doc/build/orm/cascades.rst @@ -60,9 +60,9 @@ it is no longer associated with that parent. .. warning:: The ``all`` cascade option implies the :ref:`cascade_refresh_expire` cascade setting which may not be desirable when using the - :ref:`asyncio_toplevel` extension, as it will expire related objects + ref_asyncio_toplevel extension, as it will expire related objects more aggressively than is typically appropriate in an explicit IO context. - See the notes at :ref:`asyncio_orm_avoid_lazyloads` for further background. + See the notes at ref_asyncio_orm_avoid_lazyloads for further background. The list of available values which can be specified for the :paramref:`_orm.relationship.cascade` parameter are described in the following subsections. @@ -457,10 +457,10 @@ is as follows: support ``FOREIGN KEY`` constraints and they must be enforcing: * When using MySQL, an appropriate storage engine must be - selected. See :ref:`mysql_storage_engines` for details. + selected. See ref_mysql_storage_engines for details. * When using SQLite, foreign key support must be enabled explicitly. - See :ref:`sqlite_foreign_keys` for details. + See ref_sqlite_foreign_keys for details. .. topic:: Notes on Passive Deletes diff --git a/doc/build/orm/classical.rst b/doc/build/orm/classical.rst deleted file mode 100644 index a0bc70d890a..00000000000 --- a/doc/build/orm/classical.rst +++ /dev/null @@ -1,5 +0,0 @@ -:orphan: - -Moved! :ref:`orm_imperative_mapping` - - diff --git a/doc/build/orm/collection_api.rst b/doc/build/orm/collection_api.rst index 2d56bb9b2b0..3674dc81b48 100644 --- a/doc/build/orm/collection_api.rst +++ b/doc/build/orm/collection_api.rst @@ -293,7 +293,7 @@ with a ``@property`` as mentioned earlier:: ) Dictionary mappings are often combined with the "Association Proxy" extension to produce -streamlined dictionary views. See :ref:`proxying_dictionaries` and :ref:`composite_association_proxy` +streamlined dictionary views. See ref_proxying_dictionaries and ref_composite_association_proxy for examples. .. _key_collections_mutations: @@ -400,7 +400,7 @@ about how the collection operates. of validation and simple marshaling. See :ref:`simple_validators` for an example of this. - For the second use case, the :ref:`associationproxy_toplevel` extension is a + For the second use case, the ref_associationproxy_toplevel extension is a well-tested, widely used system that provides a read/write "view" of a collection in terms of some attribute present on the target object. As the target attribute can be a ``@property`` that returns virtually anything, a diff --git a/doc/build/orm/collections.rst b/doc/build/orm/collections.rst deleted file mode 100644 index 43cbef23a9e..00000000000 --- a/doc/build/orm/collections.rst +++ /dev/null @@ -1,12 +0,0 @@ -:orphan: - -======================================= -Collection Configuration and Techniques -======================================= - -This page has been broken into two separate pages: - -:doc:`large_collections` - -:doc:`collection_api` - diff --git a/doc/build/orm/composites.rst b/doc/build/orm/composites.rst deleted file mode 100644 index 4eaf7024a93..00000000000 --- a/doc/build/orm/composites.rst +++ /dev/null @@ -1,469 +0,0 @@ -.. currentmodule:: sqlalchemy.orm - -.. _mapper_composite: - -Composite Column Types -====================== - -Sets of columns can be associated with a single user-defined datatype, -which in modern use is normally a Python dataclass_. The ORM -provides a single attribute which represents the group of columns using the -class you provide. - -A simple example represents pairs of :class:`_types.Integer` columns as a -``Point`` object, with attributes ``.x`` and ``.y``. Using a -dataclass, these attributes are defined with the corresponding ``int`` -Python type:: - - import dataclasses - - - @dataclasses.dataclass - class Point: - x: int - y: int - -Non-dataclass forms are also accepted, but require additional methods -to be implemented. For an example using a non-dataclass class, see the section -:ref:`composite_legacy_no_dataclass`. - -.. versionadded:: 2.0 The :func:`_orm.composite` construct fully supports - Python dataclasses including the ability to derive mapped column datatypes - from the composite class. - -We will create a mapping to a table ``vertices``, which represents two points -as ``x1/y1`` and ``x2/y2``. The ``Point`` class is associated with -the mapped columns using the :func:`_orm.composite` construct. - -The example below illustrates the most modern form of :func:`_orm.composite` as -used with a fully -:ref:`Annotated Declarative Table ` -configuration. :func:`_orm.mapped_column` constructs representing each column -are passed directly to :func:`_orm.composite`, indicating zero or more aspects -of the columns to be generated, in this case the names; the -:func:`_orm.composite` construct derives the column types (in this case -``int``, corresponding to :class:`_types.Integer`) from the dataclass directly:: - - from sqlalchemy.orm import DeclarativeBase, Mapped - from sqlalchemy.orm import composite, mapped_column - - - class Base(DeclarativeBase): - pass - - - class Vertex(Base): - __tablename__ = "vertices" - - id: Mapped[int] = mapped_column(primary_key=True) - - start: Mapped[Point] = composite(mapped_column("x1"), mapped_column("y1")) - end: Mapped[Point] = composite(mapped_column("x2"), mapped_column("y2")) - - def __repr__(self): - return f"Vertex(start={self.start}, end={self.end})" - -The above mapping would correspond to a CREATE TABLE statement as: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy.schema import CreateTable - >>> print(CreateTable(Vertex.__table__)) - {printsql}CREATE TABLE vertices ( - id INTEGER NOT NULL, - x1 INTEGER NOT NULL, - y1 INTEGER NOT NULL, - x2 INTEGER NOT NULL, - y2 INTEGER NOT NULL, - PRIMARY KEY (id) - ) - - -Working with Mapped Composite Column Types -------------------------------------------- - -With a mapping as illustrated in the top section, we can work with the -``Vertex`` class, where the ``.start`` and ``.end`` attributes will -transparently refer to the columns referred towards by the ``Point`` class, as -well as with instances of the ``Vertex`` class, where the ``.start`` and -``.end`` attributes will refer to instances of the ``Point`` class. The ``x1``, -``y1``, ``x2``, and ``y2`` columns are handled transparently: - -* **Persisting Point objects** - - We can create a ``Vertex`` object, assign ``Point`` objects as members, - and they will be persisted as expected: - - .. sourcecode:: pycon+sql - - >>> v = Vertex(start=Point(3, 4), end=Point(5, 6)) - >>> session.add(v) - >>> session.commit() - {execsql}BEGIN (implicit) - INSERT INTO vertices (x1, y1, x2, y2) VALUES (?, ?, ?, ?) - [generated in ...] (3, 4, 5, 6) - COMMIT - -* **Selecting Point objects as columns** - - :func:`_orm.composite` will allow the ``Vertex.start`` and ``Vertex.end`` - attributes to behave like a single SQL expression to as much an extent - as possible when using the ORM :class:`_orm.Session` (including the legacy - :class:`_orm.Query` object) to select ``Point`` objects: - - .. sourcecode:: pycon+sql - - >>> stmt = select(Vertex.start, Vertex.end) - >>> session.execute(stmt).all() - {execsql}SELECT vertices.x1, vertices.y1, vertices.x2, vertices.y2 - FROM vertices - [...] () - {stop}[(Point(x=3, y=4), Point(x=5, y=6))] - -* **Comparing Point objects in SQL expressions** - - The ``Vertex.start`` and ``Vertex.end`` attributes may be used in - WHERE criteria and similar, using ad-hoc ``Point`` objects for comparisons: - - .. sourcecode:: pycon+sql - - >>> stmt = select(Vertex).where(Vertex.start == Point(3, 4)).where(Vertex.end < Point(7, 8)) - >>> session.scalars(stmt).all() - {execsql}SELECT vertices.id, vertices.x1, vertices.y1, vertices.x2, vertices.y2 - FROM vertices - WHERE vertices.x1 = ? AND vertices.y1 = ? AND vertices.x2 < ? AND vertices.y2 < ? - [...] (3, 4, 7, 8) - {stop}[Vertex(Point(x=3, y=4), Point(x=5, y=6))] - - .. versionadded:: 2.0 :func:`_orm.composite` constructs now support - "ordering" comparisons such as ``<``, ``>=``, and similar, in addition - to the already-present support for ``==``, ``!=``. - - .. tip:: The "ordering" comparison above using the "less than" operator (``<``) - as well as the "equality" comparison using ``==``, when used to generate - SQL expressions, are implemented by the :class:`_orm.Composite.Comparator` - class, and don't make use of the comparison methods on the composite class - itself, e.g. the ``__lt__()`` or ``__eq__()`` methods. From this it - follows that the ``Point`` dataclass above also need not implement the - dataclasses ``order=True`` parameter for the above SQL operations to work. - The section :ref:`composite_operations` contains background on how - to customize the comparison operations. - -* **Updating Point objects on Vertex Instances** - - By default, the ``Point`` object **must be replaced by a new object** for - changes to be detected: - - .. sourcecode:: pycon+sql - - >>> v1 = session.scalars(select(Vertex)).one() - {execsql}SELECT vertices.id, vertices.x1, vertices.y1, vertices.x2, vertices.y2 - FROM vertices - [...] () - {stop} - - >>> v1.end = Point(x=10, y=14) - >>> session.commit() - {execsql}UPDATE vertices SET x2=?, y2=? WHERE vertices.id = ? - [...] (10, 14, 1) - COMMIT - - In order to allow in place changes on the composite object, the - :ref:`mutable_toplevel` extension must be used. See the section - :ref:`mutable_composites` for examples. - - - -.. _orm_composite_other_forms: - -Other mapping forms for composites ----------------------------------- - -The :func:`_orm.composite` construct may be passed the relevant columns -using a :func:`_orm.mapped_column` construct, a :class:`_schema.Column`, -or the string name of an existing mapped column. The following examples -illustrate an equvalent mapping as that of the main section above. - -* Map columns directly, then pass to composite - - Here we pass the existing :func:`_orm.mapped_column` instances to the - :func:`_orm.composite` construct, as in the non-annotated example below - where we also pass the ``Point`` class as the first argument to - :func:`_orm.composite`:: - - from sqlalchemy import Integer - from sqlalchemy.orm import mapped_column, composite - - - class Vertex(Base): - __tablename__ = "vertices" - - id = mapped_column(Integer, primary_key=True) - x1 = mapped_column(Integer) - y1 = mapped_column(Integer) - x2 = mapped_column(Integer) - y2 = mapped_column(Integer) - - start = composite(Point, x1, y1) - end = composite(Point, x2, y2) - -* Map columns directly, pass attribute names to composite - - We can write the same example above using more annotated forms where we have - the option to pass attribute names to :func:`_orm.composite` instead of - full column constructs:: - - from sqlalchemy.orm import mapped_column, composite, Mapped - - - class Vertex(Base): - __tablename__ = "vertices" - - id: Mapped[int] = mapped_column(primary_key=True) - x1: Mapped[int] - y1: Mapped[int] - x2: Mapped[int] - y2: Mapped[int] - - start: Mapped[Point] = composite("x1", "y1") - end: Mapped[Point] = composite("x2", "y2") - -* Imperative mapping and imperative table - - When using :ref:`imperative table ` or - fully :ref:`imperative ` mappings, we have access - to :class:`_schema.Column` objects directly. These may be passed to - :func:`_orm.composite` as well, as in the imperative example below:: - - mapper_registry.map_imperatively( - Vertex, - vertices_table, - properties={ - "start": composite(Point, vertices_table.c.x1, vertices_table.c.y1), - "end": composite(Point, vertices_table.c.x2, vertices_table.c.y2), - }, - ) - -.. _composite_legacy_no_dataclass: - -Using Legacy Non-Dataclasses ----------------------------- - - -If not using a dataclass, the requirements for the custom datatype class are -that it have a constructor -which accepts positional arguments corresponding to its column format, and -also provides a method ``__composite_values__()`` which returns the state of -the object as a list or tuple, in order of its column-based attributes. It -also should supply adequate ``__eq__()`` and ``__ne__()`` methods which test -the equality of two instances. - -To illustrate the equivalent ``Point`` class from the main section -not using a dataclass:: - - class Point: - def __init__(self, x, y): - self.x = x - self.y = y - - def __composite_values__(self): - return self.x, self.y - - def __repr__(self): - return f"Point(x={self.x!r}, y={self.y!r})" - - def __eq__(self, other): - return isinstance(other, Point) and other.x == self.x and other.y == self.y - - def __ne__(self, other): - return not self.__eq__(other) - -Usage with :func:`_orm.composite` then proceeds where the columns to be -associated with the ``Point`` class must also be declared with explicit -types, using one of the forms at :ref:`orm_composite_other_forms`. - - -Tracking In-Place Mutations on Composites ------------------------------------------ - -In-place changes to an existing composite value are -not tracked automatically. Instead, the composite class needs to provide -events to its parent object explicitly. This task is largely automated -via the usage of the :class:`.MutableComposite` mixin, which uses events -to associate each user-defined composite object with all parent associations. -Please see the example in :ref:`mutable_composites`. - -.. _composite_operations: - -Redefining Comparison Operations for Composites ------------------------------------------------ - -The "equals" comparison operation by default produces an AND of all -corresponding columns equated to one another. This can be changed using -the ``comparator_factory`` argument to :func:`.composite`, where we -specify a custom :class:`.CompositeProperty.Comparator` class -to define existing or new operations. -Below we illustrate the "greater than" operator, implementing -the same expression that the base "greater than" does:: - - import dataclasses - - from sqlalchemy.orm import composite - from sqlalchemy.orm import CompositeProperty - from sqlalchemy.orm import DeclarativeBase - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.sql import and_ - - - @dataclasses.dataclass - class Point: - x: int - y: int - - - class PointComparator(CompositeProperty.Comparator): - def __gt__(self, other): - """redefine the 'greater than' operation""" - - return and_( - *[ - a > b - for a, b in zip( - self.__clause_element__().clauses, - dataclasses.astuple(other), - ) - ] - ) - - - class Base(DeclarativeBase): - pass - - - class Vertex(Base): - __tablename__ = "vertices" - - id: Mapped[int] = mapped_column(primary_key=True) - - start: Mapped[Point] = composite( - mapped_column("x1"), mapped_column("y1"), comparator_factory=PointComparator - ) - end: Mapped[Point] = composite( - mapped_column("x2"), mapped_column("y2"), comparator_factory=PointComparator - ) - -Since ``Point`` is a dataclass, we may make use of -``dataclasses.astuple()`` to get a tuple form of ``Point`` instances. - -The custom comparator then returns the appropriate SQL expression: - -.. sourcecode:: pycon+sql - - >>> print(Vertex.start > Point(5, 6)) - {printsql}vertices.x1 > :x1_1 AND vertices.y1 > :y1_1 - - -Nesting Composites -------------------- - -Composite objects can be defined to work in simple nested schemes, by -redefining behaviors within the composite class to work as desired, then -mapping the composite class to the full length of individual columns normally. -This requires that additional methods to move between the "nested" and -"flat" forms are defined. - -Below we reorganize the ``Vertex`` class to itself be a composite object which -refers to ``Point`` objects. ``Vertex`` and ``Point`` can be dataclasses, -however we will add a custom construction method to ``Vertex`` that can be used -to create new ``Vertex`` objects given four column values, which will will -arbitrarily name ``_generate()`` and define as a classmethod so that we can -make new ``Vertex`` objects by passing values to the ``Vertex._generate()`` -method. - -We will also implement the ``__composite_values__()`` method, which is a fixed -name recognized by the :func:`_orm.composite` construct (introduced previously -at :ref:`composite_legacy_no_dataclass`) that indicates a standard way of -receiving the object as a flat tuple of column values, which in this case will -supersede the usual dataclass-oriented methodology. - -With our custom ``_generate()`` constructor and -``__composite_values__()`` serializer method, we can now move between -a flat tuple of columns and ``Vertex`` objects that contain ``Point`` -instances. The ``Vertex._generate`` method is passed as the -first argument to the :func:`_orm.composite` construct as the source of new -``Vertex`` instances, and the ``__composite_values__()`` method will be -used implicitly by :func:`_orm.composite`. - -For the purposes of the example, the ``Vertex`` composite is then mapped to a -class called ``HasVertex``, which is where the :class:`.Table` containing the -four source columns ultimately resides:: - - from __future__ import annotations - - import dataclasses - from typing import Any - from typing import Tuple - - from sqlalchemy.orm import composite - from sqlalchemy.orm import DeclarativeBase - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - - - @dataclasses.dataclass - class Point: - x: int - y: int - - - @dataclasses.dataclass - class Vertex: - start: Point - end: Point - - @classmethod - def _generate(cls, x1: int, y1: int, x2: int, y2: int) -> Vertex: - """generate a Vertex from a row""" - return Vertex(Point(x1, y1), Point(x2, y2)) - - def __composite_values__(self) -> Tuple[Any, ...]: - """generate a row from a Vertex""" - return dataclasses.astuple(self.start) + dataclasses.astuple(self.end) - - - class Base(DeclarativeBase): - pass - - - class HasVertex(Base): - __tablename__ = "has_vertex" - id: Mapped[int] = mapped_column(primary_key=True) - x1: Mapped[int] - y1: Mapped[int] - x2: Mapped[int] - y2: Mapped[int] - - vertex: Mapped[Vertex] = composite(Vertex._generate, "x1", "y1", "x2", "y2") - -The above mapping can then be used in terms of ``HasVertex``, ``Vertex``, and -``Point``:: - - hv = HasVertex(vertex=Vertex(Point(1, 2), Point(3, 4))) - - session.add(hv) - session.commit() - - stmt = select(HasVertex).where(HasVertex.vertex == Vertex(Point(1, 2), Point(3, 4))) - - hv = session.scalars(stmt).first() - print(hv.vertex.start) - print(hv.vertex.end) - -.. _dataclass: https://docs.python.org/3/library/dataclasses.html - -Composite API -------------- - -.. autofunction:: composite - diff --git a/doc/build/orm/contextual.rst b/doc/build/orm/contextual.rst deleted file mode 100644 index 3e03e93167b..00000000000 --- a/doc/build/orm/contextual.rst +++ /dev/null @@ -1,283 +0,0 @@ -.. _unitofwork_contextual: - -Contextual/Thread-local Sessions -================================ - -Recall from the section :ref:`session_faq_whentocreate`, the concept of -"session scopes" was introduced, with an emphasis on web applications -and the practice of linking the scope of a :class:`.Session` with that -of a web request. Most modern web frameworks include integration tools -so that the scope of the :class:`.Session` can be managed automatically, -and these tools should be used as they are available. - -SQLAlchemy includes its own helper object, which helps with the establishment -of user-defined :class:`.Session` scopes. It is also used by third-party -integration systems to help construct their integration schemes. - -The object is the :class:`.scoped_session` object, and it represents a -**registry** of :class:`.Session` objects. If you're not familiar with the -registry pattern, a good introduction can be found in `Patterns of Enterprise -Architecture `_. - -.. warning:: - - The :class:`.scoped_session` registry by default uses a Python - ``threading.local()`` - in order to track :class:`_orm.Session` instances. **This is not - necessarily compatible with all application servers**, particularly those - which make use of greenlets or other alternative forms of concurrency - control, which may lead to race conditions (e.g. randomly occurring - failures) when used in moderate to high concurrency scenarios. - Please read :ref:`unitofwork_contextual_threadlocal` and - :ref:`session_lifespan` below to more fully understand the implications - of using ``threading.local()`` to track :class:`_orm.Session` objects - and consider more explicit means of scoping when using application servers - which are not based on traditional threads. - -.. note:: - - The :class:`.scoped_session` object is a very popular and useful object - used by many SQLAlchemy applications. However, it is important to note - that it presents **only one approach** to the issue of :class:`.Session` - management. If you're new to SQLAlchemy, and especially if the - term "thread-local variable" seems strange to you, we recommend that - if possible you familiarize first with an off-the-shelf integration - system such as `Flask-SQLAlchemy `_ - or `zope.sqlalchemy `_. - -A :class:`.scoped_session` is constructed by calling it, passing it a -**factory** which can create new :class:`.Session` objects. A factory -is just something that produces a new object when called, and in the -case of :class:`.Session`, the most common factory is the :class:`.sessionmaker`, -introduced earlier in this section. Below we illustrate this usage:: - - >>> from sqlalchemy.orm import scoped_session - >>> from sqlalchemy.orm import sessionmaker - - >>> session_factory = sessionmaker(bind=some_engine) - >>> Session = scoped_session(session_factory) - -The :class:`.scoped_session` object we've created will now call upon the -:class:`.sessionmaker` when we "call" the registry:: - - >>> some_session = Session() - -Above, ``some_session`` is an instance of :class:`.Session`, which we -can now use to talk to the database. This same :class:`.Session` is also -present within the :class:`.scoped_session` registry we've created. If -we call upon the registry a second time, we get back the **same** :class:`.Session`:: - - >>> some_other_session = Session() - >>> some_session is some_other_session - True - -This pattern allows disparate sections of the application to call upon a global -:class:`.scoped_session`, so that all those areas may share the same session -without the need to pass it explicitly. The :class:`.Session` we've established -in our registry will remain, until we explicitly tell our registry to dispose of it, -by calling :meth:`.scoped_session.remove`:: - - >>> Session.remove() - -The :meth:`.scoped_session.remove` method first calls :meth:`.Session.close` on -the current :class:`.Session`, which has the effect of releasing any connection/transactional -resources owned by the :class:`.Session` first, then discarding the :class:`.Session` -itself. "Releasing" here means that connections are returned to their connection pool and any transactional state is rolled back, ultimately using the ``rollback()`` method of the underlying DBAPI connection. - -At this point, the :class:`.scoped_session` object is "empty", and will create -a **new** :class:`.Session` when called again. As illustrated below, this -is not the same :class:`.Session` we had before:: - - >>> new_session = Session() - >>> new_session is some_session - False - -The above series of steps illustrates the idea of the "registry" pattern in a -nutshell. With that basic idea in hand, we can discuss some of the details -of how this pattern proceeds. - -Implicit Method Access ----------------------- - -The job of the :class:`.scoped_session` is simple; hold onto a :class:`.Session` -for all who ask for it. As a means of producing more transparent access to this -:class:`.Session`, the :class:`.scoped_session` also includes **proxy behavior**, -meaning that the registry itself can be treated just like a :class:`.Session` -directly; when methods are called on this object, they are **proxied** to the -underlying :class:`.Session` being maintained by the registry:: - - Session = scoped_session(some_factory) - - # equivalent to: - # - # session = Session() - # print(session.scalars(select(MyClass)).all()) - # - print(Session.scalars(select(MyClass)).all()) - -The above code accomplishes the same task as that of acquiring the current -:class:`.Session` by calling upon the registry, then using that :class:`.Session`. - -.. _unitofwork_contextual_threadlocal: - -Thread-Local Scope ------------------- - -Users who are familiar with multithreaded programming will note that representing -anything as a global variable is usually a bad idea, as it implies that the -global object will be accessed by many threads concurrently. The :class:`.Session` -object is entirely designed to be used in a **non-concurrent** fashion, which -in terms of multithreading means "only in one thread at a time". So our -above example of :class:`.scoped_session` usage, where the same :class:`.Session` -object is maintained across multiple calls, suggests that some process needs -to be in place such that multiple calls across many threads don't actually get -a handle to the same session. We call this notion **thread local storage**, -which means, a special object is used that will maintain a distinct object -per each application thread. Python provides this via the -`threading.local() `_ -construct. The :class:`.scoped_session` object by default uses this object -as storage, so that a single :class:`.Session` is maintained for all who call -upon the :class:`.scoped_session` registry, but only within the scope of a single -thread. Callers who call upon the registry in a different thread get a -:class:`.Session` instance that is local to that other thread. - -Using this technique, the :class:`.scoped_session` provides a quick and relatively -simple (if one is familiar with thread-local storage) way of providing -a single, global object in an application that is safe to be called upon -from multiple threads. - -The :meth:`.scoped_session.remove` method, as always, removes the current -:class:`.Session` associated with the thread, if any. However, one advantage of the -``threading.local()`` object is that if the application thread itself ends, the -"storage" for that thread is also garbage collected. So it is in fact "safe" to -use thread local scope with an application that spawns and tears down threads, -without the need to call :meth:`.scoped_session.remove`. However, the scope -of transactions themselves, i.e. ending them via :meth:`.Session.commit` or -:meth:`.Session.rollback`, will usually still be something that must be explicitly -arranged for at the appropriate time, unless the application actually ties the -lifespan of a thread to the lifespan of a transaction. - -.. _session_lifespan: - -Using Thread-Local Scope with Web Applications ----------------------------------------------- - -As discussed in the section :ref:`session_faq_whentocreate`, a web application -is architected around the concept of a **web request**, and integrating -such an application with the :class:`.Session` usually implies that the :class:`.Session` -will be associated with that request. As it turns out, most Python web frameworks, -with notable exceptions such as the asynchronous frameworks Twisted and -Tornado, use threads in a simple way, such that a particular web request is received, -processed, and completed within the scope of a single *worker thread*. When -the request ends, the worker thread is released to a pool of workers where it -is available to handle another request. - -This simple correspondence of web request and thread means that to associate a -:class:`.Session` with a thread implies it is also associated with the web request -running within that thread, and vice versa, provided that the :class:`.Session` is -created only after the web request begins and torn down just before the web request ends. -So it is a common practice to use :class:`.scoped_session` as a quick way -to integrate the :class:`.Session` with a web application. The sequence -diagram below illustrates this flow: - -.. sourcecode:: text - - Web Server Web Framework SQLAlchemy ORM Code - -------------- -------------- ------------------------------ - startup -> Web framework # Session registry is established - initializes Session = scoped_session(sessionmaker()) - - incoming - web request -> web request -> # The registry is *optionally* - starts # called upon explicitly to create - # a Session local to the thread and/or request - Session() - - # the Session registry can otherwise - # be used at any time, creating the - # request-local Session() if not present, - # or returning the existing one - Session.execute(select(MyClass)) # ... - - Session.add(some_object) # ... - - # if data was modified, commit the - # transaction - Session.commit() - - web request ends -> # the registry is instructed to - # remove the Session - Session.remove() - - sends output <- - outgoing web <- - response - -Using the above flow, the process of integrating the :class:`.Session` with the -web application has exactly two requirements: - -1. Create a single :class:`.scoped_session` registry when the web application - first starts, ensuring that this object is accessible by the rest of the - application. -2. Ensure that :meth:`.scoped_session.remove` is called when the web request ends, - usually by integrating with the web framework's event system to establish - an "on request end" event. - -As noted earlier, the above pattern is **just one potential way** to integrate a :class:`.Session` -with a web framework, one which in particular makes the significant assumption -that the **web framework associates web requests with application threads**. It is -however **strongly recommended that the integration tools provided with the web framework -itself be used, if available**, instead of :class:`.scoped_session`. - -In particular, while using a thread local can be convenient, it is preferable that the :class:`.Session` be -associated **directly with the request**, rather than with -the current thread. The next section on custom scopes details a more advanced configuration -which can combine the usage of :class:`.scoped_session` with direct request based scope, or -any kind of scope. - -Using Custom Created Scopes ---------------------------- - -The :class:`.scoped_session` object's default behavior of "thread local" scope is only -one of many options on how to "scope" a :class:`.Session`. A custom scope can be defined -based on any existing system of getting at "the current thing we are working with". - -Suppose a web framework defines a library function ``get_current_request()``. An application -built using this framework can call this function at any time, and the result will be -some kind of ``Request`` object that represents the current request being processed. -If the ``Request`` object is hashable, then this function can be easily integrated with -:class:`.scoped_session` to associate the :class:`.Session` with the request. Below we illustrate -this in conjunction with a hypothetical event marker provided by the web framework -``on_request_end``, which allows code to be invoked whenever a request ends:: - - from my_web_framework import get_current_request, on_request_end - from sqlalchemy.orm import scoped_session, sessionmaker - - Session = scoped_session(sessionmaker(bind=some_engine), scopefunc=get_current_request) - - - @on_request_end - def remove_session(req): - Session.remove() - -Above, we instantiate :class:`.scoped_session` in the usual way, except that we pass -our request-returning function as the "scopefunc". This instructs :class:`.scoped_session` -to use this function to generate a dictionary key whenever the registry is called upon -to return the current :class:`.Session`. In this case it is particularly important -that we ensure a reliable "remove" system is implemented, as this dictionary is not -otherwise self-managed. - - -Contextual Session API ----------------------- - -.. autoclass:: sqlalchemy.orm.scoped_session - :members: - :inherited-members: - -.. autoclass:: sqlalchemy.util.ScopedRegistry - :members: - -.. autoclass:: sqlalchemy.util.ThreadLocalRegistry - -.. autoclass:: sqlalchemy.orm.QueryPropertyDescriptor diff --git a/doc/build/orm/dataclasses.rst b/doc/build/orm/dataclasses.rst deleted file mode 100644 index bbd05fcd5e5..00000000000 --- a/doc/build/orm/dataclasses.rst +++ /dev/null @@ -1,1085 +0,0 @@ -.. _orm_dataclasses_toplevel: - -====================================== -Integration with dataclasses and attrs -====================================== - -SQLAlchemy as of version 2.0 features "native dataclass" integration where -an :ref:`Annotated Declarative Table ` -mapping may be turned into a Python dataclass_ by adding a single mixin -or decorator to mapped classes. - -.. versionadded:: 2.0 Integrated dataclass creation with ORM Declarative classes - -There are also patterns available that allow existing dataclasses to be -mapped, as well as to map classes instrumented by the -attrs_ third party integration library. - -.. _orm_declarative_native_dataclasses: - -Declarative Dataclass Mapping -------------------------------- - -SQLAlchemy :ref:`Annotated Declarative Table ` -mappings may be augmented with an additional -mixin class or decorator directive, which will add an additional step to -the Declarative process after the mapping is complete that will convert -the mapped class **in-place** into a Python dataclass_, before completing -the mapping process which applies ORM-specific :term:`instrumentation` -to the class. The most prominent behavioral addition this provides is -generation of an ``__init__()`` method with fine-grained control over -positional and keyword arguments with or without defaults, as well as -generation of methods like ``__repr__()`` and ``__eq__()``. - -From a :pep:`484` typing perspective, the class is recognized -as having Dataclass-specific behaviors, most notably by taking advantage of :pep:`681` -"Dataclass Transforms", which allows typing tools to consider the class -as though it were explicitly decorated using the ``@dataclasses.dataclass`` -decorator. - -.. note:: Support for :pep:`681` in typing tools as of **April 4, 2023** is - limited and is currently known to be supported by Pyright_ as well - as Mypy_ as of **version 1.2**. Note that Mypy 1.1.1 introduced - :pep:`681` support but did not correctly accommodate Python descriptors - which will lead to errors when using SQLAlhcemy's ORM mapping scheme. - - .. seealso:: - - https://peps.python.org/pep-0681/#the-dataclass-transform-decorator - background - on how libraries like SQLAlchemy enable :pep:`681` support - - -Dataclass conversion may be added to any Declarative class either by adding the -:class:`_orm.MappedAsDataclass` mixin to a :class:`_orm.DeclarativeBase` class -hierarchy, or for decorator mapping by using the -:meth:`_orm.registry.mapped_as_dataclass` class decorator. - -The :class:`_orm.MappedAsDataclass` mixin may be applied either -to the Declarative ``Base`` class or any superclass, as in the example -below:: - - - from sqlalchemy.orm import DeclarativeBase - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import MappedAsDataclass - - - class Base(MappedAsDataclass, DeclarativeBase): - """subclasses will be converted to dataclasses""" - - - class User(Base): - __tablename__ = "user_account" - - id: Mapped[int] = mapped_column(init=False, primary_key=True) - name: Mapped[str] - -Or may be applied directly to classes that extend from the Declarative base:: - - from sqlalchemy.orm import DeclarativeBase - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import MappedAsDataclass - - - class Base(DeclarativeBase): - pass - - - class User(MappedAsDataclass, Base): - """User class will be converted to a dataclass""" - - __tablename__ = "user_account" - - id: Mapped[int] = mapped_column(init=False, primary_key=True) - name: Mapped[str] - -When using the decorator form, only the :meth:`_orm.registry.mapped_as_dataclass` -decorator is supported:: - - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import registry - - - reg = registry() - - - @reg.mapped_as_dataclass - class User: - __tablename__ = "user_account" - - id: Mapped[int] = mapped_column(init=False, primary_key=True) - name: Mapped[str] - -Class level feature configuration -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Support for dataclasses features is partial. Currently **supported** are -the ``init``, ``repr``, ``eq``, ``order`` and ``unsafe_hash`` features, -``match_args`` and ``kw_only`` are supported on Python 3.10+. -Currently **not supported** are the ``frozen`` and ``slots`` features. - -When using the mixin class form with :class:`_orm.MappedAsDataclass`, -class configuration arguments are passed as class-level parameters:: - - from sqlalchemy.orm import DeclarativeBase - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import MappedAsDataclass - - - class Base(DeclarativeBase): - pass - - - class User(MappedAsDataclass, Base, repr=False, unsafe_hash=True): - """User class will be converted to a dataclass""" - - __tablename__ = "user_account" - - id: Mapped[int] = mapped_column(init=False, primary_key=True) - name: Mapped[str] - -When using the decorator form with :meth:`_orm.registry.mapped_as_dataclass`, -class configuration arguments are passed to the decorator directly:: - - from sqlalchemy.orm import registry - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - - - reg = registry() - - - @reg.mapped_as_dataclass(unsafe_hash=True) - class User: - """User class will be converted to a dataclass""" - - __tablename__ = "user_account" - - id: Mapped[int] = mapped_column(init=False, primary_key=True) - name: Mapped[str] - -For background on dataclass class options, see the dataclasses_ documentation -at `@dataclasses.dataclass `_. - -Attribute Configuration -^^^^^^^^^^^^^^^^^^^^^^^ - -SQLAlchemy native dataclasses differ from normal dataclasses in that -attributes to be mapped are described using the :class:`_orm.Mapped` -generic annotation container in all cases. Mappings follow the same -forms as those documented at :ref:`orm_declarative_table`, and all -features of :func:`_orm.mapped_column` and :class:`_orm.Mapped` are supported. - -Additionally, ORM attribute configuration constructs including -:func:`_orm.mapped_column`, :func:`_orm.relationship` and :func:`_orm.composite` -support **per-attribute field options**, including ``init``, ``default``, -``default_factory`` and ``repr``. The names of these arguments is fixed -as specified in :pep:`681`. Functionality is equivalent to dataclasses: - -* ``init``, as in :paramref:`_orm.mapped_column.init`, - :paramref:`_orm.relationship.init`, if False indicates the field should - not be part of the ``__init__()`` method -* ``default``, as in :paramref:`_orm.mapped_column.default`, - :paramref:`_orm.relationship.default` - indicates a default value for the field as given as a keyword argument - in the ``__init__()`` method. -* ``default_factory``, as in :paramref:`_orm.mapped_column.default_factory`, - :paramref:`_orm.relationship.default_factory`, indicates a callable function - that will be invoked to generate a new default value for a parameter - if not passed explicitly to the ``__init__()`` method. -* ``repr`` True by default, indicates the field should be part of the generated - ``__repr__()`` method - - -Another key difference from dataclasses is that default values for attributes -**must** be configured using the ``default`` parameter of the ORM construct, -such as ``mapped_column(default=None)``. A syntax that resembles dataclass -syntax which accepts simple Python values as defaults without using -``@dataclases.field()`` is not supported. - -As an example using :func:`_orm.mapped_column`, the mapping below will -produce an ``__init__()`` method that accepts only the fields ``name`` and -``fullname``, where ``name`` is required and may be passed positionally, -and ``fullname`` is optional. The ``id`` field, which we expect to be -database-generated, is not part of the constructor at all:: - - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import registry - - reg = registry() - - - @reg.mapped_as_dataclass - class User: - __tablename__ = "user_account" - - id: Mapped[int] = mapped_column(init=False, primary_key=True) - name: Mapped[str] - fullname: Mapped[str] = mapped_column(default=None) - - - # 'fullname' is optional keyword argument - u1 = User("name") - -Column Defaults -~~~~~~~~~~~~~~~ - -In order to accommodate the name overlap of the ``default`` argument with -the existing :paramref:`_schema.Column.default` parameter of the :class:`_schema.Column` -construct, the :func:`_orm.mapped_column` construct disambiguates the two -names by adding a new parameter :paramref:`_orm.mapped_column.insert_default`, -which will be populated directly into the -:paramref:`_schema.Column.default` parameter of :class:`_schema.Column`, -independently of what may be set on -:paramref:`_orm.mapped_column.default`, which is always used for the -dataclasses configuration. For example, to configure a datetime column with -a :paramref:`_schema.Column.default` set to the ``func.utc_timestamp()`` SQL function, -but where the parameter is optional in the constructor:: - - from datetime import datetime - - from sqlalchemy import func - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import registry - - reg = registry() - - - @reg.mapped_as_dataclass - class User: - __tablename__ = "user_account" - - id: Mapped[int] = mapped_column(init=False, primary_key=True) - created_at: Mapped[datetime] = mapped_column( - insert_default=func.utc_timestamp(), default=None - ) - -With the above mapping, an ``INSERT`` for a new ``User`` object where no -parameter for ``created_at`` were passed proceeds as: - -.. sourcecode:: pycon+sql - - >>> with Session(e) as session: - ... session.add(User()) - ... session.commit() - {execsql}BEGIN (implicit) - INSERT INTO user_account (created_at) VALUES (utc_timestamp()) - [generated in 0.00010s] () - COMMIT - - - -Integration with Annotated -~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The approach introduced at :ref:`orm_declarative_mapped_column_pep593` illustrates -how to use :pep:`593` ``Annotated`` objects to package whole -:func:`_orm.mapped_column` constructs for re-use. This feature is supported -with the dataclasses feature. One aspect of the feature however requires -a workaround when working with typing tools, which is that the -:pep:`681`-specific arguments ``init``, ``default``, ``repr``, and ``default_factory`` -**must** be on the right hand side, packaged into an explicit :func:`_orm.mapped_column` -construct, in order for the typing tool to interpret the attribute correctly. -As an example, the approach below will work perfectly fine at runtime, -however typing tools will consider the ``User()`` construction to be -invalid, as they do not see the ``init=False`` parameter present:: - - from typing import Annotated - - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import registry - - # typing tools will ignore init=False here - intpk = Annotated[int, mapped_column(init=False, primary_key=True)] - - reg = registry() - - - @reg.mapped_as_dataclass - class User: - __tablename__ = "user_account" - id: Mapped[intpk] - - - # typing error: Argument missing for parameter "id" - u1 = User() - -Instead, :func:`_orm.mapped_column` must be present on the right side -as well with an explicit setting for :paramref:`_orm.mapped_column.init`; -the other arguments can remain within the ``Annotated`` construct:: - - from typing import Annotated - - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import registry - - intpk = Annotated[int, mapped_column(primary_key=True)] - - reg = registry() - - - @reg.mapped_as_dataclass - class User: - __tablename__ = "user_account" - - # init=False and other pep-681 arguments must be inline - id: Mapped[intpk] = mapped_column(init=False) - - - u1 = User() - -.. _orm_declarative_dc_mixins: - -Using mixins and abstract superclasses -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Any mixins or base classes that are used in a :class:`_orm.MappedAsDataclass` -mapped class which include :class:`_orm.Mapped` attributes must themselves be -part of a :class:`_orm.MappedAsDataclass` -hierarchy, such as in the example below using a mixin:: - - - class Mixin(MappedAsDataclass): - - create_user: Mapped[int] = mapped_column() - update_user: Mapped[Optional[int]] = mapped_column(default=None, init=False) - - - class Base(DeclarativeBase, MappedAsDataclass): - pass - - - class User(Base, Mixin): - __tablename__ = "sys_user" - - uid: Mapped[str] = mapped_column( - String(50), init=False, default_factory=uuid4, primary_key=True - ) - username: Mapped[str] = mapped_column() - email: Mapped[str] = mapped_column() - -Python type checkers which support :pep:`681` will otherwise not consider -attributes from non-dataclass mixins to be part of the dataclass. - -.. deprecated:: 2.0.8 Using mixins and abstract bases within - :class:`_orm.MappedAsDataclass` or - :meth:`_orm.registry.mapped_as_dataclass` hierarchies which are not - themselves dataclasses is deprecated, as these fields are not supported - by :pep:`681` as belonging to the dataclass. A warning is emitted for this - case which will later be an error. - - .. seealso:: - - :ref:`error_dcmx` - background on rationale - - - - -Relationship Configuration -^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The :class:`_orm.Mapped` annotation in combination with -:func:`_orm.relationship` is used in the same way as described at -:ref:`relationship_patterns`. When specifying a collection-based -:func:`_orm.relationship` as an optional keyword argument, the -:paramref:`_orm.relationship.default_factory` parameter must be passed and it -must refer to the collection class that's to be used. Many-to-one and -scalar object references may make use of -:paramref:`_orm.relationship.default` if the default value is to be ``None``:: - - from typing import List - - from sqlalchemy import ForeignKey - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import registry - from sqlalchemy.orm import relationship - - reg = registry() - - - @reg.mapped_as_dataclass - class Parent: - __tablename__ = "parent" - id: Mapped[int] = mapped_column(primary_key=True) - children: Mapped[List["Child"]] = relationship( - default_factory=list, back_populates="parent" - ) - - - @reg.mapped_as_dataclass - class Child: - __tablename__ = "child" - id: Mapped[int] = mapped_column(primary_key=True) - parent_id: Mapped[int] = mapped_column(ForeignKey("parent.id")) - parent: Mapped["Parent"] = relationship(default=None) - -The above mapping will generate an empty list for ``Parent.children`` when a -new ``Parent()`` object is constructed without passing ``children``, and -similarly a ``None`` value for ``Child.parent`` when a new ``Child()`` object -is constructed without passsing ``parent``. - -While the :paramref:`_orm.relationship.default_factory` can be automatically -derived from the given collection class of the :func:`_orm.relationship` -itself, this would break compatibility with dataclasses, as the presence -of :paramref:`_orm.relationship.default_factory` or -:paramref:`_orm.relationship.default` is what determines if the parameter is -to be required or optional when rendered into the ``__init__()`` method. - -.. _orm_declarative_native_dataclasses_non_mapped_fields: - -Using Non-Mapped Dataclass Fields -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -When using Declarative dataclasses, non-mapped fields may be used on the -class as well, which will be part of the dataclass construction process but -will not be mapped. Any field that does not use :class:`.Mapped` will -be ignored by the mapping process. In the example below, the fields -``ctrl_one`` and ``ctrl_two`` will be part of the instance-level state -of the object, but will not be persisted by the ORM:: - - - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import registry - - reg = registry() - - - @reg.mapped_as_dataclass - class Data: - __tablename__ = "data" - - id: Mapped[int] = mapped_column(init=False, primary_key=True) - status: Mapped[str] - - ctrl_one: Optional[str] = None - ctrl_two: Optional[str] = None - -Instance of ``Data`` above can be created as:: - - d1 = Data(status="s1", ctrl_one="ctrl1", ctrl_two="ctrl2") - -A more real world example might be to make use of the Dataclasses -``InitVar`` feature in conjunction with the ``__post_init__()`` feature to -receive init-only fields that can be used to compose persisted data. -In the example below, the ``User`` -class is declared using ``id``, ``name`` and ``password_hash`` as mapped features, -but makes use of init-only ``password`` and ``repeat_password`` fields to -represent the user creation process (note: to run this example, replace -the function ``your_crypt_function_here()`` with a third party crypt -function, such as `bcrypt `_ or -`argon2-cffi `_):: - - from dataclasses import InitVar - from typing import Optional - - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import registry - - reg = registry() - - - @reg.mapped_as_dataclass - class User: - __tablename__ = "user_account" - - id: Mapped[int] = mapped_column(init=False, primary_key=True) - name: Mapped[str] - - password: InitVar[str] - repeat_password: InitVar[str] - - password_hash: Mapped[str] = mapped_column(init=False, nullable=False) - - def __post_init__(self, password: str, repeat_password: str): - if password != repeat_password: - raise ValueError("passwords do not match") - - self.password_hash = your_crypt_function_here(password) - -The above object is created with parameters ``password`` and -``repeat_password``, which are consumed up front so that the ``password_hash`` -variable may be generated:: - - >>> u1 = User(name="some_user", password="xyz", repeat_password="xyz") - >>> u1.password_hash - '$6$9ppc... (example crypted string....)' - -.. versionchanged:: 2.0.0rc1 When using :meth:`_orm.registry.mapped_as_dataclass` - or :class:`.MappedAsDataclass`, fields that do not include the - :class:`.Mapped` annotation may be included, which will be treated as part - of the resulting dataclass but not be mapped, without the need to - also indicate the ``__allow_unmapped__`` class attribute. Previous 2.0 - beta releases would require this attribute to be explicitly present, - even though the purpose of this attribute was only to allow legacy - ORM typed mappings to continue to function. - -.. _dataclasses_pydantic: - -Integrating with Alternate Dataclass Providers such as Pydantic -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -SQLAlchemy's :class:`_orm.MappedAsDataclass` class -and :meth:`_orm.registry.mapped_as_dataclass` method call directly into -the Python standard library ``dataclasses.dataclass`` class decorator, after -the declarative mapping process has been applied to the class. This -function call may be swapped out for alternateive dataclasses providers, -such as that of Pydantic, using the ``dataclass_callable`` parameter -accepted by :class:`_orm.MappedAsDataclass` as a class keyword argument -as well as by :meth:`_orm.registry.mapped_as_dataclass`:: - - from sqlalchemy.orm import DeclarativeBase - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import MappedAsDataclass - from sqlalchemy.orm import registry - - - class Base( - MappedAsDataclass, - DeclarativeBase, - dataclass_callable=pydantic.dataclasses.dataclass, - ): - pass - - - class User(Base): - __tablename__ = "user" - - id: Mapped[int] = mapped_column(primary_key=True) - name: Mapped[str] - -The above ``User`` class will be applied as a dataclass, using Pydantic's -``pydantic.dataclasses.dataclasses`` callable. The process is available -both for mapped classes as well as mixins that extend from -:class:`_orm.MappedAsDataclass` or which have -:meth:`_orm.registry.mapped_as_dataclass` applied directly. - -.. versionadded:: 2.0.4 Added the ``dataclass_callable`` class and method - parameters for :class:`_orm.MappedAsDataclass` and - :meth:`_orm.registry.mapped_as_dataclass`, and adjusted some of the - dataclass internals to accommodate more strict dataclass functions such as - that of Pydantic. - - -.. _orm_declarative_dataclasses: - -Applying ORM Mappings to an existing dataclass (legacy dataclass use) ---------------------------------------------------------------------- - -.. legacy:: - - The approaches described here are superseded by - the :ref:`orm_declarative_native_dataclasses` feature new in the 2.0 - series of SQLAlchemy. This newer version of the feature builds upon - the dataclass support first added in version 1.4, which is described - in this section. - -To map an existing dataclass, SQLAlchemy's "inline" declarative directives -cannot be used directly; ORM directives are assigned using one of three -techniques: - -* Using "Declarative with Imperative Table", the table / column to be mapped - is defined using a :class:`_schema.Table` object assigned to the - ``__table__`` attribute of the class; relationships are defined within - ``__mapper_args__`` dictionary. The class is mapped using the - :meth:`_orm.registry.mapped` decorator. An example is below at - :ref:`orm_declarative_dataclasses_imperative_table`. - -* Using full "Declarative", the Declarative-interpreted directives such as - :class:`_schema.Column`, :func:`_orm.relationship` are added to the - ``.metadata`` dictionary of the ``dataclasses.field()`` construct, where - they are consumed by the declarative process. The class is again - mapped using the :meth:`_orm.registry.mapped` decorator. See the example - below at :ref:`orm_declarative_dataclasses_declarative_table`. - -* An "Imperative" mapping can be applied to an existing dataclass using - the :meth:`_orm.registry.map_imperatively` method to produce the mapping - in exactly the same way as described at :ref:`orm_imperative_mapping`. - This is illustrated below at :ref:`orm_imperative_dataclasses`. - -The general process by which SQLAlchemy applies mappings to a dataclass -is the same as that of an ordinary class, but also includes that -SQLAlchemy will detect class-level attributes that were part of the -dataclasses declaration process and replace them at runtime with -the usual SQLAlchemy ORM mapped attributes. The ``__init__`` method that -would have been generated by dataclasses is left intact, as is the same -for all the other methods that dataclasses generates such as -``__eq__()``, ``__repr__()``, etc. - -.. _orm_declarative_dataclasses_imperative_table: - -Mapping pre-existing dataclasses using Declarative With Imperative Table -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -An example of a mapping using ``@dataclass`` using -:ref:`orm_imperative_table_configuration` is below. A complete -:class:`_schema.Table` object is constructed explicitly and assigned to the -``__table__`` attribute. Instance fields are defined using normal dataclass -syntaxes. Additional :class:`.MapperProperty` -definitions such as :func:`.relationship`, are placed in the -:ref:`__mapper_args__ ` class-level -dictionary underneath the ``properties`` key, corresponding to the -:paramref:`_orm.Mapper.properties` parameter:: - - from __future__ import annotations - - from dataclasses import dataclass, field - from typing import List, Optional - - from sqlalchemy import Column, ForeignKey, Integer, String, Table - from sqlalchemy.orm import registry, relationship - - mapper_registry = registry() - - - @mapper_registry.mapped - @dataclass - class User: - __table__ = Table( - "user", - mapper_registry.metadata, - Column("id", Integer, primary_key=True), - Column("name", String(50)), - Column("fullname", String(50)), - Column("nickname", String(12)), - ) - id: int = field(init=False) - name: Optional[str] = None - fullname: Optional[str] = None - nickname: Optional[str] = None - addresses: List[Address] = field(default_factory=list) - - __mapper_args__ = { # type: ignore - "properties": { - "addresses": relationship("Address"), - } - } - - - @mapper_registry.mapped - @dataclass - class Address: - __table__ = Table( - "address", - mapper_registry.metadata, - Column("id", Integer, primary_key=True), - Column("user_id", Integer, ForeignKey("user.id")), - Column("email_address", String(50)), - ) - id: int = field(init=False) - user_id: int = field(init=False) - email_address: Optional[str] = None - -In the above example, the ``User.id``, ``Address.id``, and ``Address.user_id`` -attributes are defined as ``field(init=False)``. This means that parameters for -these won't be added to ``__init__()`` methods, but -:class:`.Session` will still be able to set them after getting their values -during flush from autoincrement or other default value generator. To -allow them to be specified in the constructor explicitly, they would instead -be given a default value of ``None``. - -For a :func:`_orm.relationship` to be declared separately, it needs to be -specified directly within the :paramref:`_orm.Mapper.properties` dictionary -which itself is specified within the ``__mapper_args__`` dictionary, so that it -is passed to the constructor for :class:`_orm.Mapper`. An alternative to this -approach is in the next example. - -.. _orm_declarative_dataclasses_declarative_table: - -Mapping pre-existing dataclasses using Declarative-style fields -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. legacy:: This approach to Declarative mapping with - dataclasses should be considered as legacy. It will remain supported - however is unlikely to offer any advantages against the new - approach detailed at :ref:`orm_declarative_native_dataclasses`. - - Note that **mapped_column() is not supported with this use**; - the :class:`_schema.Column` construct should continue to be used to declare - table metadata within the ``metadata`` field of ``dataclasses.field()``. - -The fully declarative approach requires that :class:`_schema.Column` objects -are declared as class attributes, which when using dataclasses would conflict -with the dataclass-level attributes. An approach to combine these together -is to make use of the ``metadata`` attribute on the ``dataclass.field`` -object, where SQLAlchemy-specific mapping information may be supplied. -Declarative supports extraction of these parameters when the class -specifies the attribute ``__sa_dataclass_metadata_key__``. This also -provides a more succinct method of indicating the :func:`_orm.relationship` -association:: - - - from __future__ import annotations - - from dataclasses import dataclass, field - from typing import List - - from sqlalchemy import Column, ForeignKey, Integer, String - from sqlalchemy.orm import registry, relationship - - mapper_registry = registry() - - - @mapper_registry.mapped - @dataclass - class User: - __tablename__ = "user" - - __sa_dataclass_metadata_key__ = "sa" - id: int = field(init=False, metadata={"sa": Column(Integer, primary_key=True)}) - name: str = field(default=None, metadata={"sa": Column(String(50))}) - fullname: str = field(default=None, metadata={"sa": Column(String(50))}) - nickname: str = field(default=None, metadata={"sa": Column(String(12))}) - addresses: List[Address] = field( - default_factory=list, metadata={"sa": relationship("Address")} - ) - - - @mapper_registry.mapped - @dataclass - class Address: - __tablename__ = "address" - __sa_dataclass_metadata_key__ = "sa" - id: int = field(init=False, metadata={"sa": Column(Integer, primary_key=True)}) - user_id: int = field(init=False, metadata={"sa": Column(ForeignKey("user.id"))}) - email_address: str = field(default=None, metadata={"sa": Column(String(50))}) - -.. _orm_declarative_dataclasses_mixin: - -Using Declarative Mixins with pre-existing dataclasses -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -In the section :ref:`orm_mixins_toplevel`, Declarative Mixin classes -are introduced. One requirement of declarative mixins is that certain -constructs that can't be easily duplicated must be given as callables, -using the :class:`_orm.declared_attr` decorator, such as in the -example at :ref:`orm_declarative_mixins_relationships`:: - - class RefTargetMixin: - @declared_attr - def target_id(cls): - return Column("target_id", ForeignKey("target.id")) - - @declared_attr - def target(cls): - return relationship("Target") - -This form is supported within the Dataclasses ``field()`` object by using -a lambda to indicate the SQLAlchemy construct inside the ``field()``. -Using :func:`_orm.declared_attr` to surround the lambda is optional. -If we wanted to produce our ``User`` class above where the ORM fields -came from a mixin that is itself a dataclass, the form would be:: - - @dataclass - class UserMixin: - __tablename__ = "user" - - __sa_dataclass_metadata_key__ = "sa" - - id: int = field(init=False, metadata={"sa": Column(Integer, primary_key=True)}) - - addresses: List[Address] = field( - default_factory=list, metadata={"sa": lambda: relationship("Address")} - ) - - - @dataclass - class AddressMixin: - __tablename__ = "address" - __sa_dataclass_metadata_key__ = "sa" - id: int = field(init=False, metadata={"sa": Column(Integer, primary_key=True)}) - user_id: int = field( - init=False, metadata={"sa": lambda: Column(ForeignKey("user.id"))} - ) - email_address: str = field(default=None, metadata={"sa": Column(String(50))}) - - - @mapper_registry.mapped - class User(UserMixin): - pass - - - @mapper_registry.mapped - class Address(AddressMixin): - pass - -.. versionadded:: 1.4.2 Added support for "declared attr" style mixin attributes, - namely :func:`_orm.relationship` constructs as well as :class:`_schema.Column` - objects with foreign key declarations, to be used within "Dataclasses - with Declarative Table" style mappings. - - - -.. _orm_imperative_dataclasses: - -Mapping pre-existing dataclasses using Imperative Mapping -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -As described previously, a class which is set up as a dataclass using the -``@dataclass`` decorator can then be further decorated using the -:meth:`_orm.registry.mapped` decorator in order to apply declarative-style -mapping to the class. As an alternative to using the -:meth:`_orm.registry.mapped` decorator, we may also pass the class through the -:meth:`_orm.registry.map_imperatively` method instead, so that we may pass all -:class:`_schema.Table` and :class:`_orm.Mapper` configuration imperatively to -the function rather than having them defined on the class itself as class -variables:: - - from __future__ import annotations - - from dataclasses import dataclass - from dataclasses import field - from typing import List - - from sqlalchemy import Column - from sqlalchemy import ForeignKey - from sqlalchemy import Integer - from sqlalchemy import MetaData - from sqlalchemy import String - from sqlalchemy import Table - from sqlalchemy.orm import registry - from sqlalchemy.orm import relationship - - mapper_registry = registry() - - - @dataclass - class User: - id: int = field(init=False) - name: str = None - fullname: str = None - nickname: str = None - addresses: List[Address] = field(default_factory=list) - - - @dataclass - class Address: - id: int = field(init=False) - user_id: int = field(init=False) - email_address: str = None - - - metadata_obj = MetaData() - - user = Table( - "user", - metadata_obj, - Column("id", Integer, primary_key=True), - Column("name", String(50)), - Column("fullname", String(50)), - Column("nickname", String(12)), - ) - - address = Table( - "address", - metadata_obj, - Column("id", Integer, primary_key=True), - Column("user_id", Integer, ForeignKey("user.id")), - Column("email_address", String(50)), - ) - - mapper_registry.map_imperatively( - User, - user, - properties={ - "addresses": relationship(Address, backref="user", order_by=address.c.id), - }, - ) - - mapper_registry.map_imperatively(Address, address) - -.. _orm_declarative_attrs_imperative_table: - -Applying ORM mappings to an existing attrs class -------------------------------------------------- - -The attrs_ library is a popular third party library that provides similar -features as dataclasses, with many additional features provided not -found in ordinary dataclasses. - -A class augmented with attrs_ uses the ``@define`` decorator. This decorator -initiates a process to scan the class for attributes that define the class' -behavior, which are then used to generate methods, documentation, and -annotations. - -The SQLAlchemy ORM supports mapping an attrs_ class using **Declarative with -Imperative Table** or **Imperative** mapping. The general form of these two -styles is fully equivalent to the -:ref:`orm_declarative_dataclasses_declarative_table` and -:ref:`orm_declarative_dataclasses_imperative_table` mapping forms used with -dataclasses, where the inline attribute directives used by dataclasses or attrs -are unchanged, and SQLAlchemy's table-oriented instrumentation is applied at -runtime. - -The ``@define`` decorator of attrs_ by default replaces the annotated class -with a new __slots__ based class, which is not supported. When using the old -style annotation ``@attr.s`` or using ``define(slots=False)``, the class -does not get replaced. Furthermore attrs removes its own class-bound attributes -after the decorator runs, so that SQLAlchemy's mapping process takes over these -attributes without any issue. Both decorators, ``@attr.s`` and ``@define(slots=False)`` -work with SQLAlchemy. - -Mapping attrs with Declarative "Imperative Table" -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -In the "Declarative with Imperative Table" style, a :class:`_schema.Table` -object is declared inline with the declarative class. The -``@define`` decorator is applied to the class first, then the -:meth:`_orm.registry.mapped` decorator second:: - - from __future__ import annotations - - from typing import List - from typing import Optional - - from attrs import define - from sqlalchemy import Column - from sqlalchemy import ForeignKey - from sqlalchemy import Integer - from sqlalchemy import MetaData - from sqlalchemy import String - from sqlalchemy import Table - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import registry - from sqlalchemy.orm import relationship - - mapper_registry = registry() - - - @mapper_registry.mapped - @define(slots=False) - class User: - __table__ = Table( - "user", - mapper_registry.metadata, - Column("id", Integer, primary_key=True), - Column("name", String(50)), - Column("FullName", String(50), key="fullname"), - Column("nickname", String(12)), - ) - id: Mapped[int] - name: Mapped[str] - fullname: Mapped[str] - nickname: Mapped[str] - addresses: Mapped[List[Address]] - - __mapper_args__ = { # type: ignore - "properties": { - "addresses": relationship("Address"), - } - } - - - @mapper_registry.mapped - @define(slots=False) - class Address: - __table__ = Table( - "address", - mapper_registry.metadata, - Column("id", Integer, primary_key=True), - Column("user_id", Integer, ForeignKey("user.id")), - Column("email_address", String(50)), - ) - id: Mapped[int] - user_id: Mapped[int] - email_address: Mapped[Optional[str]] - -.. note:: The ``attrs`` ``slots=True`` option, which enables ``__slots__`` on - a mapped class, cannot be used with SQLAlchemy mappings without fully - implementing alternative - :ref:`attribute instrumentation `, as mapped - classes normally rely upon direct access to ``__dict__`` for state storage. - Behavior is undefined when this option is present. - - - -Mapping attrs with Imperative Mapping -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Just as is the case with dataclasses, we can make use of -:meth:`_orm.registry.map_imperatively` to map an existing ``attrs`` class -as well:: - - from __future__ import annotations - - from typing import List - - from attrs import define - from sqlalchemy import Column - from sqlalchemy import ForeignKey - from sqlalchemy import Integer - from sqlalchemy import MetaData - from sqlalchemy import String - from sqlalchemy import Table - from sqlalchemy.orm import registry - from sqlalchemy.orm import relationship - - mapper_registry = registry() - - - @define(slots=False) - class User: - id: int - name: str - fullname: str - nickname: str - addresses: List[Address] - - - @define(slots=False) - class Address: - id: int - user_id: int - email_address: Optional[str] - - - metadata_obj = MetaData() - - user = Table( - "user", - metadata_obj, - Column("id", Integer, primary_key=True), - Column("name", String(50)), - Column("fullname", String(50)), - Column("nickname", String(12)), - ) - - address = Table( - "address", - metadata_obj, - Column("id", Integer, primary_key=True), - Column("user_id", Integer, ForeignKey("user.id")), - Column("email_address", String(50)), - ) - - mapper_registry.map_imperatively( - User, - user, - properties={ - "addresses": relationship(Address, backref="user", order_by=address.c.id), - }, - ) - - mapper_registry.map_imperatively(Address, address) - -The above form is equivalent to the previous example using -Declarative with Imperative Table. - - - -.. _dataclass: https://docs.python.org/3/library/dataclasses.html -.. _dataclasses: https://docs.python.org/3/library/dataclasses.html -.. _attrs: https://pypi.org/project/attrs/ -.. _mypy: https://mypy.readthedocs.io/en/stable/ -.. _pyright: https://github.com/microsoft/pyright diff --git a/doc/build/orm/declarative_config.rst b/doc/build/orm/declarative_config.rst index 873f16aff35..99c7451ee84 100644 --- a/doc/build/orm/declarative_config.rst +++ b/doc/build/orm/declarative_config.rst @@ -84,7 +84,7 @@ SQL expression mapped by :func:`_orm.column_property`, and an additional "deferred" basis as defined by the :paramref:`_orm.mapped_column.deferred` keyword. More documentation on these particular concepts may be found at :ref:`relationship_patterns`, -:ref:`mapper_column_property_sql_expressions`, and :ref:`orm_queryguide_column_deferral`. +ref_mapper_column_property_sql_expressions, and :ref:`orm_queryguide_column_deferral`. Properties may be specified with a declarative mapping as above using "hybrid table" style as well; the :class:`_schema.Column` objects that @@ -218,7 +218,7 @@ an ORM-maintained version counter that is updated and checked within the .. seealso:: - :ref:`mapper_version_counter` - background on the ORM version counter feature + ref_mapper_version_counter - background on the ORM version counter feature **Single Table Inheritance** @@ -406,7 +406,7 @@ be illustrated using :meth:`_orm.registry.mapped` as follows:: ``__abstract__`` causes declarative to skip the production of a table or mapper for the class entirely. A class can be added within a -hierarchy in the same way as mixin (see :ref:`declarative_mixins`), allowing +hierarchy in the same way as mixin (see ref_declarative_mixins), allowing subclasses to extend just from the special class:: class SomeAbstractBase(Base): diff --git a/doc/build/orm/declarative_mixins.rst b/doc/build/orm/declarative_mixins.rst index cf721cb4a71..e46af29a9e4 100644 --- a/doc/build/orm/declarative_mixins.rst +++ b/doc/build/orm/declarative_mixins.rst @@ -141,7 +141,7 @@ attribute is used on the newly defined class. :func:`_orm.mapped_column`. .. versionchanged:: 2.0 For users coming from the 1.4 series of SQLAlchemy - who may have been using the :ref:`mypy plugin `, the + who may have been using the ref_mypy_toplevel, the :func:`_orm.declarative_mixin` class decorator is no longer needed to mark declarative mixins, assuming the mypy plugin is no longer in use. diff --git a/doc/build/orm/declarative_tables.rst b/doc/build/orm/declarative_tables.rst index 0ee40cd07f9..abd0219502f 100644 --- a/doc/build/orm/declarative_tables.rst +++ b/doc/build/orm/declarative_tables.rst @@ -130,7 +130,7 @@ itself (more on this at :ref:`mapper_column_distinct_names`). .. seealso:: - :ref:`mapping_columns_toplevel` - contains additional notes on affecting + ref_mapping_columns_toplevel - contains additional notes on affecting how :class:`_orm.Mapper` interprets incoming :class:`.Column` objects. .. _orm_declarative_mapped_column: @@ -890,7 +890,7 @@ Dataclass features in ``mapped_column()`` The :func:`_orm.mapped_column` construct integrates with SQLAlchemy's "native dataclasses" feature, discussed at -:ref:`orm_declarative_native_dataclasses`. See that section for current +ref_orm_declarative_native_dataclasses. See that section for current background on additional directives supported by :func:`_orm.mapped_column`. @@ -911,7 +911,7 @@ available via the ``__table__`` attribute:: The above table is ultimately the same one that corresponds to the :attr:`_orm.Mapper.local_table` attribute, which we can see through the -:ref:`runtime inspection system `:: +ref_inspection_toplevel:: from sqlalchemy import inspect @@ -1268,13 +1268,13 @@ the sections :ref:`orm_mapping_joins` and :ref:`orm_mapping_arbitrary_subqueries The "imperative table" form is of particular use when the class itself is using an alternative form of attribute declaration, such as Python -dataclasses. See the section :ref:`orm_declarative_dataclasses` for detail. +dataclasses. See the section ref_orm_declarative_dataclasses for detail. .. seealso:: :ref:`metadata_describing` - :ref:`orm_declarative_dataclasses` + ref_orm_declarative_dataclasses .. _orm_imperative_table_column_naming: @@ -1407,7 +1407,7 @@ associate additional parameters with the column. Options include: * :ref:`maptojoin` - * :ref:`mapper_sql_expressions` + * ref_mapper_sql_expressions For Declarative Table configuration with :func:`_orm.mapped_column`, most options are available directly; see the section @@ -1423,7 +1423,7 @@ Mapping Declaratively with Reflected Tables There are several patterns available which provide for producing mapped classes against a series of :class:`_schema.Table` objects that were introspected from the database, using the reflection process described at -:ref:`metadata_reflection`. +ref_metadata_reflection. A simple way to map a class to a table reflected from the database is to use a declarative hybrid mapping, passing the @@ -1535,7 +1535,7 @@ Using Automap ^^^^^^^^^^^^^^ A more automated solution to mapping against an existing database where table -reflection is to be used is to use the :ref:`automap_toplevel` extension. This +reflection is to be used is to use the ref_automap_toplevel extension. This extension will generate entire mapped classes from a database schema, including relationships between classes based on observed foreign key constraints. While it includes hooks for customization, such as hooks that allow custom @@ -1547,7 +1547,7 @@ class may be preferable for its less automated approach. .. seealso:: - :ref:`automap_toplevel` + ref_automap_toplevel .. _mapper_automated_reflection_schemes: @@ -1596,8 +1596,8 @@ with our event that adds a new ".key" element, such as in a mapping as below:: __table__ = Table("some_table", Base.metadata, autoload_with=some_engine) The approach also works with both the :class:`.DeferredReflection` base class -as well as with the :ref:`automap_toplevel` extension. For automap -specifically, see the section :ref:`automap_intercepting_columns` for +as well as with the ref_automap_toplevel extension. For automap +specifically, see the section ref_automap_intercepting_columns for background. .. seealso:: @@ -1606,7 +1606,7 @@ background. :meth:`_events.DDLEvents.column_reflect` - :ref:`automap_intercepting_columns` - in the :ref:`automap_toplevel` documentation + ref_automap_intercepting_columns - in the ref_automap_toplevel documentation .. _mapper_primary_key: @@ -1723,7 +1723,7 @@ still be in effect** for those :class:`_schema.Column` objects that include them even though they may be excluded from the ORM mapping. "Schema level column defaults" refers to the defaults described at -:ref:`metadata_defaults` including those configured by the +ref_metadata_defaults including those configured by the :paramref:`_schema.Column.default`, :paramref:`_schema.Column.onupdate`, :paramref:`_schema.Column.server_default` and :paramref:`_schema.Column.server_onupdate` parameters. These constructs diff --git a/doc/build/orm/events.rst b/doc/build/orm/events.rst index 1db1137e085..19ad5570309 100644 --- a/doc/build/orm/events.rst +++ b/doc/build/orm/events.rst @@ -91,7 +91,7 @@ Attribute Events Attribute events are triggered as things occur on individual attributes of ORM mapped objects. These events form the basis for things like :ref:`custom validation functions ` as well as -:ref:`backref handlers `. +ref_relationships_backref. .. seealso:: diff --git a/doc/build/orm/examples.rst b/doc/build/orm/examples.rst deleted file mode 100644 index 9e38768b329..00000000000 --- a/doc/build/orm/examples.rst +++ /dev/null @@ -1,154 +0,0 @@ -.. _examples_toplevel: - -============ -ORM Examples -============ - -The SQLAlchemy distribution includes a variety of code examples illustrating -a select set of patterns, some typical and some not so typical. All are -runnable and can be found in the ``/examples`` directory of the -distribution. Descriptions and source code for all can be found here. - -Additional SQLAlchemy examples, some user contributed, are available on the -wiki at ``_. - - -Mapping Recipes -=============== - -.. _examples_adjacencylist: - -Adjacency List --------------- - -.. automodule:: examples.adjacency_list - -.. _examples_associations: - -Associations ------------- - -.. automodule:: examples.association - -.. _examples_asyncio: - -Asyncio Integration -------------------- - -.. automodule:: examples.asyncio - -Directed Graphs ---------------- - -.. automodule:: examples.graphs - -Dynamic Relations as Dictionaries ---------------------------------- - -.. automodule:: examples.dynamic_dict - -.. _examples_generic_associations: - -Generic Associations --------------------- - -.. automodule:: examples.generic_associations - - -Materialized Paths ------------------- - -.. automodule:: examples.materialized_paths - -Nested Sets ------------ - -.. automodule:: examples.nested_sets - -.. _examples_performance: - -Performance ------------ - -.. automodule:: examples.performance - - -.. _examples_spaceinvaders: - -Space Invaders --------------- - -.. automodule:: examples.space_invaders - - -.. _examples_versioning: - -Versioning Objects ------------------- - -.. _examples_versioned_history: - -Versioning with a History Table -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. automodule:: examples.versioned_history - -.. _examples_versioned_rows: - -Versioning using Temporal Rows -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. automodule:: examples.versioned_rows - -.. _examples_vertical_tables: - -Vertical Attribute Mapping --------------------------- - -.. automodule:: examples.vertical - - -.. _examples_inheritance: - -Inheritance Mapping Recipes -=========================== - -Basic Inheritance Mappings --------------------------- - -.. automodule:: examples.inheritance - -Special APIs -============ - -.. _examples_instrumentation: - -Attribute Instrumentation -------------------------- - -.. automodule:: examples.custom_attributes - -.. _examples_sharding: - -Horizontal Sharding -------------------- - -.. automodule:: examples.sharding - -Extending the ORM -================= - -.. _examples_session_orm_events: - -ORM Query Events ------------------ - -.. automodule:: examples.extending_query - -.. _examples_caching: - -Dogpile Caching ---------------- - -.. automodule:: examples.dogpile_caching - diff --git a/doc/build/orm/extensions/associationproxy.rst b/doc/build/orm/extensions/associationproxy.rst deleted file mode 100644 index 036969f37b3..00000000000 --- a/doc/build/orm/extensions/associationproxy.rst +++ /dev/null @@ -1,716 +0,0 @@ -.. _associationproxy_toplevel: - -Association Proxy -================= - -.. module:: sqlalchemy.ext.associationproxy - -``associationproxy`` is used to create a read/write view of a -target attribute across a relationship. It essentially conceals -the usage of a "middle" attribute between two endpoints, and -can be used to cherry-pick fields from a collection of -related objects or to reduce the verbosity of using the association -object pattern. Applied creatively, the association proxy allows -the construction of sophisticated collections and dictionary -views of virtually any geometry, persisted to the database using -standard, transparently configured relational patterns. - -.. _associationproxy_scalar_collections: - -Simplifying Scalar Collections ------------------------------- - -Consider a many-to-many mapping between two classes, ``User`` and ``Keyword``. -Each ``User`` can have any number of ``Keyword`` objects, and vice-versa -(the many-to-many pattern is described at :ref:`relationships_many_to_many`). -The example below illustrates this pattern in the same way, with the -exception of an extra attribute added to the ``User`` class called -``User.keywords``:: - - from __future__ import annotations - - from typing import Final - - from sqlalchemy import Column - from sqlalchemy import ForeignKey - from sqlalchemy import Integer - from sqlalchemy import String - from sqlalchemy import Table - from sqlalchemy.orm import DeclarativeBase - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import relationship - from sqlalchemy.ext.associationproxy import association_proxy - from sqlalchemy.ext.associationproxy import AssociationProxy - - - class Base(DeclarativeBase): - pass - - - class User(Base): - __tablename__ = "user" - id: Mapped[int] = mapped_column(primary_key=True) - name: Mapped[str] = mapped_column(String(64)) - kw: Mapped[List[Keyword]] = relationship(secondary=lambda: user_keyword_table) - - def __init__(self, name: str): - self.name = name - - # proxy the 'keyword' attribute from the 'kw' relationship - keywords: AssociationProxy[List[str]] = association_proxy("kw", "keyword") - - - class Keyword(Base): - __tablename__ = "keyword" - id: Mapped[int] = mapped_column(primary_key=True) - keyword: Mapped[str] = mapped_column(String(64)) - - def __init__(self, keyword: str): - self.keyword = keyword - - - user_keyword_table: Final[Table] = Table( - "user_keyword", - Base.metadata, - Column("user_id", Integer, ForeignKey("user.id"), primary_key=True), - Column("keyword_id", Integer, ForeignKey("keyword.id"), primary_key=True), - ) - -In the above example, :func:`.association_proxy` is applied to the ``User`` -class to produce a "view" of the ``kw`` relationship, which exposes the string -value of ``.keyword`` associated with each ``Keyword`` object. It also -creates new ``Keyword`` objects transparently when strings are added to the -collection:: - - >>> user = User("jek") - >>> user.keywords.append("cheese-inspector") - >>> user.keywords.append("snack-ninja") - >>> print(user.keywords) - ['cheese-inspector', 'snack-ninja'] - -To understand the mechanics of this, first review the behavior of -``User`` and ``Keyword`` without using the ``.keywords`` association proxy. -Normally, reading and manipulating the collection of "keyword" strings associated -with ``User`` requires traversal from each collection element to the ``.keyword`` -attribute, which can be awkward. The example below illustrates the identical -series of operations applied without using the association proxy:: - - >>> # identical operations without using the association proxy - >>> user = User("jek") - >>> user.kw.append(Keyword("cheese-inspector")) - >>> user.kw.append(Keyword("snack-ninja")) - >>> print([keyword.keyword for keyword in user.kw]) - ['cheese-inspector', 'snack-ninja'] - -The :class:`.AssociationProxy` object produced by the :func:`.association_proxy` function -is an instance of a `Python descriptor `_, -and is not considered to be "mapped" by the :class:`.Mapper` in any way. Therefore, -it's always indicated inline within the class definition of the mapped class, -regardless of whether Declarative or Imperative mappings are used. - -The proxy functions by operating upon the underlying mapped attribute -or collection in response to operations, and changes made via the proxy are immediately -apparent in the mapped attribute, as well as vice versa. The underlying -attribute remains fully accessible. - -When first accessed, the association proxy performs introspection -operations on the target collection so that its behavior corresponds correctly. -Details such as if the locally proxied attribute is a collection (as is typical) -or a scalar reference, as well as if the collection acts like a set, list, -or dictionary is taken into account, so that the proxy should act just like -the underlying collection or attribute does. - -.. _associationproxy_creator: - -Creation of New Values ----------------------- - -When a list ``append()`` event (or set ``add()``, dictionary ``__setitem__()``, -or scalar assignment event) is intercepted by the association proxy, it -instantiates a new instance of the "intermediary" object using its constructor, -passing as a single argument the given value. In our example above, an -operation like:: - - user.keywords.append("cheese-inspector") - -Is translated by the association proxy into the operation:: - - user.kw.append(Keyword("cheese-inspector")) - -The example works here because we have designed the constructor for ``Keyword`` -to accept a single positional argument, ``keyword``. For those cases where a -single-argument constructor isn't feasible, the association proxy's creational -behavior can be customized using the :paramref:`.association_proxy.creator` -argument, which references a callable (i.e. Python function) that will produce -a new object instance given the singular argument. Below we illustrate this -using a lambda as is typical:: - - class User(Base): - ... - - # use Keyword(keyword=kw) on append() events - keywords: AssociationProxy[List[str]] = association_proxy( - "kw", "keyword", creator=lambda kw: Keyword(keyword=kw) - ) - -The ``creator`` function accepts a single argument in the case of a list- -or set- based collection, or a scalar attribute. In the case of a dictionary-based -collection, it accepts two arguments, "key" and "value". An example -of this is below in :ref:`proxying_dictionaries`. - -Simplifying Association Objects -------------------------------- - -The "association object" pattern is an extended form of a many-to-many -relationship, and is described at :ref:`association_pattern`. Association -proxies are useful for keeping "association objects" out of the way during -regular use. - -Suppose our ``user_keyword`` table above had additional columns -which we'd like to map explicitly, but in most cases we don't -require direct access to these attributes. Below, we illustrate -a new mapping which introduces the ``UserKeywordAssociation`` class, which -is mapped to the ``user_keyword`` table illustrated earlier. -This class adds an additional column ``special_key``, a value which -we occasionally want to access, but not in the usual case. We -create an association proxy on the ``User`` class called -``keywords``, which will bridge the gap from the ``user_keyword_associations`` -collection of ``User`` to the ``.keyword`` attribute present on each -``UserKeywordAssociation``:: - - from __future__ import annotations - - from typing import List - from typing import Optional - - from sqlalchemy import ForeignKey - from sqlalchemy import String - from sqlalchemy.ext.associationproxy import association_proxy - from sqlalchemy.ext.associationproxy import AssociationProxy - from sqlalchemy.orm import DeclarativeBase - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import relationship - - - class Base(DeclarativeBase): - pass - - - class User(Base): - __tablename__ = "user" - - id: Mapped[int] = mapped_column(primary_key=True) - name: Mapped[str] = mapped_column(String(64)) - - user_keyword_associations: Mapped[List[UserKeywordAssociation]] = relationship( - back_populates="user", - cascade="all, delete-orphan", - ) - - # association proxy of "user_keyword_associations" collection - # to "keyword" attribute - keywords: AssociationProxy[List[Keyword]] = association_proxy( - "user_keyword_associations", - "keyword", - creator=lambda keyword_obj: UserKeywordAssociation(keyword=keyword_obj), - ) - - def __init__(self, name: str): - self.name = name - - - class UserKeywordAssociation(Base): - __tablename__ = "user_keyword" - user_id: Mapped[int] = mapped_column(ForeignKey("user.id"), primary_key=True) - keyword_id: Mapped[int] = mapped_column(ForeignKey("keyword.id"), primary_key=True) - special_key: Mapped[Optional[str]] = mapped_column(String(50)) - - user: Mapped[User] = relationship(back_populates="user_keyword_associations") - - keyword: Mapped[Keyword] = relationship() - - - class Keyword(Base): - __tablename__ = "keyword" - id: Mapped[int] = mapped_column(primary_key=True) - keyword: Mapped[str] = mapped_column("keyword", String(64)) - - def __init__(self, keyword: str): - self.keyword = keyword - - def __repr__(self) -> str: - return f"Keyword({self.keyword!r})" - -With the above configuration, we can operate upon the ``.keywords`` collection -of each ``User`` object, each of which exposes a collection of ``Keyword`` -objects that are obtained from the underlying ``UserKeywordAssociation`` elements:: - - >>> user = User("log") - >>> for kw in (Keyword("new_from_blammo"), Keyword("its_big")): - ... user.keywords.append(kw) - >>> print(user.keywords) - [Keyword('new_from_blammo'), Keyword('its_big')] - -This example is in contrast to the example illustrated previously at -:ref:`associationproxy_scalar_collections`, where the association proxy exposed -a collection of strings, rather than a collection of composed objects. -In this case, each ``.keywords.append()`` operation is equivalent to:: - - >>> user.user_keyword_associations.append( - ... UserKeywordAssociation(keyword=Keyword("its_heavy")) - ... ) - -The ``UserKeywordAssociation`` object has two attributes that are both -populated within the scope of the ``append()`` operation of the association -proxy; ``.keyword``, which refers to the -``Keyword`` object, and ``.user``, which refers to the ``User`` object. -The ``.keyword`` attribute is populated first, as the association proxy -generates a new ``UserKeywordAssociation`` object in response to the ``.append()`` -operation, assigning the given ``Keyword`` instance to the ``.keyword`` -attribute. Then, as the ``UserKeywordAssociation`` object is appended to the -``User.user_keyword_associations`` collection, the ``UserKeywordAssociation.user`` attribute, -configured as ``back_populates`` for ``User.user_keyword_associations``, is initialized -upon the given ``UserKeywordAssociation`` instance to refer to the parent ``User`` -receiving the append operation. The ``special_key`` -argument above is left at its default value of ``None``. - -For those cases where we do want ``special_key`` to have a value, we -create the ``UserKeywordAssociation`` object explicitly. Below we assign all -three attributes, wherein the assignment of ``.user`` during -construction, has the effect of appending the new ``UserKeywordAssociation`` to -the ``User.user_keyword_associations`` collection (via the relationship):: - - >>> UserKeywordAssociation( - ... keyword=Keyword("its_wood"), user=user, special_key="my special key" - ... ) - -The association proxy returns to us a collection of ``Keyword`` objects represented -by all these operations:: - - >>> print(user.keywords) - [Keyword('new_from_blammo'), Keyword('its_big'), Keyword('its_heavy'), Keyword('its_wood')] - -.. _proxying_dictionaries: - -Proxying to Dictionary Based Collections ----------------------------------------- - -The association proxy can proxy to dictionary based collections as well. SQLAlchemy -mappings usually use the :func:`.attribute_keyed_dict` collection type to -create dictionary collections, as well as the extended techniques described in -:ref:`dictionary_collections`. - -The association proxy adjusts its behavior when it detects the usage of a -dictionary-based collection. When new values are added to the dictionary, the -association proxy instantiates the intermediary object by passing two -arguments to the creation function instead of one, the key and the value. As -always, this creation function defaults to the constructor of the intermediary -class, and can be customized using the ``creator`` argument. - -Below, we modify our ``UserKeywordAssociation`` example such that the ``User.user_keyword_associations`` -collection will now be mapped using a dictionary, where the ``UserKeywordAssociation.special_key`` -argument will be used as the key for the dictionary. We also apply a ``creator`` -argument to the ``User.keywords`` proxy so that these values are assigned appropriately -when new elements are added to the dictionary:: - - from __future__ import annotations - from typing import Dict - - from sqlalchemy import ForeignKey - from sqlalchemy import String - from sqlalchemy.ext.associationproxy import association_proxy - from sqlalchemy.ext.associationproxy import AssociationProxy - from sqlalchemy.orm import DeclarativeBase - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import relationship - from sqlalchemy.orm.collections import attribute_keyed_dict - - - class Base(DeclarativeBase): - pass - - - class User(Base): - __tablename__ = "user" - id: Mapped[int] = mapped_column(primary_key=True) - name: Mapped[str] = mapped_column(String(64)) - - # user/user_keyword_associations relationship, mapping - # user_keyword_associations with a dictionary against "special_key" as key. - user_keyword_associations: Mapped[Dict[str, UserKeywordAssociation]] = relationship( - back_populates="user", - collection_class=attribute_keyed_dict("special_key"), - cascade="all, delete-orphan", - ) - # proxy to 'user_keyword_associations', instantiating - # UserKeywordAssociation assigning the new key to 'special_key', - # values to 'keyword'. - keywords: AssociationProxy[Dict[str, Keyword]] = association_proxy( - "user_keyword_associations", - "keyword", - creator=lambda k, v: UserKeywordAssociation(special_key=k, keyword=v), - ) - - def __init__(self, name: str): - self.name = name - - - class UserKeywordAssociation(Base): - __tablename__ = "user_keyword" - user_id: Mapped[int] = mapped_column(ForeignKey("user.id"), primary_key=True) - keyword_id: Mapped[int] = mapped_column(ForeignKey("keyword.id"), primary_key=True) - special_key: Mapped[str] - - user: Mapped[User] = relationship( - back_populates="user_keyword_associations", - ) - keyword: Mapped[Keyword] = relationship() - - - class Keyword(Base): - __tablename__ = "keyword" - id: Mapped[int] = mapped_column(primary_key=True) - keyword: Mapped[str] = mapped_column(String(64)) - - def __init__(self, keyword: str): - self.keyword = keyword - - def __repr__(self) -> str: - return f"Keyword({self.keyword!r})" - -We illustrate the ``.keywords`` collection as a dictionary, mapping the -``UserKeywordAssociation.special_key`` value to ``Keyword`` objects:: - - >>> user = User("log") - - >>> user.keywords["sk1"] = Keyword("kw1") - >>> user.keywords["sk2"] = Keyword("kw2") - - >>> print(user.keywords) - {'sk1': Keyword('kw1'), 'sk2': Keyword('kw2')} - -.. _composite_association_proxy: - -Composite Association Proxies ------------------------------ - -Given our previous examples of proxying from relationship to scalar -attribute, proxying across an association object, and proxying dictionaries, -we can combine all three techniques together to give ``User`` -a ``keywords`` dictionary that deals strictly with the string value -of ``special_key`` mapped to the string ``keyword``. Both the ``UserKeywordAssociation`` -and ``Keyword`` classes are entirely concealed. This is achieved by building -an association proxy on ``User`` that refers to an association proxy -present on ``UserKeywordAssociation``:: - - from __future__ import annotations - - from sqlalchemy import ForeignKey - from sqlalchemy import String - from sqlalchemy.ext.associationproxy import association_proxy - from sqlalchemy.ext.associationproxy import AssociationProxy - from sqlalchemy.orm import DeclarativeBase - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import relationship - from sqlalchemy.orm.collections import attribute_keyed_dict - - - class Base(DeclarativeBase): - pass - - - class User(Base): - __tablename__ = "user" - id: Mapped[int] = mapped_column(primary_key=True) - name: Mapped[str] = mapped_column(String(64)) - - user_keyword_associations: Mapped[Dict[str, UserKeywordAssociation]] = relationship( - back_populates="user", - collection_class=attribute_keyed_dict("special_key"), - cascade="all, delete-orphan", - ) - # the same 'user_keyword_associations'->'keyword' proxy as in - # the basic dictionary example. - keywords: AssociationProxy[Dict[str, str]] = association_proxy( - "user_keyword_associations", - "keyword", - creator=lambda k, v: UserKeywordAssociation(special_key=k, keyword=v), - ) - - def __init__(self, name: str): - self.name = name - - - class UserKeywordAssociation(Base): - __tablename__ = "user_keyword" - user_id: Mapped[int] = mapped_column(ForeignKey("user.id"), primary_key=True) - keyword_id: Mapped[int] = mapped_column(ForeignKey("keyword.id"), primary_key=True) - special_key: Mapped[str] = mapped_column(String(64)) - user: Mapped[User] = relationship( - back_populates="user_keyword_associations", - ) - - # the relationship to Keyword is now called - # 'kw' - kw: Mapped[Keyword] = relationship() - - # 'keyword' is changed to be a proxy to the - # 'keyword' attribute of 'Keyword' - keyword: AssociationProxy[Dict[str, str]] = association_proxy("kw", "keyword") - - - class Keyword(Base): - __tablename__ = "keyword" - id: Mapped[int] = mapped_column(primary_key=True) - keyword: Mapped[str] = mapped_column(String(64)) - - def __init__(self, keyword: str): - self.keyword = keyword - -``User.keywords`` is now a dictionary of string to string, where -``UserKeywordAssociation`` and ``Keyword`` objects are created and removed for us -transparently using the association proxy. In the example below, we illustrate -usage of the assignment operator, also appropriately handled by the -association proxy, to apply a dictionary value to the collection at once:: - - >>> user = User("log") - >>> user.keywords = {"sk1": "kw1", "sk2": "kw2"} - >>> print(user.keywords) - {'sk1': 'kw1', 'sk2': 'kw2'} - - >>> user.keywords["sk3"] = "kw3" - >>> del user.keywords["sk2"] - >>> print(user.keywords) - {'sk1': 'kw1', 'sk3': 'kw3'} - - >>> # illustrate un-proxied usage - ... print(user.user_keyword_associations["sk3"].kw) - <__main__.Keyword object at 0x12ceb90> - -One caveat with our example above is that because ``Keyword`` objects are created -for each dictionary set operation, the example fails to maintain uniqueness for -the ``Keyword`` objects on their string name, which is a typical requirement for -a tagging scenario such as this one. For this use case the recipe -`UniqueObject `_, or -a comparable creational strategy, is -recommended, which will apply a "lookup first, then create" strategy to the constructor -of the ``Keyword`` class, so that an already existing ``Keyword`` is returned if the -given name is already present. - -Querying with Association Proxies ---------------------------------- - -The :class:`.AssociationProxy` features simple SQL construction capabilities -which work at the class level in a similar way as other ORM-mapped attributes, -and provide rudimentary filtering support primarily based on the -SQL ``EXISTS`` keyword. - - -.. note:: The primary purpose of the association proxy extension is to allow - for improved persistence and object-access patterns with mapped object - instances that are already loaded. The class-bound querying feature - is of limited use and will not replace the need to refer to the underlying - attributes when constructing SQL queries with JOINs, eager loading - options, etc. - -For this section, assume a class with both an association proxy -that refers to a column, as well as an association proxy that refers -to a related object, as in the example mapping below:: - - from __future__ import annotations - from sqlalchemy import Column, ForeignKey, Integer, String - from sqlalchemy.ext.associationproxy import association_proxy, AssociationProxy - from sqlalchemy.orm import DeclarativeBase, relationship - from sqlalchemy.orm.collections import attribute_keyed_dict - from sqlalchemy.orm.collections import Mapped - - - class Base(DeclarativeBase): - pass - - - class User(Base): - __tablename__ = "user" - id: Mapped[int] = mapped_column(primary_key=True) - name: Mapped[str] = mapped_column(String(64)) - - user_keyword_associations: Mapped[UserKeywordAssociation] = relationship( - cascade="all, delete-orphan", - ) - - # object-targeted association proxy - keywords: AssociationProxy[List[Keyword]] = association_proxy( - "user_keyword_associations", - "keyword", - ) - - # column-targeted association proxy - special_keys: AssociationProxy[List[str]] = association_proxy( - "user_keyword_associations", "special_key" - ) - - - class UserKeywordAssociation(Base): - __tablename__ = "user_keyword" - user_id: Mapped[int] = mapped_column(ForeignKey("user.id"), primary_key=True) - keyword_id: Mapped[int] = mapped_column(ForeignKey("keyword.id"), primary_key=True) - special_key: Mapped[str] = mapped_column(String(64)) - keyword: Mapped[Keyword] = relationship() - - - class Keyword(Base): - __tablename__ = "keyword" - id: Mapped[int] = mapped_column(primary_key=True) - keyword: Mapped[str] = mapped_column(String(64)) - -The SQL generated takes the form of a correlated subquery against -the EXISTS SQL operator so that it can be used in a WHERE clause without -the need for additional modifications to the enclosing query. If the -immediate target of an association proxy is a **mapped column expression**, -standard column operators can be used which will be embedded in the subquery. -For example a straight equality operator: - -.. sourcecode:: pycon+sql - - >>> print(session.scalars(select(User).where(User.special_keys == "jek"))) - {printsql}SELECT "user".id AS user_id, "user".name AS user_name - FROM "user" - WHERE EXISTS (SELECT 1 - FROM user_keyword - WHERE "user".id = user_keyword.user_id AND user_keyword.special_key = :special_key_1) - -a LIKE operator: - -.. sourcecode:: pycon+sql - - >>> print(session.scalars(select(User).where(User.special_keys.like("%jek")))) - {printsql}SELECT "user".id AS user_id, "user".name AS user_name - FROM "user" - WHERE EXISTS (SELECT 1 - FROM user_keyword - WHERE "user".id = user_keyword.user_id AND user_keyword.special_key LIKE :special_key_1) - -For association proxies where the immediate target is a **related object or collection, -or another association proxy or attribute on the related object**, relationship-oriented -operators can be used instead, such as :meth:`_orm.PropComparator.has` and -:meth:`_orm.PropComparator.any`. The ``User.keywords`` attribute is in fact -two association proxies linked together, so when using this proxy for generating -SQL phrases, we get two levels of EXISTS subqueries: - -.. sourcecode:: pycon+sql - - >>> print(session.scalars(select(User).where(User.keywords.any(Keyword.keyword == "jek")))) - {printsql}SELECT "user".id AS user_id, "user".name AS user_name - FROM "user" - WHERE EXISTS (SELECT 1 - FROM user_keyword - WHERE "user".id = user_keyword.user_id AND (EXISTS (SELECT 1 - FROM keyword - WHERE keyword.id = user_keyword.keyword_id AND keyword.keyword = :keyword_1))) - -This is not the most efficient form of SQL, so while association proxies can be -convenient for generating WHERE criteria quickly, SQL results should be -inspected and "unrolled" into explicit JOIN criteria for best use, especially -when chaining association proxies together. - - -.. versionchanged:: 1.3 Association proxy features distinct querying modes - based on the type of target. See :ref:`change_4351`. - - - -.. _cascade_scalar_deletes: - -Cascading Scalar Deletes ------------------------- - -.. versionadded:: 1.3 - -Given a mapping as:: - - from __future__ import annotations - from sqlalchemy import Column, ForeignKey, Integer, String - from sqlalchemy.ext.associationproxy import association_proxy, AssociationProxy - from sqlalchemy.orm import DeclarativeBase, relationship - from sqlalchemy.orm.collections import attribute_keyed_dict - from sqlalchemy.orm.collections import Mapped - - - class Base(DeclarativeBase): - pass - - - class A(Base): - __tablename__ = "test_a" - id: Mapped[int] = mapped_column(primary_key=True) - ab: Mapped[AB] = relationship(uselist=False) - b: AssociationProxy[B] = association_proxy( - "ab", "b", creator=lambda b: AB(b=b), cascade_scalar_deletes=True - ) - - - class B(Base): - __tablename__ = "test_b" - id: Mapped[int] = mapped_column(primary_key=True) - - - class AB(Base): - __tablename__ = "test_ab" - a_id: Mapped[int] = mapped_column(ForeignKey(A.id), primary_key=True) - b_id: Mapped[int] = mapped_column(ForeignKey(B.id), primary_key=True) - - b: Mapped[B] = relationship() - -An assignment to ``A.b`` will generate an ``AB`` object:: - - a.b = B() - -The ``A.b`` association is scalar, and includes use of the parameter -:paramref:`.AssociationProxy.cascade_scalar_deletes`. When this parameter -is enabled, setting ``A.b`` -to ``None`` will remove ``A.ab`` as well:: - - a.b = None - assert a.ab is None - -When :paramref:`.AssociationProxy.cascade_scalar_deletes` is not set, -the association object ``a.ab`` above would remain in place. - -Note that this is not the behavior for collection-based association proxies; -in that case, the intermediary association object is always removed when -members of the proxied collection are removed. Whether or not the row is -deleted depends on the relationship cascade setting. - -.. seealso:: - - :ref:`unitofwork_cascades` - -API Documentation ------------------ - -.. autofunction:: association_proxy - -.. autoclass:: AssociationProxy - :members: - :undoc-members: - :inherited-members: - -.. autoclass:: AssociationProxyInstance - :members: - :undoc-members: - :inherited-members: - -.. autoclass:: ObjectAssociationProxyInstance - :members: - :inherited-members: - -.. autoclass:: ColumnAssociationProxyInstance - :members: - :inherited-members: - -.. autoclass:: AssociationProxyExtensionType - :members: diff --git a/doc/build/orm/extensions/asyncio.rst b/doc/build/orm/extensions/asyncio.rst deleted file mode 100644 index c57f1199cdb..00000000000 --- a/doc/build/orm/extensions/asyncio.rst +++ /dev/null @@ -1,1032 +0,0 @@ -.. _asyncio_toplevel: - -Asynchronous I/O (asyncio) -========================== - -Support for Python asyncio. Support for Core and ORM usage is -included, using asyncio-compatible dialects. - -.. versionadded:: 1.4 - -.. warning:: Please read :ref:`asyncio_install` for important platform - installation notes for many platforms, including **Apple M1 Architecture**. - -.. seealso:: - - :ref:`change_3414` - initial feature announcement - - :ref:`examples_asyncio` - example scripts illustrating working examples - of Core and ORM use within the asyncio extension. - -.. _asyncio_install: - -Asyncio Platform Installation Notes (Including Apple M1) ---------------------------------------------------------- - -The asyncio extension requires Python 3 only. It also depends -upon the `greenlet `_ library. This -dependency is installed by default on common machine platforms including: - -.. sourcecode:: text - - x86_64 aarch64 ppc64le amd64 win32 - -For the above platforms, ``greenlet`` is known to supply pre-built wheel files. -For other platforms, **greenlet does not install by default**; -the current file listing for greenlet can be seen at -`Greenlet - Download Files `_. -Note that **there are many architectures omitted, including Apple M1**. - -To install SQLAlchemy while ensuring the ``greenlet`` dependency is present -regardless of what platform is in use, the -``[asyncio]`` `setuptools extra `_ -may be installed -as follows, which will include also instruct ``pip`` to install ``greenlet``: - -.. sourcecode:: text - - pip install sqlalchemy[asyncio] - -Note that installation of ``greenlet`` on platforms that do not have a pre-built -wheel file means that ``greenlet`` will be built from source, which requires -that Python's development libraries also be present. - - -Synopsis - Core ---------------- - -For Core use, the :func:`_asyncio.create_async_engine` function creates an -instance of :class:`_asyncio.AsyncEngine` which then offers an async version of -the traditional :class:`_engine.Engine` API. The -:class:`_asyncio.AsyncEngine` delivers an :class:`_asyncio.AsyncConnection` via -its :meth:`_asyncio.AsyncEngine.connect` and :meth:`_asyncio.AsyncEngine.begin` -methods which both deliver asynchronous context managers. The -:class:`_asyncio.AsyncConnection` can then invoke statements using either the -:meth:`_asyncio.AsyncConnection.execute` method to deliver a buffered -:class:`_engine.Result`, or the :meth:`_asyncio.AsyncConnection.stream` method -to deliver a streaming server-side :class:`_asyncio.AsyncResult`:: - - import asyncio - - from sqlalchemy import Column - from sqlalchemy import MetaData - from sqlalchemy import select - from sqlalchemy import String - from sqlalchemy import Table - from sqlalchemy.ext.asyncio import create_async_engine - - meta = MetaData() - t1 = Table("t1", meta, Column("name", String(50), primary_key=True)) - - - async def async_main() -> None: - engine = create_async_engine( - "postgresql+asyncpg://scott:tiger@localhost/test", - echo=True, - ) - - async with engine.begin() as conn: - await conn.run_sync(meta.create_all) - - await conn.execute( - t1.insert(), [{"name": "some name 1"}, {"name": "some name 2"}] - ) - - async with engine.connect() as conn: - # select a Result, which will be delivered with buffered - # results - result = await conn.execute(select(t1).where(t1.c.name == "some name 1")) - - print(result.fetchall()) - - # for AsyncEngine created in function scope, close and - # clean-up pooled connections - await engine.dispose() - - - asyncio.run(async_main()) - -Above, the :meth:`_asyncio.AsyncConnection.run_sync` method may be used to -invoke special DDL functions such as :meth:`_schema.MetaData.create_all` that -don't include an awaitable hook. - -.. tip:: It's advisable to invoke the :meth:`_asyncio.AsyncEngine.dispose` method - using ``await`` when using the :class:`_asyncio.AsyncEngine` object in a - scope that will go out of context and be garbage collected, as illustrated in the - ``async_main`` function in the above example. This ensures that any - connections held open by the connection pool will be properly disposed - within an awaitable context. Unlike when using blocking IO, SQLAlchemy - cannot properly dispose of these connections within methods like ``__del__`` - or weakref finalizers as there is no opportunity to invoke ``await``. - Failing to explicitly dispose of the engine when it falls out of scope - may result in warnings emitted to standard out resembling the form - ``RuntimeError: Event loop is closed`` within garbage collection. - -The :class:`_asyncio.AsyncConnection` also features a "streaming" API via -the :meth:`_asyncio.AsyncConnection.stream` method that returns an -:class:`_asyncio.AsyncResult` object. This result object uses a server-side -cursor and provides an async/await API, such as an async iterator:: - - async with engine.connect() as conn: - async_result = await conn.stream(select(t1)) - - async for row in async_result: - print("row: %s" % (row,)) - -.. _asyncio_orm: - - -Synopsis - ORM ---------------- - -Using :term:`2.0 style` querying, the :class:`_asyncio.AsyncSession` class -provides full ORM functionality. Within the default mode of use, special care -must be taken to avoid :term:`lazy loading` or other expired-attribute access -involving ORM relationships and column attributes; the next -section :ref:`asyncio_orm_avoid_lazyloads` details this. The example below -illustrates a complete example including mapper and session configuration:: - - from __future__ import annotations - - import asyncio - import datetime - from typing import List - - from sqlalchemy import ForeignKey - from sqlalchemy import func - from sqlalchemy import select - from sqlalchemy.ext.asyncio import async_sessionmaker - from sqlalchemy.ext.asyncio import AsyncSession - from sqlalchemy.ext.asyncio import create_async_engine - from sqlalchemy.orm import DeclarativeBase - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import relationship - from sqlalchemy.orm import selectinload - - - class Base(DeclarativeBase): - pass - - - class A(Base): - __tablename__ = "a" - - id: Mapped[int] = mapped_column(primary_key=True) - data: Mapped[str] - create_date: Mapped[datetime.datetime] = mapped_column(server_default=func.now()) - bs: Mapped[List[B]] = relationship(lazy="raise") - - - class B(Base): - __tablename__ = "b" - id: Mapped[int] = mapped_column(primary_key=True) - a_id: Mapped[int] = mapped_column(ForeignKey("a.id")) - data: Mapped[str] - - - async def insert_objects(async_session: async_sessionmaker[AsyncSession]) -> None: - - async with async_session() as session: - async with session.begin(): - session.add_all( - [ - A(bs=[B(), B()], data="a1"), - A(bs=[], data="a2"), - A(bs=[B(), B()], data="a3"), - ] - ) - - - async def select_and_update_objects( - async_session: async_sessionmaker[AsyncSession], - ) -> None: - - async with async_session() as session: - stmt = select(A).options(selectinload(A.bs)) - - result = await session.execute(stmt) - - for a1 in result.scalars(): - print(a1) - print(f"created at: {a1.create_date}") - for b1 in a1.bs: - print(b1) - - result = await session.execute(select(A).order_by(A.id).limit(1)) - - a1 = result.scalars().one() - - a1.data = "new data" - - await session.commit() - - # access attribute subsequent to commit; this is what - # expire_on_commit=False allows - print(a1.data) - - - async def async_main() -> None: - engine = create_async_engine( - "postgresql+asyncpg://scott:tiger@localhost/test", - echo=True, - ) - - # async_sessionmaker: a factory for new AsyncSession objects. - # expire_on_commit - don't expire objects after transaction commit - async_session = async_sessionmaker(engine, expire_on_commit=False) - - async with engine.begin() as conn: - await conn.run_sync(Base.metadata.create_all) - - await insert_objects(async_session) - await select_and_update_objects(async_session) - - # for AsyncEngine created in function scope, close and - # clean-up pooled connections - await engine.dispose() - - - asyncio.run(async_main()) - -In the example above, the :class:`_asyncio.AsyncSession` is instantiated using -the optional :class:`_asyncio.async_sessionmaker` helper, which provides -a factory for new :class:`_asyncio.AsyncSession` objects with a fixed set -of parameters, which here includes associating it with -an :class:`_asyncio.AsyncEngine` against particular database URL. It is then -passed to other methods where it may be used in a Python asynchronous context -manager (i.e. ``async with:`` statement) so that it is automatically closed at -the end of the block; this is equivalent to calling the -:meth:`_asyncio.AsyncSession.close` method. - - -.. _asyncio_orm_avoid_lazyloads: - -Preventing Implicit IO when Using AsyncSession -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Using traditional asyncio, the application needs to avoid any points at which -IO-on-attribute access may occur. Techniques that can be used to help -this are below, many of which are illustrated in the preceding example. - -* Collections can be replaced with **write only collections** that will never - emit IO implicitly, by using the :ref:`write_only_relationship` feature in - SQLAlchemy 2.0. Using this feature, collections are never read from, only - queried using explicit SQL calls. See the example - ``async_orm_writeonly.py`` in the :ref:`examples_asyncio` section for - an example of write-only collections used with asyncio. - - When using write only collections, the program's behavior is simple and easy - to predict regarding collections. However, the downside is that there is not - any built-in system for loading many of these collections all at once, which - instead would need to be performed manually. Therefore, many of the - bullets below address specific techniques when using traditional lazy-loaded - relationships with asyncio, which requires more care. - -* If using traditional ORM relationships which are subject to lazy loading, - relationships can be declared with ``lazy="raise"`` so that by - default they will not attempt to emit SQL. In order to load collections, - :term:`eager loading` must be used in all cases. - -* The most useful eager loading strategy is the - :func:`_orm.selectinload` eager loader, which is employed in the previous - example in order to eagerly - load the ``A.bs`` collection within the scope of the - ``await session.execute()`` call:: - - stmt = select(A).options(selectinload(A.bs)) - -* When constructing new objects, **collections are always assigned a default, - empty collection**, such as a list in the above example:: - - A(bs=[], data="a2") - - This allows the ``.bs`` collection on the above ``A`` object to be present and - readable when the ``A`` object is flushed; otherwise, when the ``A`` is - flushed, ``.bs`` would be unloaded and would raise an error on access. - -* The :class:`_asyncio.AsyncSession` is configured using - :paramref:`_orm.Session.expire_on_commit` set to False, so that we may access - attributes on an object subsequent to a call to - :meth:`_asyncio.AsyncSession.commit`, as in the line at the end where we - access an attribute:: - - # create AsyncSession with expire_on_commit=False - async_session = AsyncSession(engine, expire_on_commit=False) - - # sessionmaker version - async_session = async_sessionmaker(engine, expire_on_commit=False) - - async with async_session() as session: - result = await session.execute(select(A).order_by(A.id)) - - a1 = result.scalars().first() - - # commit would normally expire all attributes - await session.commit() - - # access attribute subsequent to commit; this is what - # expire_on_commit=False allows - print(a1.data) - -Other guidelines include: - -* Methods like :meth:`_asyncio.AsyncSession.expire` should be avoided in favor of - :meth:`_asyncio.AsyncSession.refresh`; **if** expiration is absolutely needed. - Expiration should generally **not** be needed as - :paramref:`_orm.Session.expire_on_commit` - should normally be set to ``False`` when using asyncio. - -* A lazy-loaded relationship **can be loaded explicitly under asyncio** using - :meth:`_asyncio.AsyncSession.refresh`, **if** the desired attribute name - is passed explicitly to - :paramref:`_orm.Session.refresh.attribute_names`, e.g.:: - - # assume a_obj is an A that has lazy loaded A.bs collection - a_obj = await async_session.get(A, [1]) - - # force the collection to load by naming it in attribute_names - await async_session.refresh(a_obj, ["bs"]) - - # collection is present - print(f"bs collection: {a_obj.bs}") - - It's of course preferable to use eager loading up front in order to have - collections already set up without the need to lazy-load. - - .. versionadded:: 2.0.4 Added support for - :meth:`_asyncio.AsyncSession.refresh` and the underlying - :meth:`_orm.Session.refresh` method to force lazy-loaded relationships - to load, if they are named explicitly in the - :paramref:`_orm.Session.refresh.attribute_names` parameter. - In previous versions, the relationship would be silently skipped even - if named in the parameter. - -* Avoid using the ``all`` cascade option documented at :ref:`unitofwork_cascades` - in favor of listing out the desired cascade features explicitly. The - ``all`` cascade option implies among others the :ref:`cascade_refresh_expire` - setting, which means that the :meth:`.AsyncSession.refresh` method will - expire the attributes on related objects, but not necessarily refresh those - related objects assuming eager loading is not configured within the - :func:`_orm.relationship`, leaving them in an expired state. - -* Appropriate loader options should be employed for :func:`_orm.deferred` - columns, if used at all, in addition to that of :func:`_orm.relationship` - constructs as noted above. See :ref:`orm_queryguide_column_deferral` for - background on deferred column loading. - -.. _dynamic_asyncio: - -* The "dynamic" relationship loader strategy described at - :ref:`dynamic_relationship` is not compatible by default with the asyncio approach. - It can be used directly only if invoked within the - :meth:`_asyncio.AsyncSession.run_sync` method described at - :ref:`session_run_sync`, or by using its ``.statement`` attribute - to obtain a normal select:: - - user = await session.get(User, 42) - addresses = (await session.scalars(user.addresses.statement)).all() - stmt = user.addresses.statement.where(Address.email_address.startswith("patrick")) - addresses_filter = (await session.scalars(stmt)).all() - - The :ref:`write only ` technique, introduced in - version 2.0 of SQLAlchemy, is fully compatible with asyncio and should be - preferred. - - .. seealso:: - - :ref:`migration_20_dynamic_loaders` - notes on migration to 2.0 style - -* If using asyncio with a database that does not support RETURNING, such as - MySQL 8, server default values such as generated timestamps will not be - available on newly flushed objects unless the - :paramref:`_orm.Mapper.eager_defaults` option is used. In SQLAlchemy 2.0, - this behavior is applied automatically to backends like PostgreSQL, SQLite - and MariaDB which use RETURNING to fetch new values when rows are - INSERTed. - -.. _session_run_sync: - -Running Synchronous Methods and Functions under asyncio -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. deepalchemy:: This approach is essentially exposing publicly the - mechanism by which SQLAlchemy is able to provide the asyncio interface - in the first place. While there is no technical issue with doing so, overall - the approach can probably be considered "controversial" as it works against - some of the central philosophies of the asyncio programming model, which - is essentially that any programming statement that can potentially result - in IO being invoked **must** have an ``await`` call, lest the program - does not make it explicitly clear every line at which IO may occur. - This approach does not change that general idea, except that it allows - a series of synchronous IO instructions to be exempted from this rule - within the scope of a function call, essentially bundled up into a single - awaitable. - -As an alternative means of integrating traditional SQLAlchemy "lazy loading" -within an asyncio event loop, an **optional** method known as -:meth:`_asyncio.AsyncSession.run_sync` is provided which will run any -Python function inside of a greenlet, where traditional synchronous -programming concepts will be translated to use ``await`` when they reach the -database driver. A hypothetical approach here is an asyncio-oriented -application can package up database-related methods into functions that are -invoked using :meth:`_asyncio.AsyncSession.run_sync`. - -Altering the above example, if we didn't use :func:`_orm.selectinload` -for the ``A.bs`` collection, we could accomplish our treatment of these -attribute accesses within a separate function:: - - import asyncio - - from sqlalchemy import select - from sqlalchemy.ext.asyncio import AsyncSession, create_async_engine - - - def fetch_and_update_objects(session): - """run traditional sync-style ORM code in a function that will be - invoked within an awaitable. - - """ - - # the session object here is a traditional ORM Session. - # all features are available here including legacy Query use. - - stmt = select(A) - - result = session.execute(stmt) - for a1 in result.scalars(): - print(a1) - - # lazy loads - for b1 in a1.bs: - print(b1) - - # legacy Query use - a1 = session.query(A).order_by(A.id).first() - - a1.data = "new data" - - - async def async_main(): - engine = create_async_engine( - "postgresql+asyncpg://scott:tiger@localhost/test", - echo=True, - ) - async with engine.begin() as conn: - await conn.run_sync(Base.metadata.drop_all) - await conn.run_sync(Base.metadata.create_all) - - async with AsyncSession(engine) as session: - async with session.begin(): - session.add_all( - [ - A(bs=[B(), B()], data="a1"), - A(bs=[B()], data="a2"), - A(bs=[B(), B()], data="a3"), - ] - ) - - await session.run_sync(fetch_and_update_objects) - - await session.commit() - - # for AsyncEngine created in function scope, close and - # clean-up pooled connections - await engine.dispose() - - - asyncio.run(async_main()) - -The above approach of running certain functions within a "sync" runner -has some parallels to an application that runs a SQLAlchemy application -on top of an event-based programming library such as ``gevent``. The -differences are as follows: - -1. unlike when using ``gevent``, we can continue to use the standard Python - asyncio event loop, or any custom event loop, without the need to integrate - into the ``gevent`` event loop. - -2. There is no "monkeypatching" whatsoever. The above example makes use of - a real asyncio driver and the underlying SQLAlchemy connection pool is also - using the Python built-in ``asyncio.Queue`` for pooling connections. - -3. The program can freely switch between async/await code and contained - functions that use sync code with virtually no performance penalty. There - is no "thread executor" or any additional waiters or synchronization in use. - -4. The underlying network drivers are also using pure Python asyncio - concepts, no third party networking libraries as ``gevent`` and ``eventlet`` - provides are in use. - -.. _asyncio_events: - -Using events with the asyncio extension ---------------------------------------- - -The SQLAlchemy :ref:`event system ` is not directly exposed -by the asyncio extension, meaning there is not yet an "async" version of a -SQLAlchemy event handler. - -However, as the asyncio extension surrounds the usual synchronous SQLAlchemy -API, regular "synchronous" style event handlers are freely available as they -would be if asyncio were not used. - -As detailed below, there are two current strategies to register events given -asyncio-facing APIs: - -* Events can be registered at the instance level (e.g. a specific - :class:`_asyncio.AsyncEngine` instance) by associating the event with the - ``sync`` attribute that refers to the proxied object. For example to register - the :meth:`_events.PoolEvents.connect` event against an - :class:`_asyncio.AsyncEngine` instance, use its - :attr:`_asyncio.AsyncEngine.sync_engine` attribute as target. Targets - include: - - :attr:`_asyncio.AsyncEngine.sync_engine` - - :attr:`_asyncio.AsyncConnection.sync_connection` - - :attr:`_asyncio.AsyncConnection.sync_engine` - - :attr:`_asyncio.AsyncSession.sync_session` - -* To register an event at the class level, targeting all instances of the same type (e.g. - all :class:`_asyncio.AsyncSession` instances), use the corresponding - sync-style class. For example to register the - :meth:`_ormevents.SessionEvents.before_commit` event against the - :class:`_asyncio.AsyncSession` class, use the :class:`_orm.Session` class as - the target. - -* To register at the :class:`_orm.sessionmaker` level, combine an explicit - :class:`_orm.sessionmaker` with an :class:`_asyncio.async_sessionmaker` - using :paramref:`_asyncio.async_sessionmaker.sync_session_class`, and - associate events with the :class:`_orm.sessionmaker`. - -When working within an event handler that is within an asyncio context, objects -like the :class:`_engine.Connection` continue to work in their usual -"synchronous" way without requiring ``await`` or ``async`` usage; when messages -are ultimately received by the asyncio database adapter, the calling style is -transparently adapted back into the asyncio calling style. For events that -are passed a DBAPI level connection, such as :meth:`_events.PoolEvents.connect`, -the object is a :term:`pep-249` compliant "connection" object which will adapt -sync-style calls into the asyncio driver. - -Examples of Event Listeners with Async Engines / Sessions / Sessionmakers -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Some examples of sync style event handlers associated with async-facing API -constructs are illustrated below: - -* **Core Events on AsyncEngine** - - In this example, we access the :attr:`_asyncio.AsyncEngine.sync_engine` - attribute of :class:`_asyncio.AsyncEngine` as the target for - :class:`.ConnectionEvents` and :class:`.PoolEvents`:: - - import asyncio - - from sqlalchemy import event - from sqlalchemy import text - from sqlalchemy.engine import Engine - from sqlalchemy.ext.asyncio import create_async_engine - - engine = create_async_engine("postgresql+asyncpg://scott:tiger@localhost:5432/test") - - - # connect event on instance of Engine - @event.listens_for(engine.sync_engine, "connect") - def my_on_connect(dbapi_con, connection_record): - print("New DBAPI connection:", dbapi_con) - cursor = dbapi_con.cursor() - - # sync style API use for adapted DBAPI connection / cursor - cursor.execute("select 'execute from event'") - print(cursor.fetchone()[0]) - - - # before_execute event on all Engine instances - @event.listens_for(Engine, "before_execute") - def my_before_execute( - conn, - clauseelement, - multiparams, - params, - execution_options, - ): - print("before execute!") - - - async def go(): - async with engine.connect() as conn: - await conn.execute(text("select 1")) - await engine.dispose() - - - asyncio.run(go()) - - Output: - - .. sourcecode:: text - - New DBAPI connection: > - execute from event - before execute! - - -* **ORM Events on AsyncSession** - - In this example, we access :attr:`_asyncio.AsyncSession.sync_session` as the - target for :class:`_orm.SessionEvents`:: - - import asyncio - - from sqlalchemy import event - from sqlalchemy import text - from sqlalchemy.ext.asyncio import AsyncSession - from sqlalchemy.ext.asyncio import create_async_engine - from sqlalchemy.orm import Session - - engine = create_async_engine("postgresql+asyncpg://scott:tiger@localhost:5432/test") - - session = AsyncSession(engine) - - - # before_commit event on instance of Session - @event.listens_for(session.sync_session, "before_commit") - def my_before_commit(session): - print("before commit!") - - # sync style API use on Session - connection = session.connection() - - # sync style API use on Connection - result = connection.execute(text("select 'execute from event'")) - print(result.first()) - - - # after_commit event on all Session instances - @event.listens_for(Session, "after_commit") - def my_after_commit(session): - print("after commit!") - - - async def go(): - await session.execute(text("select 1")) - await session.commit() - - await session.close() - await engine.dispose() - - - asyncio.run(go()) - - Output: - - .. sourcecode:: text - - before commit! - execute from event - after commit! - - -* **ORM Events on async_sessionmaker** - - For this use case, we make a :class:`_orm.sessionmaker` as the event target, - then assign it to the :class:`_asyncio.async_sessionmaker` using - the :paramref:`_asyncio.async_sessionmaker.sync_session_class` parameter:: - - import asyncio - - from sqlalchemy import event - from sqlalchemy.ext.asyncio import async_sessionmaker - from sqlalchemy.orm import sessionmaker - - sync_maker = sessionmaker() - maker = async_sessionmaker(sync_session_class=sync_maker) - - - @event.listens_for(sync_maker, "before_commit") - def before_commit(session): - print("before commit") - - - async def main(): - async_session = maker() - - await async_session.commit() - - - asyncio.run(main()) - - Output: - - .. sourcecode:: text - - before commit - - -.. topic:: asyncio and events, two opposites - - SQLAlchemy events by their nature take place within the **interior** of a - particular SQLAlchemy process; that is, an event always occurs *after* some - particular SQLAlchemy API has been invoked by end-user code, and *before* - some other internal aspect of that API occurs. - - Contrast this to the architecture of the asyncio extension, which takes - place on the **exterior** of SQLAlchemy's usual flow from end-user API to - DBAPI function. - - The flow of messaging may be visualized as follows: - - .. sourcecode:: text - - SQLAlchemy SQLAlchemy SQLAlchemy SQLAlchemy plain - asyncio asyncio ORM/Core asyncio asyncio - (public (internal) (internal) - facing) - -------------|------------|------------------------|-----------|------------ - asyncio API | | | | - call -> | | | | - | -> -> | | -> -> | - |~~~~~~~~~~~~| sync API call -> |~~~~~~~~~~~| - | asyncio | event hooks -> | sync | - | to | invoke action -> | to | - | sync | event hooks -> | asyncio | - | (greenlet) | dialect -> | (leave | - |~~~~~~~~~~~~| event hooks -> | greenlet) | - | -> -> | sync adapted |~~~~~~~~~~~| - | | DBAPI -> | -> -> | asyncio - | | | | driver -> database - - - Where above, an API call always starts as asyncio, flows through the - synchronous API, and ends as asyncio, before results are propagated through - this same chain in the opposite direction. In between, the message is - adapted first into sync-style API use, and then back out to async style. - Event hooks then by their nature occur in the middle of the "sync-style API - use". From this it follows that the API presented within event hooks - occurs inside the process by which asyncio API requests have been adapted - to sync, and outgoing messages to the database API will be converted - to asyncio transparently. - -.. _asyncio_events_run_async: - -Using awaitable-only driver methods in connection pool and other events -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -As discussed in the above section, event handlers such as those oriented -around the :class:`.PoolEvents` event handlers receive a sync-style "DBAPI" connection, -which is a wrapper object supplied by SQLAlchemy asyncio dialects to adapt -the underlying asyncio "driver" connection into one that can be used by -SQLAlchemy's internals. A special use case arises when the user-defined -implementation for such an event handler needs to make use of the -ultimate "driver" connection directly, using awaitable only methods on that -driver connection. One such example is the ``.set_type_codec()`` method -supplied by the asyncpg driver. - -To accommodate this use case, SQLAlchemy's :class:`.AdaptedConnection` -class provides a method :meth:`.AdaptedConnection.run_async` that allows -an awaitable function to be invoked within the "synchronous" context of -an event handler or other SQLAlchemy internal. This method is directly -analogous to the :meth:`_asyncio.AsyncConnection.run_sync` method that -allows a sync-style method to run under async. - -:meth:`.AdaptedConnection.run_async` should be passed a function that will -accept the innermost "driver" connection as a single argument, and return -an awaitable that will be invoked by the :meth:`.AdaptedConnection.run_async` -method. The given function itself does not need to be declared as ``async``; -it's perfectly fine for it to be a Python ``lambda:``, as the return awaitable -value will be invoked after being returned:: - - from sqlalchemy import event - from sqlalchemy.ext.asyncio import create_async_engine - - engine = create_async_engine(...) - - - @event.listens_for(engine.sync_engine, "connect") - def register_custom_types(dbapi_connection, *args): - dbapi_connection.run_async( - lambda connection: connection.set_type_codec( - "MyCustomType", - encoder, - decoder, # ... - ) - ) - -Above, the object passed to the ``register_custom_types`` event handler -is an instance of :class:`.AdaptedConnection`, which provides a DBAPI-like -interface to an underlying async-only driver-level connection object. -The :meth:`.AdaptedConnection.run_async` method then provides access to an -awaitable environment where the underlying driver level connection may be -acted upon. - -.. versionadded:: 1.4.30 - - -Using multiple asyncio event loops ----------------------------------- - -An application that makes use of multiple event loops, for example in the -uncommon case of combining asyncio with multithreading, should not share the -same :class:`_asyncio.AsyncEngine` with different event loops when using the -default pool implementation. - -If an :class:`_asyncio.AsyncEngine` is be passed from one event loop to another, -the method :meth:`_asyncio.AsyncEngine.dispose()` should be called before it's -re-used on a new event loop. Failing to do so may lead to a ``RuntimeError`` -along the lines of -``Task got Future attached to a different loop`` - -If the same engine must be shared between different loop, it should be configured -to disable pooling using :class:`~sqlalchemy.pool.NullPool`, preventing the Engine -from using any connection more than once:: - - from sqlalchemy.ext.asyncio import create_async_engine - from sqlalchemy.pool import NullPool - - engine = create_async_engine( - "postgresql+asyncpg://user:pass@host/dbname", - poolclass=NullPool, - ) - -.. _asyncio_scoped_session: - -Using asyncio scoped session ----------------------------- - -The "scoped session" pattern used in threaded SQLAlchemy with the -:class:`.scoped_session` object is also available in asyncio, using -an adapted version called :class:`_asyncio.async_scoped_session`. - -.. tip:: SQLAlchemy generally does not recommend the "scoped" pattern - for new development as it relies upon mutable global state that must also be - explicitly torn down when work within the thread or task is complete. - Particularly when using asyncio, it's likely a better idea to pass the - :class:`_asyncio.AsyncSession` directly to the awaitable functions that need - it. - -When using :class:`_asyncio.async_scoped_session`, as there's no "thread-local" -concept in the asyncio context, the "scopefunc" parameter must be provided to -the constructor. The example below illustrates using the -``asyncio.current_task()`` function for this purpose:: - - from asyncio import current_task - - from sqlalchemy.ext.asyncio import ( - async_scoped_session, - async_sessionmaker, - ) - - async_session_factory = async_sessionmaker( - some_async_engine, - expire_on_commit=False, - ) - AsyncScopedSession = async_scoped_session( - async_session_factory, - scopefunc=current_task, - ) - some_async_session = AsyncScopedSession() - -.. warning:: The "scopefunc" used by :class:`_asyncio.async_scoped_session` - is invoked **an arbitrary number of times** within a task, once for each - time the underlying :class:`_asyncio.AsyncSession` is accessed. The function - should therefore be **idempotent** and lightweight, and should not attempt - to create or mutate any state, such as establishing callbacks, etc. - -.. warning:: Using ``current_task()`` for the "key" in the scope requires that - the :meth:`_asyncio.async_scoped_session.remove` method is called from - within the outermost awaitable, to ensure the key is removed from the - registry when the task is complete, otherwise the task handle as well as - the :class:`_asyncio.AsyncSession` will remain in memory, essentially - creating a memory leak. See the following example which illustrates - the correct use of :meth:`_asyncio.async_scoped_session.remove`. - -:class:`_asyncio.async_scoped_session` includes **proxy -behavior** similar to that of :class:`.scoped_session`, which means it can be -treated as a :class:`_asyncio.AsyncSession` directly, keeping in mind that -the usual ``await`` keywords are necessary, including for the -:meth:`_asyncio.async_scoped_session.remove` method:: - - async def some_function(some_async_session, some_object): - # use the AsyncSession directly - some_async_session.add(some_object) - - # use the AsyncSession via the context-local proxy - await AsyncScopedSession.commit() - - # "remove" the current proxied AsyncSession for the local context - await AsyncScopedSession.remove() - -.. versionadded:: 1.4.19 - -.. currentmodule:: sqlalchemy.ext.asyncio - - -.. _asyncio_inspector: - -Using the Inspector to inspect schema objects ---------------------------------------------------- - -SQLAlchemy does not yet offer an asyncio version of the -:class:`_reflection.Inspector` (introduced at :ref:`metadata_reflection_inspector`), -however the existing interface may be used in an asyncio context by -leveraging the :meth:`_asyncio.AsyncConnection.run_sync` method of -:class:`_asyncio.AsyncConnection`:: - - import asyncio - - from sqlalchemy import inspect - from sqlalchemy.ext.asyncio import create_async_engine - - engine = create_async_engine("postgresql+asyncpg://scott:tiger@localhost/test") - - - def use_inspector(conn): - inspector = inspect(conn) - # use the inspector - print(inspector.get_view_names()) - # return any value to the caller - return inspector.get_table_names() - - - async def async_main(): - async with engine.connect() as conn: - tables = await conn.run_sync(use_inspector) - - - asyncio.run(async_main()) - -.. seealso:: - - :ref:`metadata_reflection` - - :ref:`inspection_toplevel` - -Engine API Documentation -------------------------- - -.. autofunction:: create_async_engine - -.. autofunction:: async_engine_from_config - -.. autofunction:: create_async_pool_from_url - -.. autoclass:: AsyncEngine - :members: - -.. autoclass:: AsyncConnection - :members: - -.. autoclass:: AsyncTransaction - :members: - -Result Set API Documentation ----------------------------------- - -The :class:`_asyncio.AsyncResult` object is an async-adapted version of the -:class:`_result.Result` object. It is only returned when using the -:meth:`_asyncio.AsyncConnection.stream` or :meth:`_asyncio.AsyncSession.stream` -methods, which return a result object that is on top of an active database -cursor. - -.. autoclass:: AsyncResult - :members: - :inherited-members: - -.. autoclass:: AsyncScalarResult - :members: - :inherited-members: - -.. autoclass:: AsyncMappingResult - :members: - :inherited-members: - -.. autoclass:: AsyncTupleResult - -ORM Session API Documentation ------------------------------ - -.. autofunction:: async_object_session - -.. autofunction:: async_session - -.. autoclass:: async_sessionmaker - :members: - :inherited-members: - -.. autoclass:: async_scoped_session - :members: - :inherited-members: - -.. autoclass:: AsyncSession - :members: - :exclude-members: sync_session_class - - .. autoattribute:: sync_session_class - -.. autoclass:: AsyncSessionTransaction - :members: - - - diff --git a/doc/build/orm/extensions/automap.rst b/doc/build/orm/extensions/automap.rst deleted file mode 100644 index d1d200609fb..00000000000 --- a/doc/build/orm/extensions/automap.rst +++ /dev/null @@ -1,22 +0,0 @@ -.. _automap_toplevel: - -Automap -======= - -.. automodule:: sqlalchemy.ext.automap - -API Reference -------------- - -.. autofunction:: automap_base - -.. autoclass:: AutomapBase - :members: - -.. autofunction:: classname_for_table - -.. autofunction:: name_for_scalar_relationship - -.. autofunction:: name_for_collection_relationship - -.. autofunction:: generate_relationship diff --git a/doc/build/orm/extensions/baked.rst b/doc/build/orm/extensions/baked.rst deleted file mode 100644 index b495f42a422..00000000000 --- a/doc/build/orm/extensions/baked.rst +++ /dev/null @@ -1,483 +0,0 @@ -.. _baked_toplevel: - -Baked Queries -============= - -.. module:: sqlalchemy.ext.baked - -``baked`` provides an alternative creational pattern for -:class:`~.query.Query` objects, which allows for caching of the object's -construction and string-compilation steps. This means that for a -particular :class:`~.query.Query` building scenario that is used more than -once, all of the Python function invocation involved in building the query -from its initial construction up through generating a SQL string will only -occur **once**, rather than for each time that query is built up and executed. - -The rationale for this system is to greatly reduce Python interpreter -overhead for everything that occurs **before the SQL is emitted**. -The caching of the "baked" system does **not** in any way reduce SQL calls or -cache the **return results** from the database. A technique that demonstrates -the caching of the SQL calls and result sets themselves is available in -:ref:`examples_caching`. - -.. deprecated:: 1.4 SQLAlchemy 1.4 and 2.0 feature an all-new direct query - caching system that removes the need for the :class:`.BakedQuery` system. - Caching is now transparently active for all Core and ORM queries with no - action taken by the user, using the system described at :ref:`sql_caching`. - - -.. deepalchemy:: - - The :mod:`sqlalchemy.ext.baked` extension is **not for beginners**. Using - it correctly requires a good high level understanding of how SQLAlchemy, the - database driver, and the backend database interact with each other. This - extension presents a very specific kind of optimization that is not ordinarily - needed. As noted above, it **does not cache queries**, only the string - formulation of the SQL itself. - -Synopsis --------- - -Usage of the baked system starts by producing a so-called "bakery", which -represents storage for a particular series of query objects:: - - from sqlalchemy.ext import baked - - bakery = baked.bakery() - -The above "bakery" will store cached data in an LRU cache that defaults -to 200 elements, noting that an ORM query will typically contain one entry -for the ORM query as invoked, as well as one entry per database dialect for -the SQL string. - -The bakery allows us to build up a :class:`~.query.Query` object by specifying -its construction as a series of Python callables, which are typically lambdas. -For succinct usage, it overrides the ``+=`` operator so that a typical -query build-up looks like the following:: - - from sqlalchemy import bindparam - - - def search_for_user(session, username, email=None): - baked_query = bakery(lambda session: session.query(User)) - baked_query += lambda q: q.filter(User.name == bindparam("username")) - - baked_query += lambda q: q.order_by(User.id) - - if email: - baked_query += lambda q: q.filter(User.email == bindparam("email")) - - result = baked_query(session).params(username=username, email=email).all() - - return result - -Following are some observations about the above code: - -1. The ``baked_query`` object is an instance of :class:`.BakedQuery`. This - object is essentially the "builder" for a real orm :class:`~.query.Query` - object, but it is not itself the *actual* :class:`~.query.Query` - object. - -2. The actual :class:`~.query.Query` object is not built at all, until the - very end of the function when :meth:`_baked.Result.all` is called. - -3. The steps that are added to the ``baked_query`` object are all expressed - as Python functions, typically lambdas. The first lambda given - to the :func:`.bakery` function receives a :class:`.Session` as its - argument. The remaining lambdas each receive a :class:`~.query.Query` - as their argument. - -4. In the above code, even though our application may call upon - ``search_for_user()`` many times, and even though within each invocation - we build up an entirely new :class:`.BakedQuery` object, - *all of the lambdas are only called once*. Each lambda is **never** called - a second time for as long as this query is cached in the bakery. - -5. The caching is achieved by storing references to the **lambda objects - themselves** in order to formulate a cache key; that is, the fact that the - Python interpreter assigns an in-Python identity to these functions is - what determines how to identify the query on successive runs. For - those invocations of ``search_for_user()`` where the ``email`` parameter - is specified, the callable ``lambda q: q.filter(User.email == bindparam('email'))`` - will be part of the cache key that's retrieved; when ``email`` is - ``None``, this callable is not part of the cache key. - -6. Because the lambdas are all called only once, it is essential that no - variables which may change across calls are referenced **within** the - lambdas; instead, assuming these are values to be bound into the - SQL string, we use :func:`.bindparam` to construct named parameters, - where we apply their actual values later using :meth:`_baked.Result.params`. - - -Performance ------------ - -The baked query probably looks a little odd, a little bit awkward and -a little bit verbose. However, the savings in -Python performance for a query which is invoked lots of times in an -application are very dramatic. The example suite ``short_selects`` -demonstrated in :ref:`examples_performance` illustrates a comparison -of queries which each return only one row, such as the following regular -query:: - - session = Session(bind=engine) - for id_ in random.sample(ids, n): - session.query(Customer).filter(Customer.id == id_).one() - -compared to the equivalent "baked" query:: - - bakery = baked.bakery() - s = Session(bind=engine) - for id_ in random.sample(ids, n): - q = bakery(lambda s: s.query(Customer)) - q += lambda q: q.filter(Customer.id == bindparam("id")) - q(s).params(id=id_).one() - -The difference in Python function call count for an iteration of 10000 -calls to each block are: - -.. sourcecode:: text - - test_baked_query : test a baked query of the full entity. - (10000 iterations); total fn calls 1951294 - - test_orm_query : test a straight ORM query of the full entity. - (10000 iterations); total fn calls 7900535 - -In terms of number of seconds on a powerful laptop, this comes out as: - -.. sourcecode:: text - - test_baked_query : test a baked query of the full entity. - (10000 iterations); total time 2.174126 sec - - test_orm_query : test a straight ORM query of the full entity. - (10000 iterations); total time 7.958516 sec - -Note that this test very intentionally features queries that only return one row. -For queries that return many rows, the performance advantage of the baked query will have -less and less of an impact, proportional to the time spent fetching rows. -It is critical to keep in mind that the **baked query feature only applies to -building the query itself, not the fetching of results**. Using the -baked feature is by no means a guarantee to a much faster application; it is -only a potentially useful feature for those applications that have been measured -as being impacted by this particular form of overhead. - -.. topic:: Measure twice, cut once - - For background on how to profile a SQLAlchemy application, please see - the section :ref:`faq_performance`. It is essential that performance - measurement techniques are used when attempting to improve the performance - of an application. - -Rationale ---------- - -The "lambda" approach above is a superset of what would be a more -traditional "parameterized" approach. Suppose we wished to build -a simple system where we build a :class:`~.query.Query` just once, then -store it in a dictionary for re-use. This is possible right now by -just building up the query, and removing its :class:`.Session` by calling -``my_cached_query = query.with_session(None)``:: - - my_simple_cache = {} - - - def lookup(session, id_argument): - if "my_key" not in my_simple_cache: - query = session.query(Model).filter(Model.id == bindparam("id")) - my_simple_cache["my_key"] = query.with_session(None) - else: - query = my_simple_cache["my_key"].with_session(session) - - return query.params(id=id_argument).all() - -The above approach gets us a very minimal performance benefit. -By re-using a :class:`~.query.Query`, we save on the Python work within -the ``session.query(Model)`` constructor as well as calling upon -``filter(Model.id == bindparam('id'))``, which will skip for us the building -up of the Core expression as well as sending it to :meth:`_query.Query.filter`. -However, the approach still regenerates the full :class:`_expression.Select` -object every time when :meth:`_query.Query.all` is called and additionally this -brand new :class:`_expression.Select` is sent off to the string compilation step every -time, which for a simple case like the above is probably about 70% of the -overhead. - -To reduce the additional overhead, we need some more specialized logic, -some way to memoize the construction of the select object and the -construction of the SQL. There is an example of this on the wiki -in the section `BakedQuery `_, -a precursor to this feature, however in that system, we aren't caching -the *construction* of the query. In order to remove all the overhead, -we need to cache both the construction of the query as well as the SQL -compilation. Let's assume we adapted the recipe in this way -and made ourselves a method ``.bake()`` that pre-compiles the SQL for the -query, producing a new object that can be invoked with minimal overhead. -Our example becomes:: - - my_simple_cache = {} - - - def lookup(session, id_argument): - if "my_key" not in my_simple_cache: - query = session.query(Model).filter(Model.id == bindparam("id")) - my_simple_cache["my_key"] = query.with_session(None).bake() - else: - query = my_simple_cache["my_key"].with_session(session) - - return query.params(id=id_argument).all() - -Above, we've fixed the performance situation, but we still have this -string cache key to deal with. - -We can use the "bakery" approach to re-frame the above in a way that -looks less unusual than the "building up lambdas" approach, and more like -a simple improvement upon the simple "reuse a query" approach:: - - bakery = baked.bakery() - - - def lookup(session, id_argument): - def create_model_query(session): - return session.query(Model).filter(Model.id == bindparam("id")) - - parameterized_query = bakery.bake(create_model_query) - return parameterized_query(session).params(id=id_argument).all() - -Above, we use the "baked" system in a manner that is -very similar to the simplistic "cache a query" system. However, it -uses two fewer lines of code, does not need to manufacture a cache key of -"my_key", and also includes the same feature as our custom "bake" function -that caches 100% of the Python invocation work from the -constructor of the query, to the filter call, to the production -of the :class:`_expression.Select` object, to the string compilation step. - -From the above, if we ask ourselves, "what if lookup needs to make conditional decisions -as to the structure of the query?", this is where hopefully it becomes apparent -why "baked" is the way it is. Instead of a parameterized query building -off from exactly one function (which is how we thought baked might work -originally), we can build it from *any number* of functions. Consider -our naive example, if we needed to have an additional clause in our -query on a conditional basis:: - - my_simple_cache = {} - - - def lookup(session, id_argument, include_frobnizzle=False): - if include_frobnizzle: - cache_key = "my_key_with_frobnizzle" - else: - cache_key = "my_key_without_frobnizzle" - - if cache_key not in my_simple_cache: - query = session.query(Model).filter(Model.id == bindparam("id")) - if include_frobnizzle: - query = query.filter(Model.frobnizzle == True) - - my_simple_cache[cache_key] = query.with_session(None).bake() - else: - query = my_simple_cache[cache_key].with_session(session) - - return query.params(id=id_argument).all() - -Our "simple" parameterized system must now be tasked with generating -cache keys which take into account whether or not the "include_frobnizzle" -flag was passed, as the presence of this flag means that the generated -SQL would be entirely different. It should be apparent that as the -complexity of query building goes up, the task of caching these queries -becomes burdensome very quickly. We can convert the above example -into a direct use of "bakery" as follows:: - - - bakery = baked.bakery() - - - def lookup(session, id_argument, include_frobnizzle=False): - def create_model_query(session): - return session.query(Model).filter(Model.id == bindparam("id")) - - parameterized_query = bakery.bake(create_model_query) - - if include_frobnizzle: - - def include_frobnizzle_in_query(query): - return query.filter(Model.frobnizzle == True) - - parameterized_query = parameterized_query.with_criteria( - include_frobnizzle_in_query - ) - - return parameterized_query(session).params(id=id_argument).all() - -Above, we again cache not just the query object but all the work it needs -to do in order to generate SQL. We also no longer need to deal with -making sure we generate a cache key that accurately takes into account -all of the structural modifications we've made; this is now handled -automatically and without the chance of mistakes. - -This code sample is a few lines shorter than the naive example, removes -the need to deal with cache keys, and has the vast performance benefits -of the full so-called "baked" feature. But -still a little verbose! Hence we take methods like :meth:`.BakedQuery.add_criteria` -and :meth:`.BakedQuery.with_criteria` and shorten them into operators, and -encourage (though certainly not require!) using simple lambdas, only as a -means to reduce verbosity:: - - bakery = baked.bakery() - - - def lookup(session, id_argument, include_frobnizzle=False): - parameterized_query = bakery.bake( - lambda s: s.query(Model).filter(Model.id == bindparam("id")) - ) - - if include_frobnizzle: - parameterized_query += lambda q: q.filter(Model.frobnizzle == True) - - return parameterized_query(session).params(id=id_argument).all() - -Where above, the approach is simpler to implement and much more similar -in code flow to what a non-cached querying function would look like, -hence making code easier to port. - -The above description is essentially a summary of the design process used -to arrive at the current "baked" approach. Starting from the -"normal" approaches, the additional issues of cache key construction and -management, removal of all redundant Python execution, and queries built up -with conditionals needed to be addressed, leading to the final approach. - -Special Query Techniques ------------------------- - -This section will describe some techniques for specific query situations. - -.. _baked_in: - -Using IN expressions -^^^^^^^^^^^^^^^^^^^^ - -The :meth:`.ColumnOperators.in_` method in SQLAlchemy historically renders -a variable set of bound parameters based on the list of items that's passed -to the method. This doesn't work for baked queries as the length of that -list can change on different calls. To solve this problem, the -:paramref:`.bindparam.expanding` parameter supports a late-rendered IN -expression that is safe to be cached inside of baked query. The actual list -of elements is rendered at statement execution time, rather than at -statement compilation time:: - - bakery = baked.bakery() - - baked_query = bakery(lambda session: session.query(User)) - baked_query += lambda q: q.filter(User.name.in_(bindparam("username", expanding=True))) - - result = baked_query.with_session(session).params(username=["ed", "fred"]).all() - -.. seealso:: - - :paramref:`.bindparam.expanding` - - :meth:`.ColumnOperators.in_` - -Using Subqueries -^^^^^^^^^^^^^^^^ - -When using :class:`_query.Query` objects, it is often needed that one :class:`_query.Query` -object is used to generate a subquery within another. In the case where the -:class:`_query.Query` is currently in baked form, an interim method may be used to -retrieve the :class:`_query.Query` object, using the :meth:`.BakedQuery.to_query` -method. This method is passed the :class:`.Session` or :class:`_query.Query` that is -the argument to the lambda callable used to generate a particular step -of the baked query:: - - bakery = baked.bakery() - - # a baked query that will end up being used as a subquery - my_subq = bakery(lambda s: s.query(User.id)) - my_subq += lambda q: q.filter(User.id == Address.user_id) - - # select a correlated subquery in the top columns list, - # we have the "session" argument, pass that - my_q = bakery(lambda s: s.query(Address.id, my_subq.to_query(s).as_scalar())) - - # use a correlated subquery in some of the criteria, we have - # the "query" argument, pass that. - my_q += lambda q: q.filter(my_subq.to_query(q).exists()) - -.. versionadded:: 1.3 - -.. _baked_with_before_compile: - -Using the before_compile event -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -As of SQLAlchemy 1.3.11, the use of the :meth:`.QueryEvents.before_compile` -event against a particular :class:`_query.Query` will disallow the baked query -system from caching the query, if the event hook returns a new :class:`_query.Query` -object that is different from the one passed in. This is so that the -:meth:`.QueryEvents.before_compile` hook may be invoked against a particular -:class:`_query.Query` every time it is used, to accommodate for hooks that -alter the query differently each time. To allow a -:meth:`.QueryEvents.before_compile` to alter a :meth:`_query.Query` object, but -still to allow the result to be cached, the event can be registered -passing the ``bake_ok=True`` flag:: - - @event.listens_for(Query, "before_compile", retval=True, bake_ok=True) - def my_event(query): - for desc in query.column_descriptions: - if desc["type"] is User: - entity = desc["entity"] - query = query.filter(entity.deleted == False) - return query - -The above strategy is appropriate for an event that will modify a -given :class:`_query.Query` in exactly the same way every time, not dependent -on specific parameters or external state that changes. - -.. versionadded:: 1.3.11 - added the "bake_ok" flag to the - :meth:`.QueryEvents.before_compile` event and disallowed caching via - the "baked" extension from occurring for event handlers that - return a new :class:`_query.Query` object if this flag is not set. - - -Disabling Baked Queries Session-wide ------------------------------------- - -The flag :paramref:`.Session.enable_baked_queries` may be set to False, -causing all baked queries to not use the cache when used against that -:class:`.Session`:: - - session = Session(engine, enable_baked_queries=False) - -Like all session flags, it is also accepted by factory objects like -:class:`.sessionmaker` and methods like :meth:`.sessionmaker.configure`. - -The immediate rationale for this flag is so that an application -which is seeing issues potentially due to cache key conflicts from user-defined -baked queries or other baked query issues can turn the behavior off, in -order to identify or eliminate baked queries as the cause of an issue. - -.. versionadded:: 1.2 - -Lazy Loading Integration ------------------------- - -.. versionchanged:: 1.4 As of SQLAlchemy 1.4, the "baked query" system is no - longer part of the relationship loading system. - The :ref:`native caching ` system is used instead. - - -API Documentation ------------------ - -.. autofunction:: bakery - -.. autoclass:: BakedQuery - :members: - -.. autoclass:: Bakery - :members: - -.. autoclass:: Result - :members: - :noindex: - diff --git a/doc/build/orm/extensions/declarative/api.rst b/doc/build/orm/extensions/declarative/api.rst deleted file mode 100644 index 98924c2e275..00000000000 --- a/doc/build/orm/extensions/declarative/api.rst +++ /dev/null @@ -1,28 +0,0 @@ -:orphan: - -.. automodule:: sqlalchemy.ext.declarative - -=============== -Declarative API -=============== - -API Reference -============= - -.. versionchanged:: 1.4 The fundamental structures of the declarative - system are now part of SQLAlchemy ORM directly. For these components - see: - - * :func:`_orm.declarative_base` - - * :class:`_orm.declared_attr` - - * :func:`_orm.has_inherited_table` - - * :func:`_orm.synonym_for` - - * :meth:`_orm.as_declarative` - -See :ref:`declarative_toplevel` for the remaining Declarative extension -classes. - diff --git a/doc/build/orm/extensions/declarative/basic_use.rst b/doc/build/orm/extensions/declarative/basic_use.rst deleted file mode 100644 index 49903559d5c..00000000000 --- a/doc/build/orm/extensions/declarative/basic_use.rst +++ /dev/null @@ -1,40 +0,0 @@ -:orphan: - -========= -Basic Use -========= - -This section has moved to :ref:`orm_declarative_mapping`. - -Defining Attributes -=================== - -This section is covered by :ref:`mapping_columns_toplevel` - - - -Accessing the MetaData -====================== - -This section has moved to :ref:`orm_declarative_metadata`. - - -Class Constructor -================= - -This section has moved to :ref:`orm_mapper_configuration_overview`. - -Mapper Configuration -==================== - -This section is moved to :ref:`orm_declarative_mapper_options`. - - -.. _declarative_sql_expressions: - -Defining SQL Expressions -======================== - -See :ref:`mapper_sql_expressions` for examples on declaratively -mapping attributes to SQL expressions. - diff --git a/doc/build/orm/extensions/declarative/index.rst b/doc/build/orm/extensions/declarative/index.rst deleted file mode 100644 index 6cf1a60a1c6..00000000000 --- a/doc/build/orm/extensions/declarative/index.rst +++ /dev/null @@ -1,24 +0,0 @@ -.. _declarative_toplevel: - -.. currentmodule:: sqlalchemy.ext.declarative - -====================== -Declarative Extensions -====================== - -Extensions specific to the :ref:`Declarative ` -mapping API. - -.. versionchanged:: 1.4 The vast majority of the Declarative extension is now - integrated into the SQLAlchemy ORM and is importable from the - ``sqlalchemy.orm`` namespace. See the documentation at - :ref:`orm_declarative_mapping` for new documentation. - For an overview of the change, see :ref:`change_5508`. - -.. autoclass:: AbstractConcreteBase - -.. autoclass:: ConcreteBase - -.. autoclass:: DeferredReflection - :members: - diff --git a/doc/build/orm/extensions/declarative/inheritance.rst b/doc/build/orm/extensions/declarative/inheritance.rst deleted file mode 100644 index 849664a3c33..00000000000 --- a/doc/build/orm/extensions/declarative/inheritance.rst +++ /dev/null @@ -1,8 +0,0 @@ -:orphan: - -.. _declarative_inheritance: - -Declarative Inheritance -======================= - -See :ref:`inheritance_toplevel` for this section. diff --git a/doc/build/orm/extensions/declarative/mixins.rst b/doc/build/orm/extensions/declarative/mixins.rst deleted file mode 100644 index 7a18f07a7f3..00000000000 --- a/doc/build/orm/extensions/declarative/mixins.rst +++ /dev/null @@ -1,8 +0,0 @@ -:orphan: - -.. _declarative_mixins: - -Mixin and Custom Base Classes -============================= - -See :ref:`orm_mixins_toplevel` for this section. diff --git a/doc/build/orm/extensions/declarative/relationships.rst b/doc/build/orm/extensions/declarative/relationships.rst deleted file mode 100644 index c0df8b49cff..00000000000 --- a/doc/build/orm/extensions/declarative/relationships.rst +++ /dev/null @@ -1,25 +0,0 @@ -:orphan: - -.. _declarative_configuring_relationships: - -========================= -Configuring Relationships -========================= - -This section is covered by :ref:`orm_declarative_properties`. - -.. _declarative_relationship_eval: - -Evaluation of relationship arguments -===================================== - -This section is moved to :ref:`orm_declarative_relationship_eval`. - - -.. _declarative_many_to_many: - -Configuring Many-to-Many Relationships -====================================== - -This section is moved to :ref:`orm_declarative_relationship_secondary_eval`. - diff --git a/doc/build/orm/extensions/declarative/table_config.rst b/doc/build/orm/extensions/declarative/table_config.rst deleted file mode 100644 index 05ad46d6ccc..00000000000 --- a/doc/build/orm/extensions/declarative/table_config.rst +++ /dev/null @@ -1,24 +0,0 @@ -:orphan: - -.. _declarative_table_args: - -=================== -Table Configuration -=================== - -This section has moved; see :ref:`orm_declarative_table_configuration`. - - -.. _declarative_hybrid_table: - -Using a Hybrid Approach with __table__ -====================================== - -This section has moved; see :ref:`orm_imperative_table_configuration`. - - -Using Reflection with Declarative -================================= - -This section has moved to :ref:`orm_declarative_reflected`. - diff --git a/doc/build/orm/extensions/horizontal_shard.rst b/doc/build/orm/extensions/horizontal_shard.rst deleted file mode 100644 index b0467f1abe5..00000000000 --- a/doc/build/orm/extensions/horizontal_shard.rst +++ /dev/null @@ -1,19 +0,0 @@ -.. _horizontal_sharding_toplevel: - -Horizontal Sharding -=================== - -.. automodule:: sqlalchemy.ext.horizontal_shard - -API Documentation ------------------ - -.. autoclass:: ShardedSession - :members: - -.. autoclass:: set_shard_id - :members: - -.. autoclass:: ShardedQuery - :members: - diff --git a/doc/build/orm/extensions/hybrid.rst b/doc/build/orm/extensions/hybrid.rst deleted file mode 100644 index 9773316d495..00000000000 --- a/doc/build/orm/extensions/hybrid.rst +++ /dev/null @@ -1,21 +0,0 @@ -.. _hybrids_toplevel: - -Hybrid Attributes -================= - -.. automodule:: sqlalchemy.ext.hybrid - -API Reference -------------- - -.. autoclass:: hybrid_method - :members: - -.. autoclass:: hybrid_property - :members: - -.. autoclass:: Comparator - - -.. autoclass:: HybridExtensionType - :members: diff --git a/doc/build/orm/extensions/index.rst b/doc/build/orm/extensions/index.rst deleted file mode 100644 index 0dda58affa6..00000000000 --- a/doc/build/orm/extensions/index.rst +++ /dev/null @@ -1,30 +0,0 @@ -.. _plugins: -.. _sqlalchemy.ext: - -ORM Extensions -============== - -SQLAlchemy has a variety of ORM extensions available, which add additional -functionality to the core behavior. - -The extensions build almost entirely on public core and ORM APIs and users should -be encouraged to read their source code to further their understanding of their -behavior. In particular the "Horizontal Sharding", "Hybrid Attributes", and -"Mutation Tracking" extensions are very succinct. - -.. toctree:: - :maxdepth: 1 - - asyncio - associationproxy - automap - baked - declarative/index - mypy - mutable - orderinglist - horizontal_shard - hybrid - indexable - instrumentation - diff --git a/doc/build/orm/extensions/indexable.rst b/doc/build/orm/extensions/indexable.rst deleted file mode 100644 index 8639e5782d8..00000000000 --- a/doc/build/orm/extensions/indexable.rst +++ /dev/null @@ -1,13 +0,0 @@ -.. _indexable_toplevel: - -Indexable -========= - -.. automodule:: sqlalchemy.ext.indexable - -API Reference -------------- - -.. autoclass:: sqlalchemy.ext.indexable.index_property - :members: - diff --git a/doc/build/orm/extensions/instrumentation.rst b/doc/build/orm/extensions/instrumentation.rst deleted file mode 100644 index 422721ffeeb..00000000000 --- a/doc/build/orm/extensions/instrumentation.rst +++ /dev/null @@ -1,25 +0,0 @@ -.. _instrumentation_toplevel: - -Alternate Class Instrumentation -=============================== - -.. automodule:: sqlalchemy.ext.instrumentation - -API Reference -------------- - -.. autodata:: INSTRUMENTATION_MANAGER - -.. autoclass:: sqlalchemy.orm.instrumentation.InstrumentationFactory - -.. autoclass:: InstrumentationManager - :members: - :undoc-members: - -.. autodata:: instrumentation_finders - -.. autoclass:: ExtendedInstrumentationRegistry - :members: - - - diff --git a/doc/build/orm/extensions/mutable.rst b/doc/build/orm/extensions/mutable.rst deleted file mode 100644 index 3e49b86cb16..00000000000 --- a/doc/build/orm/extensions/mutable.rst +++ /dev/null @@ -1,34 +0,0 @@ -.. _mutable_toplevel: - -Mutation Tracking -================= - -.. automodule:: sqlalchemy.ext.mutable - -API Reference -------------- - -.. autoclass:: MutableBase - :members: _parents, coerce - -.. autoclass:: Mutable - :members: - :inherited-members: - :private-members: - -.. autoclass:: MutableComposite - :members: - -.. autoclass:: MutableDict - :members: - :undoc-members: - -.. autoclass:: MutableList - :members: - :undoc-members: - -.. autoclass:: MutableSet - :members: - :undoc-members: - - diff --git a/doc/build/orm/extensions/mypy.rst b/doc/build/orm/extensions/mypy.rst deleted file mode 100644 index 6639924e94d..00000000000 --- a/doc/build/orm/extensions/mypy.rst +++ /dev/null @@ -1,593 +0,0 @@ -.. _mypy_toplevel: - -Mypy / Pep-484 Support for ORM Mappings -======================================== - -Support for :pep:`484` typing annotations as well as the -MyPy_ type checking tool when using SQLAlchemy -:ref:`declarative ` mappings -that refer to the :class:`_schema.Column` object directly, rather than -the :func:`_orm.mapped_column` construct introduced in SQLAlchemy 2.0. - -.. topic:: SQLAlchemy Mypy Plugin Status Update - - **Updated December 2022** - - For SQLAlchemy 2.0, the Mypy plugin continues to work at the level at which - it reached in the SQLAlchemy 1.4 release. However, SQLAlchemy 2.0, - when released, will feature an - :ref:`all new typing system ` - for ORM Declarative models that removes the need for the Mypy plugin and - delivers much more consistent behavior with generally superior capabilities. - Note that this new capability is **not - part of SQLAlchemy 1.4, it is only in SQLAlchemy 2.0, which is out with beta - releases as of December 2022**. - - The SQLAlchemy Mypy plugin, while it has technically never left the "alpha" - stage, should **now be considered as deprecated in SQLAlchemy 2.0, even - though it is still necessary for full Mypy support when using - SQLAlchemy 1.4**. - - The Mypy plugin itself does not solve the issue of supplying correct typing - with other typing tools such as Pylance/Pyright, Pytype, Pycharm, etc, which - cannot make use of Mypy plugins. Additionally, Mypy plugins are extremely - difficult to develop, maintain and test, as a Mypy plugin must be deeply - integrated with Mypy's internal datastructures and processes, which itself - are not stable within the Mypy project itself. The SQLAlchemy Mypy plugin - has lots of limitations when used with code that deviates from very basic - patterns which are reported regularly. - - For these reasons, new non-regression issues reported against the Mypy - plugin are unlikely to be fixed. When SQLAlchemy 2.0 is released, it will - continue to include the plugin, which will have been updated to continue to - function as well as it does in SQLAlchemy 1.4, when running under SQLAlchemy - 2.0. **Existing code that passes Mypy checks using the plugin with - SQLAlchemy 1.4 installed will continue to pass all checks in SQLAlchemy 2.0 - without any changes required, provided the plugin is still used. The - upcoming API to be released with SQLAlchemy 2.0 is fully backwards - compatible with the SQLAlchemy 1.4 API and Mypy plugin behavior.** - - End-user code that passes all checks under SQLAlchemy 1.4 with the Mypy - plugin will be able to incrementally migrate to the new structures, once - that code is running exclusively on SQLAlchemy 2.0. See the section - :ref:`whatsnew_20_orm_declarative_typing` for background on how this - migration may proceed. - - Code that is running exclusively on SQLAlchemy version - 2.0 and has fully migrated to the new declarative constructs will enjoy full - compliance with pep-484 as well as working correctly within IDEs and other - typing tools, without the need for plugins. - - -Installation ------------- - -For **SQLAlchemy 2.0 only**: No stubs should be installed and packages -like sqlalchemy-stubs_ and sqlalchemy2-stubs_ should be fully uninstalled. - -The Mypy_ package itself is a dependency. - -Mypy may be installed using the "mypy" extras hook using pip: - -.. sourcecode:: text - - pip install sqlalchemy[mypy] - -The plugin itself is configured as described in -`Configuring mypy to use Plugins `_, -using the ``sqlalchemy.ext.mypy.plugin`` module name, such as within -``setup.cfg``:: - - [mypy] - plugins = sqlalchemy.ext.mypy.plugin - -.. _sqlalchemy-stubs: https://github.com/dropbox/sqlalchemy-stubs - -.. _sqlalchemy2-stubs: https://github.com/sqlalchemy/sqlalchemy2-stubs - -What the Plugin Does --------------------- - -The primary purpose of the Mypy plugin is to intercept and alter the static -definition of SQLAlchemy -:ref:`declarative mappings ` so that -they match up to how they are structured after they have been -:term:`instrumented` by their :class:`_orm.Mapper` objects. This allows both -the class structure itself as well as code that uses the class to make sense to -the Mypy tool, which otherwise would not be the case based on how declarative -mappings currently function. The plugin is not unlike similar plugins -that are required for libraries like -`dataclasses `_ which -alter classes dynamically at runtime. - -To cover the major areas where this occurs, consider the following ORM -mapping, using the typical example of the ``User`` class:: - - from sqlalchemy import Column, Integer, String, select - from sqlalchemy.orm import declarative_base - - # "Base" is a class that is created dynamically from the - # declarative_base() function - Base = declarative_base() - - - class User(Base): - __tablename__ = "user" - - id = Column(Integer, primary_key=True) - name = Column(String) - - - # "some_user" is an instance of the User class, which - # accepts "id" and "name" kwargs based on the mapping - some_user = User(id=5, name="user") - - # it has an attribute called .name that's a string - print(f"Username: {some_user.name}") - - # a select() construct makes use of SQL expressions derived from the - # User class itself - select_stmt = select(User).where(User.id.in_([3, 4, 5])).where(User.name.contains("s")) - -Above, the steps that the Mypy extension can take include: - -* Interpretation of the ``Base`` dynamic class generated by - :func:`_orm.declarative_base`, so that classes which inherit from it - are known to be mapped. It also can accommodate the class decorator - approach described at :ref:`orm_declarative_decorator`. - -* Type inference for ORM mapped attributes that are defined in declarative - "inline" style, in the above example the ``id`` and ``name`` attributes of - the ``User`` class. This includes that an instance of ``User`` will use - ``int`` for ``id`` and ``str`` for ``name``. It also includes that when the - ``User.id`` and ``User.name`` class-level attributes are accessed, as they - are above in the ``select()`` statement, they are compatible with SQL - expression behavior, which is derived from the - :class:`_orm.InstrumentedAttribute` attribute descriptor class. - -* Application of an ``__init__()`` method to mapped classes that do not - already include an explicit constructor, which accepts keyword arguments - of specific types for all mapped attributes detected. - -When the Mypy plugin processes the above file, the resulting static class -definition and Python code passed to the Mypy tool is equivalent to the -following:: - - from sqlalchemy import Column, Integer, String, select - from sqlalchemy.orm import Mapped - from sqlalchemy.orm.decl_api import DeclarativeMeta - - - class Base(metaclass=DeclarativeMeta): - __abstract__ = True - - - class User(Base): - __tablename__ = "user" - - id: Mapped[Optional[int]] = Mapped._special_method( - Column(Integer, primary_key=True) - ) - name: Mapped[Optional[str]] = Mapped._special_method(Column(String)) - - def __init__(self, id: Optional[int] = ..., name: Optional[str] = ...) -> None: - ... - - - some_user = User(id=5, name="user") - - print(f"Username: {some_user.name}") - - select_stmt = select(User).where(User.id.in_([3, 4, 5])).where(User.name.contains("s")) - -The key steps which have been taken above include: - -* The ``Base`` class is now defined in terms of the :class:`_orm.DeclarativeMeta` - class explicitly, rather than being a dynamic class. - -* The ``id`` and ``name`` attributes are defined in terms of the - :class:`_orm.Mapped` class, which represents a Python descriptor that - exhibits different behaviors at the class vs. instance levels. The - :class:`_orm.Mapped` class is now the base class for the :class:`_orm.InstrumentedAttribute` - class that is used for all ORM mapped attributes. - - :class:`_orm.Mapped` is defined as a generic class against arbitrary Python - types, meaning specific occurrences of :class:`_orm.Mapped` are associated - with a specific Python type, such as ``Mapped[Optional[int]]`` and - ``Mapped[Optional[str]]`` above. - -* The right-hand side of the declarative mapped attribute assignments are - **removed**, as this resembles the operation that the :class:`_orm.Mapper` - class would normally be doing, which is that it would be replacing these - attributes with specific instances of :class:`_orm.InstrumentedAttribute`. - The original expression is moved into a function call that will allow it to - still be type-checked without conflicting with the left-hand side of the - expression. For Mypy purposes, the left-hand typing annotation is sufficient - for the attribute's behavior to be understood. - -* A type stub for the ``User.__init__()`` method is added which includes the - correct keywords and datatypes. - -Usage ------- - -The following subsections will address individual uses cases that have -so far been considered for pep-484 compliance. - - -Introspection of Columns based on TypeEngine -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -For mapped columns that include an explicit datatype, when they are mapped -as inline attributes, the mapped type will be introspected automatically:: - - class MyClass(Base): - # ... - - id = Column(Integer, primary_key=True) - name = Column("employee_name", String(50), nullable=False) - other_name = Column(String(50)) - -Above, the ultimate class-level datatypes of ``id``, ``name`` and -``other_name`` will be introspected as ``Mapped[Optional[int]]``, -``Mapped[Optional[str]]`` and ``Mapped[Optional[str]]``. The types are by -default **always** considered to be ``Optional``, even for the primary key and -non-nullable column. The reason is because while the database columns "id" and -"name" can't be NULL, the Python attributes ``id`` and ``name`` most certainly -can be ``None`` without an explicit constructor:: - - >>> m1 = MyClass() - >>> m1.id - None - -The types of the above columns can be stated **explicitly**, providing the -two advantages of clearer self-documentation as well as being able to -control which types are optional:: - - class MyClass(Base): - # ... - - id: int = Column(Integer, primary_key=True) - name: str = Column("employee_name", String(50), nullable=False) - other_name: Optional[str] = Column(String(50)) - -The Mypy plugin will accept the above ``int``, ``str`` and ``Optional[str]`` -and convert them to include the ``Mapped[]`` type surrounding them. The -``Mapped[]`` construct may also be used explicitly:: - - from sqlalchemy.orm import Mapped - - - class MyClass(Base): - # ... - - id: Mapped[int] = Column(Integer, primary_key=True) - name: Mapped[str] = Column("employee_name", String(50), nullable=False) - other_name: Mapped[Optional[str]] = Column(String(50)) - -When the type is non-optional, it simply means that the attribute as accessed -from an instance of ``MyClass`` will be considered to be non-None:: - - mc = MyClass(...) - - # will pass mypy --strict - name: str = mc.name - -For optional attributes, Mypy considers that the type must include None -or otherwise be ``Optional``:: - - mc = MyClass(...) - - # will pass mypy --strict - other_name: Optional[str] = mc.name - -Whether or not the mapped attribute is typed as ``Optional``, the -generation of the ``__init__()`` method will **still consider all keywords -to be optional**. This is again matching what the SQLAlchemy ORM actually -does when it creates the constructor, and should not be confused with the -behavior of a validating system such as Python ``dataclasses`` which will -generate a constructor that matches the annotations in terms of optional -vs. required attributes. - - -Columns that Don't have an Explicit Type -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Columns that include a :class:`_schema.ForeignKey` modifier do not need -to specify a datatype in a SQLAlchemy declarative mapping. For -this type of attribute, the Mypy plugin will inform the user that it -needs an explicit type to be sent:: - - # .. other imports - from sqlalchemy.sql.schema import ForeignKey - - Base = declarative_base() - - - class User(Base): - __tablename__ = "user" - - id = Column(Integer, primary_key=True) - name = Column(String) - - - class Address(Base): - __tablename__ = "address" - - id = Column(Integer, primary_key=True) - user_id = Column(ForeignKey("user.id")) - -The plugin will deliver the message as follows: - -.. sourcecode:: text - - $ mypy test3.py --strict - test3.py:20: error: [SQLAlchemy Mypy plugin] Can't infer type from - ORM mapped expression assigned to attribute 'user_id'; please specify a - Python type or Mapped[] on the left hand side. - Found 1 error in 1 file (checked 1 source file) - -To resolve, apply an explicit type annotation to the ``Address.user_id`` -column:: - - class Address(Base): - __tablename__ = "address" - - id = Column(Integer, primary_key=True) - user_id: int = Column(ForeignKey("user.id")) - -Mapping Columns with Imperative Table -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -In :ref:`imperative table style `, the -:class:`_schema.Column` definitions are given inside of a :class:`_schema.Table` -construct which is separate from the mapped attributes themselves. The Mypy -plugin does not consider this :class:`_schema.Table`, but instead supports that -the attributes can be explicitly stated with a complete annotation that -**must** use the :class:`_orm.Mapped` class to identify them as mapped attributes:: - - class MyClass(Base): - __table__ = Table( - "mytable", - Base.metadata, - Column(Integer, primary_key=True), - Column("employee_name", String(50), nullable=False), - Column(String(50)), - ) - - id: Mapped[int] - name: Mapped[str] - other_name: Mapped[Optional[str]] - -The above :class:`_orm.Mapped` annotations are considered as mapped columns and -will be included in the default constructor, as well as provide the correct -typing profile for ``MyClass`` both at the class level and the instance level. - -Mapping Relationships -^^^^^^^^^^^^^^^^^^^^^^ - -The plugin has limited support for using type inference to detect the types -for relationships. For all those cases where it can't detect the type, -it will emit an informative error message, and in all cases the appropriate -type may be provided explicitly, either with the :class:`_orm.Mapped` -class or optionally omitting it for an inline declaration. The plugin -also needs to determine whether or not the relationship refers to a collection -or a scalar, and for that it relies upon the explicit value of -the :paramref:`_orm.relationship.uselist` and/or :paramref:`_orm.relationship.collection_class` -parameters. An explicit type is needed if neither of these parameters are -present, as well as if the target type of the :func:`_orm.relationship` -is a string or callable, and not a class:: - - class User(Base): - __tablename__ = "user" - - id = Column(Integer, primary_key=True) - name = Column(String) - - - class Address(Base): - __tablename__ = "address" - - id = Column(Integer, primary_key=True) - user_id: int = Column(ForeignKey("user.id")) - - user = relationship(User) - -The above mapping will produce the following error: - -.. sourcecode:: text - - test3.py:22: error: [SQLAlchemy Mypy plugin] Can't infer scalar or - collection for ORM mapped expression assigned to attribute 'user' - if both 'uselist' and 'collection_class' arguments are absent from the - relationship(); please specify a type annotation on the left hand side. - Found 1 error in 1 file (checked 1 source file) - -The error can be resolved either by using ``relationship(User, uselist=False)`` -or by providing the type, in this case the scalar ``User`` object:: - - class Address(Base): - __tablename__ = "address" - - id = Column(Integer, primary_key=True) - user_id: int = Column(ForeignKey("user.id")) - - user: User = relationship(User) - -For collections, a similar pattern applies, where in the absence of -``uselist=True`` or a :paramref:`_orm.relationship.collection_class`, -a collection annotation such as ``List`` may be used. It is also fully -appropriate to use the string name of the class in the annotation as supported -by pep-484, ensuring the class is imported with in -the `TYPE_CHECKING block `_ -as appropriate:: - - from typing import TYPE_CHECKING, List - - from .mymodel import Base - - if TYPE_CHECKING: - # if the target of the relationship is in another module - # that cannot normally be imported at runtime - from .myaddressmodel import Address - - - class User(Base): - __tablename__ = "user" - - id = Column(Integer, primary_key=True) - name = Column(String) - addresses: List["Address"] = relationship("Address") - -As is the case with columns, the :class:`_orm.Mapped` class may also be -applied explicitly:: - - class User(Base): - __tablename__ = "user" - - id = Column(Integer, primary_key=True) - name = Column(String) - - addresses: Mapped[List["Address"]] = relationship("Address", back_populates="user") - - - class Address(Base): - __tablename__ = "address" - - id = Column(Integer, primary_key=True) - user_id: int = Column(ForeignKey("user.id")) - - user: Mapped[User] = relationship(User, back_populates="addresses") - -.. _mypy_declarative_mixins: - -Using @declared_attr and Declarative Mixins -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The :class:`_orm.declared_attr` class allows Declarative mapped attributes to -be declared in class level functions, and is particularly useful when using -:ref:`declarative mixins `. For these functions, the return -type of the function should be annotated using either the ``Mapped[]`` -construct or by indicating the exact kind of object returned by the function. -Additionally, "mixin" classes that are not otherwise mapped (i.e. don't extend -from a :func:`_orm.declarative_base` class nor are they mapped with a method -such as :meth:`_orm.registry.mapped`) should be decorated with the -:func:`_orm.declarative_mixin` decorator, which provides a hint to the Mypy -plugin that a particular class intends to serve as a declarative mixin:: - - from sqlalchemy.orm import declarative_mixin, declared_attr - - - @declarative_mixin - class HasUpdatedAt: - @declared_attr - def updated_at(cls) -> Column[DateTime]: # uses Column - return Column(DateTime) - - - @declarative_mixin - class HasCompany: - @declared_attr - def company_id(cls) -> Mapped[int]: # uses Mapped - return Column(ForeignKey("company.id")) - - @declared_attr - def company(cls) -> Mapped["Company"]: - return relationship("Company") - - - class Employee(HasUpdatedAt, HasCompany, Base): - __tablename__ = "employee" - - id = Column(Integer, primary_key=True) - name = Column(String) - -Note the mismatch between the actual return type of a method like -``HasCompany.company`` vs. what is annotated. The Mypy plugin converts -all ``@declared_attr`` functions into simple annotated attributes to avoid -this complexity:: - - # what Mypy sees - class HasCompany: - company_id: Mapped[int] - company: Mapped["Company"] - -Combining with Dataclasses or Other Type-Sensitive Attribute Systems -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The examples of Python dataclasses integration at :ref:`orm_declarative_dataclasses` -presents a problem; Python dataclasses expect an explicit type that it will -use to build the class, and the value given in each assignment statement -is significant. That is, a class as follows has to be stated exactly -as it is in order to be accepted by dataclasses:: - - mapper_registry: registry = registry() - - - @mapper_registry.mapped - @dataclass - class User: - __table__ = Table( - "user", - mapper_registry.metadata, - Column("id", Integer, primary_key=True), - Column("name", String(50)), - Column("fullname", String(50)), - Column("nickname", String(12)), - ) - id: int = field(init=False) - name: Optional[str] = None - fullname: Optional[str] = None - nickname: Optional[str] = None - addresses: List[Address] = field(default_factory=list) - - __mapper_args__ = { # type: ignore - "properties": {"addresses": relationship("Address")} - } - -We can't apply our ``Mapped[]`` types to the attributes ``id``, ``name``, -etc. because they will be rejected by the ``@dataclass`` decorator. Additionally, -Mypy has another plugin for dataclasses explicitly which can also get in the -way of what we're doing. - -The above class will actually pass Mypy's type checking without issue; the -only thing we are missing is the ability for attributes on ``User`` to be -used in SQL expressions, such as:: - - stmt = select(User.name).where(User.id.in_([1, 2, 3])) - -To provide a workaround for this, the Mypy plugin has an additional feature -whereby we can specify an extra attribute ``_mypy_mapped_attrs``, that is -a list that encloses the class-level objects or their string names. -This attribute can be conditional within the ``TYPE_CHECKING`` variable:: - - @mapper_registry.mapped - @dataclass - class User: - __table__ = Table( - "user", - mapper_registry.metadata, - Column("id", Integer, primary_key=True), - Column("name", String(50)), - Column("fullname", String(50)), - Column("nickname", String(12)), - ) - id: int = field(init=False) - name: Optional[str] = None - fullname: Optional[str] - nickname: Optional[str] - addresses: List[Address] = field(default_factory=list) - - if TYPE_CHECKING: - _mypy_mapped_attrs = [id, name, "fullname", "nickname", addresses] - - __mapper_args__ = { # type: ignore - "properties": {"addresses": relationship("Address")} - } - -With the above recipe, the attributes listed in ``_mypy_mapped_attrs`` -will be applied with the :class:`_orm.Mapped` typing information so that the -``User`` class will behave as a SQLAlchemy mapped class when used in a -class-bound context. - -.. _Mypy: https://mypy.readthedocs.io/ diff --git a/doc/build/orm/extensions/orderinglist.rst b/doc/build/orm/extensions/orderinglist.rst deleted file mode 100644 index 19599a57828..00000000000 --- a/doc/build/orm/extensions/orderinglist.rst +++ /dev/null @@ -1,18 +0,0 @@ -Ordering List -============= - -.. automodule:: sqlalchemy.ext.orderinglist - -API Reference -------------- - -.. autofunction:: ordering_list - -.. autofunction:: count_from_0 - -.. autofunction:: count_from_1 - -.. autofunction:: count_from_n_factory - -.. autoclass:: OrderingList - :members: diff --git a/doc/build/orm/index.rst b/doc/build/orm/index.rst index c3fa9929b35..d14dba571f1 100644 --- a/doc/build/orm/index.rst +++ b/doc/build/orm/index.rst @@ -17,6 +17,4 @@ tutorial. queryguide/index session extending - extensions/index - examples diff --git a/doc/build/orm/inheritance.rst b/doc/build/orm/inheritance.rst index 6746a0cda56..1645717a4f1 100644 --- a/doc/build/orm/inheritance.rst +++ b/doc/build/orm/inheritance.rst @@ -21,7 +21,7 @@ return objects of multiple types. :ref:`loading_joined_inheritance` - in the :ref:`queryguide_toplevel` - :ref:`examples_inheritance` - complete examples of joined, single and + ref_examples_inheritance - complete examples of joined, single and concrete inheritance .. _joined_inheritance: diff --git a/doc/build/orm/internals.rst b/doc/build/orm/internals.rst index 9bb7e83a490..a6ffa663f56 100644 --- a/doc/build/orm/internals.rst +++ b/doc/build/orm/internals.rst @@ -22,9 +22,6 @@ sections, are listed here. .. autoclass:: Composite -.. autoclass:: CompositeProperty - :members: - .. autoclass:: AttributeEventToken :members: diff --git a/doc/build/orm/join_conditions.rst b/doc/build/orm/join_conditions.rst index 79c3e98adf7..7a21a13d4d1 100644 --- a/doc/build/orm/join_conditions.rst +++ b/doc/build/orm/join_conditions.rst @@ -105,7 +105,7 @@ one :class:`_schema.Column` we need:: .. warning:: When passed as a Python-evaluable string, the :paramref:`_orm.relationship.foreign_keys` argument is interpreted using Python's ``eval()`` function. **DO NOT PASS UNTRUSTED INPUT TO THIS STRING**. See - :ref:`declarative_relationship_eval` for details on declarative + ref_declarative_relationship_eval for details on declarative evaluation of :func:`_orm.relationship` arguments. @@ -168,7 +168,7 @@ argument. :paramref:`_orm.relationship.primaryjoin` argument is interpreted using Python's ``eval()`` function. **DO NOT PASS UNTRUSTED INPUT TO THIS STRING**. See - :ref:`declarative_relationship_eval` for details on declarative + ref_declarative_relationship_eval for details on declarative evaluation of :func:`_orm.relationship` arguments. @@ -600,7 +600,7 @@ use the string name of the table as it is present in the :class:`_schema.MetaDat :paramref:`_orm.relationship.primaryjoin` and :paramref:`_orm.relationship.secondaryjoin` arguments are interpreted using Python's ``eval()`` function. **DO NOT PASS UNTRUSTED INPUT TO THESE - STRINGS**. See :ref:`declarative_relationship_eval` for details on + STRINGS**. See ref_declarative_relationship_eval for details on declarative evaluation of :func:`_orm.relationship` arguments. diff --git a/doc/build/orm/large_collections.rst b/doc/build/orm/large_collections.rst deleted file mode 100644 index ef1e651e1f4..00000000000 --- a/doc/build/orm/large_collections.rst +++ /dev/null @@ -1,686 +0,0 @@ -.. highlight:: pycon+sql -.. doctest-enable - -.. currentmodule:: sqlalchemy.orm - -.. _largecollections: - -Working with Large Collections -============================== - -The default behavior of :func:`_orm.relationship` is to fully load -the contents of collections into memory, based on a configured -:ref:`loader strategy ` that controls -when and how these contents are loaded from the database. Related collections -may be loaded into memory not just when they are accessed, or eagerly loaded, -but in most cases will require population when the collection -itself is mutated, as well as in cases where the owning object is to be -deleted by the unit of work system. - -When a related collection is potentially very large, it may not be feasible -for such a collection to be populated into memory under any circumstances, -as the operation may be overly consuming of time, network and memory -resources. - -This section includes API features intended to allow :func:`_orm.relationship` -to be used with large collections while maintaining adequate performance. - - -.. _write_only_relationship: - -Write Only Relationships ------------------------- - -The **write only** loader strategy is the primary means of configuring a -:func:`_orm.relationship` that will remain writeable, but will not load -its contents into memory. A write-only ORM configuration in modern -type-annotated Declarative form is illustrated below:: - - >>> from decimal import Decimal - >>> from datetime import datetime - - >>> from sqlalchemy import ForeignKey - >>> from sqlalchemy import func - >>> from sqlalchemy.orm import DeclarativeBase - >>> from sqlalchemy.orm import Mapped - >>> from sqlalchemy.orm import mapped_column - >>> from sqlalchemy.orm import relationship - >>> from sqlalchemy.orm import Session - >>> from sqlalchemy.orm import WriteOnlyMapped - - >>> class Base(DeclarativeBase): - ... pass - - >>> class Account(Base): - ... __tablename__ = "account" - ... id: Mapped[int] = mapped_column(primary_key=True) - ... identifier: Mapped[str] - ... - ... account_transactions: WriteOnlyMapped["AccountTransaction"] = relationship( - ... cascade="all, delete-orphan", - ... passive_deletes=True, - ... order_by="AccountTransaction.timestamp", - ... ) - ... - ... def __repr__(self): - ... return f"Account(identifier={self.identifier!r})" - - >>> class AccountTransaction(Base): - ... __tablename__ = "account_transaction" - ... id: Mapped[int] = mapped_column(primary_key=True) - ... account_id: Mapped[int] = mapped_column( - ... ForeignKey("account.id", ondelete="cascade") - ... ) - ... description: Mapped[str] - ... amount: Mapped[Decimal] - ... timestamp: Mapped[datetime] = mapped_column(default=func.now()) - ... - ... def __repr__(self): - ... return ( - ... f"AccountTransaction(amount={self.amount:.2f}, " - ... f"timestamp={self.timestamp.isoformat()!r})" - ... ) - ... - ... __mapper_args__ = {"eager_defaults": True} - - -.. setup code not for display - - >>> from sqlalchemy import create_engine - >>> from sqlalchemy import event - >>> engine = create_engine("sqlite://", echo=True) - >>> @event.listens_for(engine, "connect") - ... def set_sqlite_pragma(dbapi_connection, connection_record): - ... cursor = dbapi_connection.cursor() - ... cursor.execute("PRAGMA foreign_keys=ON") - ... cursor.close() - - >>> Base.metadata.create_all(engine) - BEGIN... - - -Above, the ``account_transactions`` relationship is configured not using the -ordinary :class:`.Mapped` annotation, but instead -using the :class:`.WriteOnlyMapped` type annotation, which at runtime will -assign the :ref:`loader strategy ` of -``lazy="write_only"`` to the target :func:`_orm.relationship`. -The :class:`.WriteOnlyMapped` annotation is an -alternative form of the :class:`_orm.Mapped` annotation which indicate the use -of the :class:`_orm.WriteOnlyCollection` collection type on instances of the -object. - -The above :func:`_orm.relationship` configuration also includes several -elements that are specific to what action to take when ``Account`` objects -are deleted, as well as when ``AccountTransaction`` objects are removed from the -``account_transactions`` collection. These elements are: - -* ``passive_deletes=True`` - allows the :term:`unit of work` to forego having - to load the collection when ``Account`` is deleted; see - :ref:`passive_deletes`. -* ``ondelete="cascade"`` configured on the :class:`.ForeignKey` constraint. - This is also detailed at :ref:`passive_deletes`. -* ``cascade="all, delete-orphan"`` - instructs the :term:`unit of work` to - delete ``AccountTransaction`` objects when they are removed from the - collection. See :ref:`cascade_delete_orphan` in the :ref:`unitofwork_cascades` - document. - -.. versionadded:: 2.0 Added "Write only" relationship loaders. - - -Creating and Persisting New Write Only Collections -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The write-only collection allows for direct assignment of the collection -as a whole **only** for :term:`transient` or :term:`pending` objects. -With our above mapping, this indicates we can create a new ``Account`` -object with a sequence of ``AccountTransaction`` objects to be added -to a :class:`_orm.Session`. Any Python iterable may be used as the -source of objects to start, where below we use a Python ``list``:: - - >>> new_account = Account( - ... identifier="account_01", - ... account_transactions=[ - ... AccountTransaction(description="initial deposit", amount=Decimal("500.00")), - ... AccountTransaction(description="transfer", amount=Decimal("1000.00")), - ... AccountTransaction(description="withdrawal", amount=Decimal("-29.50")), - ... ], - ... ) - - >>> with Session(engine) as session: - ... session.add(new_account) - ... session.commit() - {execsql}BEGIN (implicit) - INSERT INTO account (identifier) VALUES (?) - [...] ('account_01',) - INSERT INTO account_transaction (account_id, description, amount, timestamp) - VALUES (?, ?, ?, CURRENT_TIMESTAMP) RETURNING id, timestamp - [... (insertmanyvalues) 1/3 (ordered; batch not supported)] (1, 'initial deposit', 500.0) - INSERT INTO account_transaction (account_id, description, amount, timestamp) - VALUES (?, ?, ?, CURRENT_TIMESTAMP) RETURNING id, timestamp - [insertmanyvalues 2/3 (ordered; batch not supported)] (1, 'transfer', 1000.0) - INSERT INTO account_transaction (account_id, description, amount, timestamp) - VALUES (?, ?, ?, CURRENT_TIMESTAMP) RETURNING id, timestamp - [insertmanyvalues 3/3 (ordered; batch not supported)] (1, 'withdrawal', -29.5) - COMMIT - - -Once an object is database-persisted (i.e. in the :term:`persistent` or -:term:`detached` state), the collection has the ability to be extended with new -items as well as the ability for individual items to be removed. However, the -collection may **no longer be re-assigned with a full replacement collection**, -as such an operation requires that the previous collection is fully -loaded into memory in order to reconcile the old entries with the new ones:: - - >>> new_account.account_transactions = [ - ... AccountTransaction(description="some transaction", amount=Decimal("10.00")) - ... ] - Traceback (most recent call last): - ... - sqlalchemy.exc.InvalidRequestError: Collection "Account.account_transactions" does not - support implicit iteration; collection replacement operations can't be used - -Adding New Items to an Existing Collection -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -For write-only collections of persistent objects, -modifications to the collection using :term:`unit of work` processes may proceed -only by using the :meth:`.WriteOnlyCollection.add`, -:meth:`.WriteOnlyCollection.add_all` and :meth:`.WriteOnlyCollection.remove` -methods:: - - >>> from sqlalchemy import select - >>> session = Session(engine, expire_on_commit=False) - >>> existing_account = session.scalar(select(Account).filter_by(identifier="account_01")) - {execsql}BEGIN (implicit) - SELECT account.id, account.identifier - FROM account - WHERE account.identifier = ? - [...] ('account_01',) - {stop} - >>> existing_account.account_transactions.add_all( - ... [ - ... AccountTransaction(description="paycheck", amount=Decimal("2000.00")), - ... AccountTransaction(description="rent", amount=Decimal("-800.00")), - ... ] - ... ) - >>> session.commit() - {execsql}INSERT INTO account_transaction (account_id, description, amount, timestamp) - VALUES (?, ?, ?, CURRENT_TIMESTAMP) RETURNING id, timestamp - [... (insertmanyvalues) 1/2 (ordered; batch not supported)] (1, 'paycheck', 2000.0) - INSERT INTO account_transaction (account_id, description, amount, timestamp) - VALUES (?, ?, ?, CURRENT_TIMESTAMP) RETURNING id, timestamp - [insertmanyvalues 2/2 (ordered; batch not supported)] (1, 'rent', -800.0) - COMMIT - - -The items added above are held in a pending queue within the -:class:`_orm.Session` until the next flush, at which point they are INSERTed -into the database, assuming the added objects were previously :term:`transient`. - -Querying Items -~~~~~~~~~~~~~~ - -The :class:`_orm.WriteOnlyCollection` does not at any point store a reference -to the current contents of the collection, nor does it have any behavior where -it would directly emit a SELECT to the database in order to load them; the -overriding assumption is that the collection may contain many thousands or -millions of rows, and should never be fully loaded into memory as a side effect -of any other operation. - -Instead, the :class:`_orm.WriteOnlyCollection` includes SQL-generating helpers -such as :meth:`_orm.WriteOnlyCollection.select`, which will generate -a :class:`.Select` construct pre-configured with the correct WHERE / FROM -criteria for the current parent row, which can then be further modified in -order to SELECT any range of rows desired, as well as invoked using features -like :ref:`server side cursors ` for processes that -wish to iterate through the full collection in a memory-efficient manner. - -The statement generated is illustrated below. Note it also includes ORDER BY -criteria, indicated in the example mapping by the -:paramref:`_orm.relationship.order_by` parameter of :func:`_orm.relationship`; -this criteria would be omitted if the parameter were not configured:: - - >>> print(existing_account.account_transactions.select()) - {printsql}SELECT account_transaction.id, account_transaction.account_id, account_transaction.description, - account_transaction.amount, account_transaction.timestamp - FROM account_transaction - WHERE :param_1 = account_transaction.account_id ORDER BY account_transaction.timestamp - -We may use this :class:`.Select` construct along with the :class:`_orm.Session` -in order to query for ``AccountTransaction`` objects, most easily using the -:meth:`_orm.Session.scalars` method that will return a :class:`.Result` that -yields ORM objects directly. It's typical, though not required, that the -:class:`.Select` would be modified further to limit the records returned; in -the example below, additional WHERE criteria to load only "debit" account -transactions is added, along with "LIMIT 10" to retrieve only the first ten -rows:: - - >>> account_transactions = session.scalars( - ... existing_account.account_transactions.select() - ... .where(AccountTransaction.amount < 0) - ... .limit(10) - ... ).all() - {execsql}BEGIN (implicit) - SELECT account_transaction.id, account_transaction.account_id, account_transaction.description, - account_transaction.amount, account_transaction.timestamp - FROM account_transaction - WHERE ? = account_transaction.account_id AND account_transaction.amount < ? - ORDER BY account_transaction.timestamp LIMIT ? OFFSET ? - [...] (1, 0, 10, 0) - {stop}>>> print(account_transactions) - [AccountTransaction(amount=-29.50, timestamp='...'), AccountTransaction(amount=-800.00, timestamp='...')] - - -Removing Items -~~~~~~~~~~~~~~ - -Individual items that are loaded in the :term:`persistent` -state against the current :class:`_orm.Session` may be marked for removal -from the collection using the :meth:`.WriteOnlyCollection.remove` method. -The flush process will implicitly consider the object to be already part -of the collection when the operation proceeds. The example below -illustrates removal of an individual ``AccountTransaction`` item, -which per :ref:`cascade ` settings results in a -DELETE of that row:: - - >>> existing_transaction = account_transactions[0] - >>> existing_account.account_transactions.remove(existing_transaction) - >>> session.commit() - {execsql}DELETE FROM account_transaction WHERE account_transaction.id = ? - [...] (3,) - COMMIT - -As with any ORM-mapped collection, object removal may proceed either to -de-associate the object from the collection while leaving the object present in -the database, or may issue a DELETE for its row, based on the -:ref:`cascade_delete_orphan` configuration of the :func:`_orm.relationship`. - -Collection removal without deletion involves setting foreign key columns to -NULL for a :ref:`one-to-many ` relationship, or -deleting the corresponding association row for a -:ref:`many-to-many ` relationship. - - - -Bulk INSERT of New Items -~~~~~~~~~~~~~~~~~~~~~~~~ - -The :class:`.WriteOnlyCollection` can generate DML constructs such as -:class:`_dml.Insert` objects, which may be used in an ORM context to -produce bulk insert behavior. See the section -:ref:`orm_queryguide_bulk_insert` for an overview of ORM bulk inserts. - -One to Many Collections -^^^^^^^^^^^^^^^^^^^^^^^ -For a **regular one to many collection only**, the :meth:`.WriteOnlyCollection.insert` -method will produce an :class:`_dml.Insert` construct which is pre-established with -VALUES criteria corresponding to the parent object. As this VALUES criteria -is entirely against the related table, the statement can be used to -INSERT new rows that will at the same time become new records in the -related collection:: - - >>> session.execute( - ... existing_account.account_transactions.insert(), - ... [ - ... {"description": "transaction 1", "amount": Decimal("47.50")}, - ... {"description": "transaction 2", "amount": Decimal("-501.25")}, - ... {"description": "transaction 3", "amount": Decimal("1800.00")}, - ... {"description": "transaction 4", "amount": Decimal("-300.00")}, - ... ], - ... ) - {execsql}BEGIN (implicit) - INSERT INTO account_transaction (account_id, description, amount, timestamp) VALUES (?, ?, ?, CURRENT_TIMESTAMP) - [...] [(1, 'transaction 1', 47.5), (1, 'transaction 2', -501.25), (1, 'transaction 3', 1800.0), (1, 'transaction 4', -300.0)] - <...> - {stop} - >>> session.commit() - COMMIT - -.. seealso:: - - :ref:`orm_queryguide_bulk_insert` - in the :ref:`queryguide_toplevel` - - :ref:`relationship_patterns_o2m` - at :ref:`relationship_patterns` - - -Many to Many Collections -^^^^^^^^^^^^^^^^^^^^^^^^ - -For a **many to many collection**, the relationship between two classes -involves a third table that is configured using the -:paramref:`_orm.relationship.secondary` parameter of :class:`_orm.relationship`. -To bulk insert rows into a collection of this type using -:class:`.WriteOnlyCollection`, the new records may be bulk-inserted separately -first, retrieved using RETURNING, and those records then passed to the -:meth:`.WriteOnlyCollection.add_all` method where the unit of work process -will proceed to persist them as part of the collection. - -Supposing a class ``BankAudit`` referred to many ``AccountTransaction`` -records using a many-to-many table:: - - >>> from sqlalchemy import Table, Column - >>> audit_to_transaction = Table( - ... "audit_transaction", - ... Base.metadata, - ... Column("audit_id", ForeignKey("audit.id", ondelete="CASCADE"), primary_key=True), - ... Column( - ... "transaction_id", - ... ForeignKey("account_transaction.id", ondelete="CASCADE"), - ... primary_key=True, - ... ), - ... ) - >>> class BankAudit(Base): - ... __tablename__ = "audit" - ... id: Mapped[int] = mapped_column(primary_key=True) - ... account_transactions: WriteOnlyMapped["AccountTransaction"] = relationship( - ... secondary=audit_to_transaction, passive_deletes=True - ... ) - -.. setup code not for display - - >>> Base.metadata.create_all(engine) - BEGIN... - -To illustrate the two operations, we add more ``AccountTransaction`` objects -using bulk insert, which we retrieve using RETURNING by adding -``returning(AccountTransaction)`` to the bulk INSERT statement (note that -we could just as easily use existing ``AccountTransaction`` objects as well):: - - >>> new_transactions = session.scalars( - ... existing_account.account_transactions.insert().returning(AccountTransaction), - ... [ - ... {"description": "odd trans 1", "amount": Decimal("50000.00")}, - ... {"description": "odd trans 2", "amount": Decimal("25000.00")}, - ... {"description": "odd trans 3", "amount": Decimal("45.00")}, - ... ], - ... ).all() - {execsql}BEGIN (implicit) - INSERT INTO account_transaction (account_id, description, amount, timestamp) VALUES - (?, ?, ?, CURRENT_TIMESTAMP), (?, ?, ?, CURRENT_TIMESTAMP), (?, ?, ?, CURRENT_TIMESTAMP) - RETURNING id, account_id, description, amount, timestamp - [...] (1, 'odd trans 1', 50000.0, 1, 'odd trans 2', 25000.0, 1, 'odd trans 3', 45.0) - {stop} - -With a list of ``AccountTransaction`` objects ready, the -:meth:`_orm.WriteOnlyCollection.add_all` method is used to associate many rows -at once with a new ``BankAudit`` object:: - - >>> bank_audit = BankAudit() - >>> session.add(bank_audit) - >>> bank_audit.account_transactions.add_all(new_transactions) - >>> session.commit() - {execsql}INSERT INTO audit DEFAULT VALUES - [...] () - INSERT INTO audit_transaction (audit_id, transaction_id) VALUES (?, ?) - [...] [(1, 10), (1, 11), (1, 12)] - COMMIT - -.. seealso:: - - :ref:`orm_queryguide_bulk_insert` - in the :ref:`queryguide_toplevel` - - :ref:`relationships_many_to_many` - at :ref:`relationship_patterns` - - -Bulk UPDATE and DELETE of Items -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -In a similar way in which :class:`.WriteOnlyCollection` can generate -:class:`.Select` constructs with WHERE criteria pre-established, it can -also generate :class:`.Update` and :class:`.Delete` constructs with that -same WHERE criteria, to allow criteria-oriented UPDATE and DELETE statements -against the elements in a large collection. - -One To Many Collections -^^^^^^^^^^^^^^^^^^^^^^^ - -As is the case with INSERT, this feature is most straightforward with **one -to many collections**. - -In the example below, the :meth:`.WriteOnlyCollection.update` method is used -to generate an UPDATE statement is emitted against the elements -in the collection, locating rows where the "amount" is equal to ``-800`` and -adding the amount of ``200`` to them:: - - >>> session.execute( - ... existing_account.account_transactions.update() - ... .values(amount=AccountTransaction.amount + 200) - ... .where(AccountTransaction.amount == -800), - ... ) - {execsql}BEGIN (implicit) - UPDATE account_transaction SET amount=(account_transaction.amount + ?) - WHERE ? = account_transaction.account_id AND account_transaction.amount = ? - [...] (200, 1, -800) - {stop}<...> - -In a similar way, :meth:`.WriteOnlyCollection.delete` will produce a -DELETE statement that is invoked in the same way:: - - >>> session.execute( - ... existing_account.account_transactions.delete().where( - ... AccountTransaction.amount.between(0, 30) - ... ), - ... ) - {execsql}DELETE FROM account_transaction WHERE ? = account_transaction.account_id - AND account_transaction.amount BETWEEN ? AND ? RETURNING id - [...] (1, 0, 30) - <...> - {stop} - -Many to Many Collections -^^^^^^^^^^^^^^^^^^^^^^^^ - -.. tip:: - - The techniques here involve multi-table UPDATE expressions, which are - slightly more advanced. - -For bulk UPDATE and DELETE of **many to many collections**, in order for -an UPDATE or DELETE statement to relate to the primary key of the -parent object, the association table must be explicitly part of the -UPDATE/DELETE statement, which requires -either that the backend includes supports for non-standard SQL syntaxes, -or extra explicit steps when constructing the UPDATE or DELETE statement. - -For backends that support multi-table versions of UPDATE, the -:meth:`.WriteOnlyCollection.update` method should work without extra steps -for a many-to-many collection, as in the example below where an UPDATE -is emitted against ``AccountTransaction`` objects in terms of the -many-to-many ``BankAudit.account_transactions`` collection:: - - >>> session.execute( - ... bank_audit.account_transactions.update().values( - ... description=AccountTransaction.description + " (audited)" - ... ) - ... ) - {execsql}UPDATE account_transaction SET description=(account_transaction.description || ?) - FROM audit_transaction WHERE ? = audit_transaction.audit_id - AND account_transaction.id = audit_transaction.transaction_id RETURNING id - [...] (' (audited)', 1) - {stop}<...> - -The above statement automatically makes use of "UPDATE..FROM" syntax, -supported by SQLite and others, to name the additional ``audit_transaction`` -table in the WHERE clause. - -To UPDATE or DELETE a many-to-many collection where multi-table syntax is -not available, the many-to-many criteria may be moved into SELECT that -for example may be combined with IN to match rows. -The :class:`.WriteOnlyCollection` still helps us here, as we use the -:meth:`.WriteOnlyCollection.select` method to generate this SELECT for -us, making use of the :meth:`_sql.Select.with_only_columns` method to -produce a :term:`scalar subquery`:: - - >>> from sqlalchemy import update - >>> subq = bank_audit.account_transactions.select().with_only_columns(AccountTransaction.id) - >>> session.execute( - ... update(AccountTransaction) - ... .values(description=AccountTransaction.description + " (audited)") - ... .where(AccountTransaction.id.in_(subq)) - ... ) - {execsql}UPDATE account_transaction SET description=(account_transaction.description || ?) - WHERE account_transaction.id IN (SELECT account_transaction.id - FROM audit_transaction - WHERE ? = audit_transaction.audit_id AND account_transaction.id = audit_transaction.transaction_id) - RETURNING id - [...] (' (audited)', 1) - <...> - -Write Only Collections - API Documentation -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - -.. autoclass:: sqlalchemy.orm.WriteOnlyCollection - :members: - :inherited-members: - -.. autoclass:: sqlalchemy.orm.WriteOnlyMapped - :members: - -.. highlight:: python -.. doctest-disable - -.. _dynamic_relationship: - -Dynamic Relationship Loaders ----------------------------- - -.. legacy:: The "dynamic" lazy loader strategy is the legacy form of what is - now the "write_only" strategy described in the section - :ref:`write_only_relationship`. - - The "dynamic" strategy produces a legacy :class:`_orm.Query` object from the - related collection. However, a major drawback of "dynamic" relationships is - that there are several cases where the collection will fully iterate, some - of which are non-obvious, which can only be prevented with careful - programming and testing on a case-by-case basis. Therefore, for truly large - collection management, the :class:`_orm.WriteOnlyCollection` should be - preferred. - - The dynamic loader is also not compatible with the :ref:`asyncio_toplevel` - extension. It can be used with some limitations, as indicated in - :ref:`Asyncio dynamic guidelines `, but again the - :class:`_orm.WriteOnlyCollection`, which is fully compatible with asyncio, - should be preferred. - -The dynamic relationship strategy allows configuration of a -:func:`_orm.relationship` which when accessed on an instance will return a -legacy :class:`_orm.Query` object in place of the collection. The -:class:`_orm.Query` can then be modified further so that the database -collection may be iterated based on filtering criteria. The returned -:class:`_orm.Query` object is an instance of :class:`_orm.AppenderQuery`, which -combines the loading and iteration behavior of :class:`_orm.Query` along with -rudimentary collection mutation methods such as -:meth:`_orm.AppenderQuery.append` and :meth:`_orm.AppenderQuery.remove`. - -The "dynamic" loader strategy may be configured with -type-annotated Declarative form using the :class:`_orm.DynamicMapped` -annotation class:: - - from sqlalchemy.orm import DynamicMapped - - - class User(Base): - __tablename__ = "user" - - id: Mapped[int] = mapped_column(primary_key=True) - posts: DynamicMapped[Post] = relationship() - -Above, the ``User.posts`` collection on an individual ``User`` object -will return the :class:`_orm.AppenderQuery` object, which is a subclass -of :class:`_orm.Query` that also supports basic collection mutation -operations:: - - - jack = session.get(User, id) - - # filter Jack's blog posts - posts = jack.posts.filter(Post.headline == "this is a post") - - # apply array slices - posts = jack.posts[5:20] - -The dynamic relationship supports limited write operations, via the -:meth:`_orm.AppenderQuery.append` and :meth:`_orm.AppenderQuery.remove` methods:: - - oldpost = jack.posts.filter(Post.headline == "old post").one() - jack.posts.remove(oldpost) - - jack.posts.append(Post("new post")) - -Since the read side of the dynamic relationship always queries the -database, changes to the underlying collection will not be visible -until the data has been flushed. However, as long as "autoflush" is -enabled on the :class:`.Session` in use, this will occur -automatically each time the collection is about to emit a -query. - - -Dynamic Relationship Loaders - API -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. autoclass:: sqlalchemy.orm.AppenderQuery - :members: - :inherited-members: Query - -.. autoclass:: sqlalchemy.orm.DynamicMapped - :members: - -.. _collections_raiseload: - -Setting RaiseLoad ------------------ - -A "raise"-loaded relationship will raise an -:exc:`~sqlalchemy.exc.InvalidRequestError` where the attribute would normally -emit a lazy load:: - - class MyClass(Base): - __tablename__ = "some_table" - - # ... - - children: Mapped[List[MyRelatedClass]] = relationship(lazy="raise") - -Above, attribute access on the ``children`` collection will raise an exception -if it was not previously populated. This includes read access but for -collections will also affect write access, as collections can't be mutated -without first loading them. The rationale for this is to ensure that an -application is not emitting any unexpected lazy loads within a certain context. -Rather than having to read through SQL logs to determine that all necessary -attributes were eager loaded, the "raise" strategy will cause unloaded -attributes to raise immediately if accessed. The raise strategy is -also available on a query option basis using the :func:`_orm.raiseload` -loader option. - -.. seealso:: - - :ref:`prevent_lazy_with_raiseload` - -Using Passive Deletes ---------------------- - -An important aspect of collection management in SQLAlchemy is that when an -object that refers to a collection is deleted, SQLAlchemy needs to consider the -objects that are inside this collection. Those objects will need to be -de-associated from the parent, which for a one-to-many collection would mean -that foreign key columns are set to NULL, or based on -:ref:`cascade ` settings, may instead want to emit a -DELETE for these rows. - -The :term:`unit of work` process only considers objects on a row-by-row basis, -meaning a DELETE operation implies that all rows within a collection must be -fully loaded into memory inside the flush process. This is not feasible for -large collections, so we instead seek to rely upon the database's own -capability to update or delete the rows automatically using foreign key ON -DELETE rules, instructing the unit of work to forego actually needing to load -these rows in order to handle them. The unit of work can be instructed to work -in this manner by configuring :paramref:`_orm.relationship.passive_deletes` on -the :func:`_orm.relationship` construct; the foreign key constraints in use -must also be correctly configured. - -For further detail on a complete "passive delete" configuration, see the -section :ref:`passive_deletes`. - - - diff --git a/doc/build/orm/loading_columns.rst b/doc/build/orm/loading_columns.rst deleted file mode 100644 index c1fad1710d3..00000000000 --- a/doc/build/orm/loading_columns.rst +++ /dev/null @@ -1,4 +0,0 @@ -:orphan: - -This document has moved to :doc:`queryguide/columns` - diff --git a/doc/build/orm/mapped_attributes.rst b/doc/build/orm/mapped_attributes.rst index d0610f4e0fa..6f23f6f5710 100644 --- a/doc/build/orm/mapped_attributes.rst +++ b/doc/build/orm/mapped_attributes.rst @@ -252,7 +252,7 @@ attribute, a SQL function is rendered which produces the same effect: (0, 12, 'address') {stop} -Read more about Hybrids at :ref:`hybrids_toplevel`. +Read more about Hybrids at ref_hybrids_toplevel. .. _synonyms: diff --git a/doc/build/orm/mapped_sql_expr.rst b/doc/build/orm/mapped_sql_expr.rst deleted file mode 100644 index 357949c8fee..00000000000 --- a/doc/build/orm/mapped_sql_expr.rst +++ /dev/null @@ -1,347 +0,0 @@ -.. currentmodule:: sqlalchemy.orm - -.. _mapper_sql_expressions: - -SQL Expressions as Mapped Attributes -==================================== - -Attributes on a mapped class can be linked to SQL expressions, which can -be used in queries. - -Using a Hybrid --------------- - -The easiest and most flexible way to link relatively simple SQL expressions to a class is to use a so-called -"hybrid attribute", -described in the section :ref:`hybrids_toplevel`. The hybrid provides -for an expression that works at both the Python level as well as at the -SQL expression level. For example, below we map a class ``User``, -containing attributes ``firstname`` and ``lastname``, and include a hybrid that -will provide for us the ``fullname``, which is the string concatenation of the two:: - - from sqlalchemy.ext.hybrid import hybrid_property - - - class User(Base): - __tablename__ = "user" - id = mapped_column(Integer, primary_key=True) - firstname = mapped_column(String(50)) - lastname = mapped_column(String(50)) - - @hybrid_property - def fullname(self): - return self.firstname + " " + self.lastname - -Above, the ``fullname`` attribute is interpreted at both the instance and -class level, so that it is available from an instance:: - - some_user = session.scalars(select(User).limit(1)).first() - print(some_user.fullname) - -as well as usable within queries:: - - some_user = session.scalars( - select(User).where(User.fullname == "John Smith").limit(1) - ).first() - -The string concatenation example is a simple one, where the Python expression -can be dual purposed at the instance and class level. Often, the SQL expression -must be distinguished from the Python expression, which can be achieved using -:meth:`.hybrid_property.expression`. Below we illustrate the case where a conditional -needs to be present inside the hybrid, using the ``if`` statement in Python and the -:func:`_expression.case` construct for SQL expressions:: - - from sqlalchemy.ext.hybrid import hybrid_property - from sqlalchemy.sql import case - - - class User(Base): - __tablename__ = "user" - id = mapped_column(Integer, primary_key=True) - firstname = mapped_column(String(50)) - lastname = mapped_column(String(50)) - - @hybrid_property - def fullname(self): - if self.firstname is not None: - return self.firstname + " " + self.lastname - else: - return self.lastname - - @fullname.expression - def fullname(cls): - return case( - (cls.firstname != None, cls.firstname + " " + cls.lastname), - else_=cls.lastname, - ) - -.. _mapper_column_property_sql_expressions: - -Using column_property ---------------------- - -The :func:`_orm.column_property` function can be used to map a SQL -expression in a manner similar to a regularly mapped :class:`_schema.Column`. -With this technique, the attribute is loaded -along with all other column-mapped attributes at load time. This is in some -cases an advantage over the usage of hybrids, as the value can be loaded -up front at the same time as the parent row of the object, particularly if -the expression is one which links to other tables (typically as a correlated -subquery) to access data that wouldn't normally be -available on an already loaded object. - -Disadvantages to using :func:`_orm.column_property` for SQL expressions include that -the expression must be compatible with the SELECT statement emitted for the class -as a whole, and there are also some configurational quirks which can occur -when using :func:`_orm.column_property` from declarative mixins. - -Our "fullname" example can be expressed using :func:`_orm.column_property` as -follows:: - - from sqlalchemy.orm import column_property - - - class User(Base): - __tablename__ = "user" - id = mapped_column(Integer, primary_key=True) - firstname = mapped_column(String(50)) - lastname = mapped_column(String(50)) - fullname = column_property(firstname + " " + lastname) - -Correlated subqueries may be used as well. Below we use the -:func:`_expression.select` construct to create a :class:`_sql.ScalarSelect`, -representing a column-oriented SELECT statement, that links together the count -of ``Address`` objects available for a particular ``User``:: - - from sqlalchemy.orm import column_property - from sqlalchemy import select, func - from sqlalchemy import Column, Integer, String, ForeignKey - - from sqlalchemy.orm import DeclarativeBase - - - class Base(DeclarativeBase): - pass - - - class Address(Base): - __tablename__ = "address" - id = mapped_column(Integer, primary_key=True) - user_id = mapped_column(Integer, ForeignKey("user.id")) - - - class User(Base): - __tablename__ = "user" - id = mapped_column(Integer, primary_key=True) - address_count = column_property( - select(func.count(Address.id)) - .where(Address.user_id == id) - .correlate_except(Address) - .scalar_subquery() - ) - -In the above example, we define a :func:`_expression.ScalarSelect` construct like the following:: - - stmt = ( - select(func.count(Address.id)) - .where(Address.user_id == id) - .correlate_except(Address) - .scalar_subquery() - ) - -Above, we first use :func:`_sql.select` to create a :class:`_sql.Select` -construct, which we then convert into a :term:`scalar subquery` using the -:meth:`_sql.Select.scalar_subquery` method, indicating our intent to use this -:class:`_sql.Select` statement in a column expression context. - -Within the :class:`_sql.Select` itself, we select the count of ``Address.id`` rows -where the ``Address.user_id`` column is equated to ``id``, which in the context -of the ``User`` class is the :class:`_schema.Column` named ``id`` (note that ``id`` is -also the name of a Python built in function, which is not what we want to use -here - if we were outside of the ``User`` class definition, we'd use ``User.id``). - -The :meth:`_sql.Select.correlate_except` method indicates that each element in the -FROM clause of this :func:`_expression.select` may be omitted from the FROM list (that is, correlated -to the enclosing SELECT statement against ``User``) except for the one corresponding -to ``Address``. This isn't strictly necessary, but prevents ``Address`` from -being inadvertently omitted from the FROM list in the case of a long string -of joins between ``User`` and ``Address`` tables where SELECT statements against -``Address`` are nested. - -For a :func:`.column_property` that refers to columns linked from a -many-to-many relationship, use :func:`.and_` to join the fields of the -association table to both tables in a relationship:: - - from sqlalchemy import and_ - - - class Author(Base): - # ... - - book_count = column_property( - select(func.count(books.c.id)) - .where( - and_( - book_authors.c.author_id == authors.c.id, - book_authors.c.book_id == books.c.id, - ) - ) - .scalar_subquery() - ) - -Adding column_property() to an existing Declarative mapped class -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -If import issues prevent the :func:`.column_property` from being defined -inline with the class, it can be assigned to the class after both -are configured. When using mappings that make use of a Declarative -base class (i.e. produced by the :class:`_orm.DeclarativeBase` superclass -or legacy functions such as :func:`_orm.declarative_base`), -this attribute assignment has the effect of calling :meth:`_orm.Mapper.add_property` -to add an additional property after the fact:: - - # only works if a declarative base class is in use - User.address_count = column_property( - select(func.count(Address.id)).where(Address.user_id == User.id).scalar_subquery() - ) - -When using mapping styles that don't use Declarative base classes -such as the :meth:`_orm.registry.mapped` decorator, the :meth:`_orm.Mapper.add_property` -method may be invoked explicitly on the underlying :class:`_orm.Mapper` object, -which can be obtained using :func:`_sa.inspect`:: - - from sqlalchemy.orm import registry - - reg = registry() - - - @reg.mapped - class User: - __tablename__ = "user" - - # ... additional mapping directives - - - # later ... - - # works for any kind of mapping - from sqlalchemy import inspect - - inspect(User).add_property( - column_property( - select(func.count(Address.id)) - .where(Address.user_id == User.id) - .scalar_subquery() - ) - ) - -.. seealso:: - - :ref:`orm_declarative_table_adding_columns` - - -.. _mapper_column_property_sql_expressions_composed: - -Composing from Column Properties at Mapping Time -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -It is possible to create mappings that combine multiple -:class:`.ColumnProperty` objects together. The :class:`.ColumnProperty` will -be interpreted as a SQL expression when used in a Core expression context, -provided that it is targeted by an existing expression object; this works by -the Core detecting that the object has a ``__clause_element__()`` method which -returns a SQL expression. However, if the :class:`.ColumnProperty` is used as -a lead object in an expression where there is no other Core SQL expression -object to target it, the :attr:`.ColumnProperty.expression` attribute will -return the underlying SQL expression so that it can be used to build SQL -expressions consistently. Below, the ``File`` class contains an attribute -``File.path`` that concatenates a string token to the ``File.filename`` -attribute, which is itself a :class:`.ColumnProperty`:: - - - class File(Base): - __tablename__ = "file" - - id = mapped_column(Integer, primary_key=True) - name = mapped_column(String(64)) - extension = mapped_column(String(8)) - filename = column_property(name + "." + extension) - path = column_property("C:/" + filename.expression) - -When the ``File`` class is used in expressions normally, the attributes -assigned to ``filename`` and ``path`` are usable directly. The use of the -:attr:`.ColumnProperty.expression` attribute is only necessary when using -the :class:`.ColumnProperty` directly within the mapping definition:: - - stmt = select(File.path).where(File.filename == "foo.txt") - -Using Column Deferral with ``column_property()`` -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The column deferral feature introduced in the :ref:`queryguide_toplevel` -at :ref:`orm_queryguide_column_deferral` may be applied at mapping time -to a SQL expression mapped by :func:`_orm.column_property` by using the -:func:`_orm.deferred` function in place of :func:`_orm.column_property`:: - - from sqlalchemy.orm import deferred - - - class User(Base): - __tablename__ = "user" - - id: Mapped[int] = mapped_column(primary_key=True) - firstname: Mapped[str] = mapped_column() - lastname: Mapped[str] = mapped_column() - fullname: Mapped[str] = deferred(firstname + " " + lastname) - -.. seealso:: - - :ref:`orm_queryguide_deferred_imperative` - - - -Using a plain descriptor ------------------------- - -In cases where a SQL query more elaborate than what :func:`_orm.column_property` -or :class:`.hybrid_property` can provide must be emitted, a regular Python -function accessed as an attribute can be used, assuming the expression -only needs to be available on an already-loaded instance. The function -is decorated with Python's own ``@property`` decorator to mark it as a read-only -attribute. Within the function, :func:`.object_session` -is used to locate the :class:`.Session` corresponding to the current object, -which is then used to emit a query:: - - from sqlalchemy.orm import object_session - from sqlalchemy import select, func - - - class User(Base): - __tablename__ = "user" - id = mapped_column(Integer, primary_key=True) - firstname = mapped_column(String(50)) - lastname = mapped_column(String(50)) - - @property - def address_count(self): - return object_session(self).scalar( - select(func.count(Address.id)).where(Address.user_id == self.id) - ) - -The plain descriptor approach is useful as a last resort, but is less performant -in the usual case than both the hybrid and column property approaches, in that -it needs to emit a SQL query upon each access. - -.. _mapper_querytime_expression: - -Query-time SQL expressions as mapped attributes ------------------------------------------------ - -In addition to being able to configure fixed SQL expressions on mapped classes, -the SQLAlchemy ORM also includes a feature wherein objects may be loaded -with the results of arbitrary SQL expressions which are set up at query time as part -of their state. This behavior is available by configuring an ORM mapped -attribute using :func:`_orm.query_expression` and then using the -:func:`_orm.with_expression` loader option at query time. See the section -:ref:`orm_queryguide_with_expression` for an example mapping and usage. - diff --git a/doc/build/orm/mapper_config.rst b/doc/build/orm/mapper_config.rst index 68218491d53..b9ae434b47c 100644 --- a/doc/build/orm/mapper_config.rst +++ b/doc/build/orm/mapper_config.rst @@ -22,16 +22,8 @@ in SQLAlchemy, it's first introduced in the :ref:`unified_tutorial` at mapping_styles declarative_mapping - dataclasses - mapped_sql_expr mapped_attributes - composites inheritance nonstandard_mappings - versioning mapping_api -.. toctree:: - :hidden: - - scalar_mapping diff --git a/doc/build/orm/mapping_api.rst b/doc/build/orm/mapping_api.rst index 57ef5e00e0f..578c77948b0 100644 --- a/doc/build/orm/mapping_api.rst +++ b/doc/build/orm/mapping_api.rst @@ -76,7 +76,7 @@ Class Mapping API .. seealso:: - :ref:`declarative_inheritance` + ref_declarative_inheritance :ref:`mixin_inheritance_columns` diff --git a/doc/build/orm/mapping_columns.rst b/doc/build/orm/mapping_columns.rst deleted file mode 100644 index 25c6604fafa..00000000000 --- a/doc/build/orm/mapping_columns.rst +++ /dev/null @@ -1,9 +0,0 @@ -.. _mapping_columns_toplevel: - -Mapping Table Columns -===================== - -This section has been integrated into the -:ref:`orm_declarative_table_config_toplevel` Declarative section. - - diff --git a/doc/build/orm/mapping_styles.rst b/doc/build/orm/mapping_styles.rst index b4c21a353de..4f650c593f2 100644 --- a/doc/build/orm/mapping_styles.rst +++ b/doc/build/orm/mapping_styles.rst @@ -364,7 +364,7 @@ An object of type ``User`` above will have a constructor which allows .. tip:: - The :ref:`orm_declarative_native_dataclasses` feature provides an alternate + The ref_orm_declarative_native_dataclasses feature provides an alternate means of generating a default ``__init__()`` method by using Python dataclasses, and allows for a highly configurable constructor form. @@ -453,7 +453,7 @@ Inspection of Mapper objects As illustrated in the previous section, the :class:`_orm.Mapper` object is available from any mapped class, regardless of method, using the -:ref:`core_inspection_toplevel` system. Using the +ref_core_inspection_toplevel system. Using the :func:`_sa.inspect` function, one can acquire the :class:`_orm.Mapper` from a mapped class:: @@ -528,7 +528,7 @@ The :class:`_orm.Session` to which the object is :term:`attached`, if any:: >>> insp.session -Information about the current :ref:`persistence state ` +Information about the current ref_session_object_states for the object:: >>> insp.persistent diff --git a/doc/build/orm/persistence_techniques.rst b/doc/build/orm/persistence_techniques.rst index dca6cefffd8..61bdbcbabb2 100644 --- a/doc/build/orm/persistence_techniques.rst +++ b/doc/build/orm/persistence_techniques.rst @@ -248,7 +248,7 @@ value and pass it through, rather than omitting it as a "missing" value:: Fetching Server-Generated Defaults =================================== -As introduced in the sections :ref:`server_defaults` and :ref:`triggered_columns`, +As introduced in the sections ref_server_defaults and ref_triggered_columns, the Core supports the notion of database columns for which the database itself generates a value upon INSERT and in less common cases upon UPDATE statements. The ORM features support for such columns regarding being @@ -375,7 +375,7 @@ to retreive the newly generated primary key value: .. seealso:: - :ref:`mssql_insert_behavior` - background on the SQL Server dialect's + ref_mssql_insert_behavior - background on the SQL Server dialect's methods of fetching newly generated primary key values Case 3: non primary key, RETURNING or equivalent is not supported or not needed @@ -574,7 +574,7 @@ INSERT looks like: .. seealso:: - :ref:`metadata_defaults_toplevel` + ref_metadata_defaults_toplevel Notes on eagerly fetching client invoked SQL expressions used for INSERT or UPDATE ----------------------------------------------------------------------------------- @@ -584,7 +584,7 @@ to create tables that include default-generation functions within their DDL. SQLAlchemy also supports non-DDL server side defaults, as documented at -:ref:`defaults_client_invoked_sql`; these "client invoked SQL expressions" +ref_defaults_client_invoked_sql; these "client invoked SQL expressions" are set up using the :paramref:`_schema.Column.default` and :paramref:`_schema.Column.onupdate` parameters. @@ -826,19 +826,8 @@ Horizontal partitioning partitions the rows of a single table (or a set of tables) across multiple databases. The SQLAlchemy :class:`.Session` contains support for this concept, however to use it fully requires that :class:`.Session` and :class:`_query.Query` subclasses are used. A basic version -of these subclasses are available in the :ref:`horizontal_sharding_toplevel` -ORM extension. An example of use is at: :ref:`examples_sharding`. +of these subclasses are available in the ref_horizontal_sharding_toplevel +ORM extension. An example of use is at: ref_examples_sharding. .. _bulk_operations: -Bulk Operations -=============== - -.. legacy:: - - SQLAlchemy 2.0 has integrated the :class:`_orm.Session` "bulk insert" and - "bulk update" capabilities into 2.0 style :meth:`_orm.Session.execute` - method, making direct use of :class:`_dml.Insert` and :class:`_dml.Update` - constructs. See the document at :doc:`queryguide/dml` for documentation, - including :ref:`orm_queryguide_legacy_bulk_insert` which illustrates migration - from the older methods to the new methods. diff --git a/doc/build/orm/queryguide/api.rst b/doc/build/orm/queryguide/api.rst index 15301cbd003..38e5a3a799d 100644 --- a/doc/build/orm/queryguide/api.rst +++ b/doc/build/orm/queryguide/api.rst @@ -118,7 +118,7 @@ The ``populate_existing`` execution option is equvialent to the :ref:`faq_session_identity` - in :doc:`/faq/index` - :ref:`session_expire` - in the ORM :class:`_orm.Session` + ref_session_expire - in the ORM :class:`_orm.Session` documentation .. _orm_queryguide_autoflush: @@ -287,12 +287,6 @@ Identity Token .. doctest-disable: -.. deepalchemy:: This option is an advanced-use feature mostly intended - to be used with the :ref:`horizontal_sharding_toplevel` extension. For - typical cases of loading objects with identical primary keys from different - "shards" or partitions, consider using individual :class:`_orm.Session` - objects per shard first. - The "identity token" is an arbitrary value that can be associated within the :term:`identity key` of newly loaded objects. This element exists @@ -300,7 +294,7 @@ first and foremost to support extensions which perform per-row "sharding", where objects may be loaded from any number of replicas of a particular database table that nonetheless have overlapping primary key values. The primary consumer of "identity token" is the -:ref:`horizontal_sharding_toplevel` extension, which supplies a general +ref_horizontal_sharding_toplevel extension, which supplies a general framework for persisting objects among multiple "shards" of a particular database table. @@ -361,7 +355,7 @@ into both ``test_schema.my_table`` as well as ``test_schema_2.my_table``. The :class:`_orm.Session` objects above are independent. If we wanted to persist both objects in one transaction, we would need to use the -:ref:`horizontal_sharding_toplevel` extension to do this. +ref_horizontal_sharding_toplevel extension to do this. However, we can illustrate querying for these objects in one session as follows: @@ -399,14 +393,14 @@ to view the two distinct identity tokens:: The above logic takes place automatically when using the -:ref:`horizontal_sharding_toplevel` extension. +ref_horizontal_sharding_toplevel extension. .. versionadded:: 2.0.0rc1 - added the ``identity_token`` ORM level execution option. .. seealso:: - :ref:`examples_sharding` - in the :ref:`examples_toplevel` section. + ref_examples_sharding - in the ref_examples_toplevel section. See the script ``separate_schema_translates.py`` for a demonstration of the above use case using the full sharding API. diff --git a/doc/build/orm/queryguide/columns.rst b/doc/build/orm/queryguide/columns.rst index 255c1f9028b..56dbec238dc 100644 --- a/doc/build/orm/queryguide/columns.rst +++ b/doc/build/orm/queryguide/columns.rst @@ -104,7 +104,7 @@ See the section :ref:`orm_queryguide_deferred_raiseload` for background and examples. .. tip:: as noted elsewhere, lazy loading is not available when using - :ref:`asyncio_toplevel`. + ref_asyncio_toplevel. Using ``load_only()`` with multiple entities ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -437,8 +437,8 @@ when mapped SQL expressions should be loaded on a deferred basis: .. seealso:: - :ref:`mapper_column_property_sql_expressions` - in the section - :ref:`mapper_sql_expressions` + ref_mapper_column_property_sql_expressions - in the section + ref_mapper_sql_expressions :ref:`orm_imperative_table_column_options` - in the section :ref:`orm_declarative_table_config_toplevel` @@ -823,7 +823,7 @@ The :func:`.query_expression` mapping has these caveats: The :func:`_orm.with_expression` option is a special option used to apply SQL expressions to mapped classes dynamically at query time. For ordinary fixed SQL expressions configured on mappers, - see the section :ref:`mapper_sql_expressions`. + see the section ref_mapper_sql_expressions. .. _orm_queryguide_with_expression_unions: diff --git a/doc/build/orm/queryguide/dml.rst b/doc/build/orm/queryguide/dml.rst index 04a4fb2bb05..ce3687d0b2c 100644 --- a/doc/build/orm/queryguide/dml.rst +++ b/doc/build/orm/queryguide/dml.rst @@ -75,7 +75,7 @@ as much as possible for many rows:: The parameter dictionaries contain key/value pairs which may correspond to ORM mapped attributes that line up with mapped :class:`._schema.Column` or :func:`_orm.mapped_column` declarations, as well as with -:ref:`composite ` declarations. The keys should match +ref_mapper_composite declarations. The keys should match the **ORM mapped attribute name** and **not** the actual database column name, if these two names happen to be different. @@ -510,9 +510,9 @@ on the capabilities of the backend in use. The dialects included with SQLAlchemy that include dialect-specific "upsert" API features are: -* SQLite - using :class:`_sqlite.Insert` documented at :ref:`sqlite_on_conflict_insert` -* PostgreSQL - using :class:`_postgresql.Insert` documented at :ref:`postgresql_insert_on_conflict` -* MySQL/MariaDB - using :class:`_mysql.Insert` documented at :ref:`mysql_insert_on_duplicate_key_update` +* SQLite - using :class:`_sqlite.Insert` documented at ref_sqlite_on_conflict_insert +* PostgreSQL - using :class:`_postgresql.Insert` documented at ref_postgresql_insert_on_conflict +* MySQL/MariaDB - using :class:`_mysql.Insert` documented at ref_mysql_insert_on_duplicate_key_update Users should review the above sections for background on proper construction of these objects; in particular, the "upsert" method typically needs to diff --git a/doc/build/orm/queryguide/inheritance.rst b/doc/build/orm/queryguide/inheritance.rst index 7040128f405..00612dea59b 100644 --- a/doc/build/orm/queryguide/inheritance.rst +++ b/doc/build/orm/queryguide/inheritance.rst @@ -100,7 +100,7 @@ loaded, in the case that the consuming application will need to be accessing subclass-specific attributes, as this would be an example of the :term:`N plus one` problem that emits additional SQL per row. This additional SQL can impact performance and also be incompatible with approaches such as -using :ref:`asyncio `. Additionally, in our query for +using ref_asyncio_toplevel. Additionally, in our query for ``Employee`` objects, since the query is against the base table only, we did not have a way to add SQL criteria involving subclass-specific attributes in terms of ``Manager`` or ``Engineer``. The next two sections detail two diff --git a/doc/build/orm/queryguide/queryguide_nav_include.rst b/doc/build/orm/queryguide/queryguide_nav_include.rst index a860021648d..0534c3e7e12 100644 --- a/doc/build/orm/queryguide/queryguide_nav_include.rst +++ b/doc/build/orm/queryguide/queryguide_nav_include.rst @@ -8,7 +8,4 @@ Previous: |prev| | Next: |next| -.. footer_topic:: |tutorial_title| - - Next Query Guide Section: |next| diff --git a/doc/build/orm/queryguide/relationships.rst b/doc/build/orm/queryguide/relationships.rst index 9c621ea9ac1..5665264a1b4 100644 --- a/doc/build/orm/queryguide/relationships.rst +++ b/doc/build/orm/queryguide/relationships.rst @@ -92,7 +92,7 @@ The primary forms of relationship loading are: :meth:`.WriteOnlyCollection.add_all` and :meth:`.WriteOnlyCollection.remove` methods. Querying the collection is performed by invoking a SELECT statement which is constructed using the :meth:`.WriteOnlyCollection.select` - method. Write only loading is discussed at :ref:`write_only_relationship`. + method. Write only loading is discussed at ref_write_only_relationship. * **dynamic loading** - available via ``lazy='dynamic'``, or by annotating the left side of the :class:`_orm.Relationship` object using the @@ -102,9 +102,9 @@ The primary forms of relationship loading are: contents. However, dynamic loaders will implicitly iterate the underlying collection in various circumstances which makes them less useful for managing truly large collections. Dynamic loaders are superseded by - :ref:`"write only" ` collections, which will prevent + ref_write_only_relationship collections, which will prevent the underlying collection from being implicitly loaded under any - circumstances. Dynamic loaders are discussed at :ref:`dynamic_relationship`. + circumstances. Dynamic loaders are discussed at ref_dynamic_relationship. .. _relationship_lazy_option: @@ -850,18 +850,6 @@ Things to know about this kind of loading include: Subquery Eager Loading ---------------------- -.. legacy:: The :func:`_orm.subqueryload` eager loader is mostly legacy - at this point, superseded by the :func:`_orm.selectinload` strategy - which is of much simpler design, more flexible with features such as - :ref:`Yield Per `, and emits more efficient SQL - statements in most cases. As :func:`_orm.subqueryload` relies upon - re-interpreting the original SELECT statement, it may fail to work - efficiently when given very complex source queries. - - :func:`_orm.subqueryload` may continue to be useful for the specific - case of an eager loaded collection for objects that use composite primary - keys, on the Microsoft SQL Server backend that continues to not have - support for the "tuple IN" syntax. Subquery loading is similar in operation to selectin eager loading, however the SELECT statement which is emitted is derived from the original statement, diff --git a/doc/build/orm/quickstart.rst b/doc/build/orm/quickstart.rst index 652ed235a57..73bb308153c 100644 --- a/doc/build/orm/quickstart.rst +++ b/doc/build/orm/quickstart.rst @@ -121,7 +121,7 @@ The :func:`_orm.relationship` construct is introduced in the Finally, the above example classes include a ``__repr__()`` method, which is not required but is useful for debugging. Mapped classes can be created with methods such as ``__repr__()`` generated automatically, using dataclasses. More -on dataclass mapping at :ref:`orm_declarative_native_dataclasses`. +on dataclass mapping at ref_orm_declarative_native_dataclasses. Create an Engine @@ -131,7 +131,7 @@ Create an Engine The :class:`_engine.Engine` is a **factory** that can create new database connections for us, which also holds onto connections inside of a :ref:`Connection Pool ` for fast reuse. For learning -purposes, we normally use a :ref:`SQLite ` memory-only database +purposes, we normally use a ref_sqlite_toplevel memory-only database for convenience:: >>> from sqlalchemy import create_engine diff --git a/doc/build/orm/relationship_persistence.rst b/doc/build/orm/relationship_persistence.rst index 9a5a036c695..f56773caa83 100644 --- a/doc/build/orm/relationship_persistence.rst +++ b/doc/build/orm/relationship_persistence.rst @@ -223,7 +223,7 @@ object, and we also illustrate the ``mysql_engine='InnoDB'`` setting which, on a MySQL backend, ensures that the ``InnoDB`` engine supporting referential integrity is used. When using SQLite, referential integrity should be enabled, using the configuration described at -:ref:`sqlite_foreign_keys`. +ref_sqlite_foreign_keys. .. seealso:: diff --git a/doc/build/orm/relationships.rst b/doc/build/orm/relationships.rst index ab0402721de..bd75123a909 100644 --- a/doc/build/orm/relationships.rst +++ b/doc/build/orm/relationships.rst @@ -15,9 +15,7 @@ of its usage. For an introduction to relationships, start with basic_relationships self_referential join_conditions - large_collections collection_api relationship_persistence - backref relationship_api diff --git a/doc/build/orm/scalar_mapping.rst b/doc/build/orm/scalar_mapping.rst deleted file mode 100644 index f6863edadab..00000000000 --- a/doc/build/orm/scalar_mapping.rst +++ /dev/null @@ -1,13 +0,0 @@ -=============================== -Mapping SQL Expressions -=============================== - -This page has been merged into the -:ref:`mapper_config_toplevel` index. - - -.. toctree:: - :hidden: - - mapping_columns - diff --git a/doc/build/orm/self_referential.rst b/doc/build/orm/self_referential.rst index 70dfb4be934..9f13468e205 100644 --- a/doc/build/orm/self_referential.rst +++ b/doc/build/orm/self_referential.rst @@ -87,7 +87,7 @@ relationship using the :func:`.backref` function:: .. seealso:: - :ref:`examples_adjacencylist` - working example + ref_examples_adjacencylist - working example Composite Adjacency Lists ~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/doc/build/orm/session.rst b/doc/build/orm/session.rst index 00d2e6f6a10..9f3fbd2062d 100644 --- a/doc/build/orm/session.rst +++ b/doc/build/orm/session.rst @@ -16,11 +16,9 @@ persistence operations is the :maxdepth: 3 session_basics - session_state_management cascades session_transaction persistence_techniques - contextual session_events session_api diff --git a/doc/build/orm/session_basics.rst b/doc/build/orm/session_basics.rst index 12413336a06..b3cdb829ea6 100644 --- a/doc/build/orm/session_basics.rst +++ b/doc/build/orm/session_basics.rst @@ -533,11 +533,11 @@ ways to refresh its contents with new data from the current transaction: .. Further discussion on the refresh / expire concept can be found at -:ref:`session_expire`. +ref_session_expire. .. seealso:: - :ref:`session_expire` + ref_session_expire :ref:`faq_session_identity` @@ -924,7 +924,7 @@ The :class:`.Session` is not designed to be a global object from which everyone consults as a "registry" of objects. That's more the job of a **second level cache**. SQLAlchemy provides a pattern for implementing second level caching using `dogpile.cache `_, -via the :ref:`examples_caching` example. +via the ref_examples_caching example. How can I get the :class:`~sqlalchemy.orm.session.Session` for a certain object? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -934,7 +934,7 @@ available on :class:`~sqlalchemy.orm.session.Session`:: session = Session.object_session(someobject) -The newer :ref:`core_inspection_toplevel` system can also be used:: +The newer ref_core_inspection_toplevel system can also be used:: from sqlalchemy import inspect @@ -952,7 +952,7 @@ time. The :class:`.Session` should be used in such a way that one instance exists for a single series of operations within a single transaction. One expedient way to get this effect is by associating -a :class:`.Session` with the current thread (see :ref:`unitofwork_contextual` +a :class:`.Session` with the current thread (see ref_unitofwork_contextual for background). Another is to use a pattern where the :class:`.Session` is passed between functions and is otherwise not shared with other threads. diff --git a/doc/build/orm/session_events.rst b/doc/build/orm/session_events.rst index f61421ea920..d98be69f5a4 100644 --- a/doc/build/orm/session_events.rst +++ b/doc/build/orm/session_events.rst @@ -108,7 +108,7 @@ apply to subsequent relationship loads, which includes lazy loads, selectinloads, etc. For a series of classes that all feature some common column structure, -if the classes are composed using a :ref:`declarative mixin `, +if the classes are composed using a ref_declarative_mixins, the mixin class itself may be used in conjunction with the :func:`_orm.with_loader_criteria` option by making use of a Python lambda. The Python lambda will be invoked at query compilation time against the specific entities which match the criteria. @@ -160,7 +160,7 @@ to intercept all objects that extend from ``HasTimestamp`` and filter their .. seealso:: - :ref:`examples_session_orm_events` - includes working examples of the + ref_examples_session_orm_events - includes working examples of the above :func:`_orm.with_loader_criteria` recipes. .. _do_orm_execute_re_executing: @@ -168,13 +168,6 @@ to intercept all objects that extend from ``HasTimestamp`` and filter their Re-Executing Statements ^^^^^^^^^^^^^^^^^^^^^^^ -.. deepalchemy:: the statement re-execution feature involves a slightly - intricate recursive sequence, and is intended to solve the fairly hard - problem of being able to re-route the execution of a SQL statement into - various non-SQL contexts. The twin examples of "dogpile caching" and - "horizontal sharding", linked below, should be used as a guide for when this - rather advanced feature is appropriate to be used. - The :class:`_orm.ORMExecuteState` is capable of controlling the execution of the given statement; this includes the ability to either not invoke the statement at all, allowing a pre-constructed result set retrieved from a cache to @@ -244,7 +237,7 @@ result from the "frozen" result, the :func:`_orm.loading.merge_frozen_result` function is used to merge the "frozen" data from the result object into the current session. -The above example is implemented as a complete example in :ref:`examples_caching`. +The above example is implemented as a complete example in ref_examples_caching. The :meth:`_orm.ORMExecuteState.invoke_statement` method may also be called multiple times, passing along different information to the @@ -253,14 +246,14 @@ that the :class:`_orm.Session` will make use of different :class:`_engine.Engine` objects each time. This will return a different :class:`_engine.Result` object each time; these results can be merged together using the :meth:`_engine.Result.merge` method. This is the technique employed -by the :ref:`horizontal_sharding_toplevel` extension; see the source code to +by the ref_horizontal_sharding_toplevel extension; see the source code to familiarize. .. seealso:: - :ref:`examples_caching` + ref_examples_caching - :ref:`examples_sharding` + ref_examples_sharding @@ -296,8 +289,8 @@ scanning the collections :attr:`.Session.new`, :attr:`.Session.dirty` and where something will be happening. For illustrations of :meth:`.SessionEvents.before_flush`, see -examples such as :ref:`examples_versioned_history` and -:ref:`examples_versioned_rows`. +examples such as ref_examples_versioned_history and +ref_examples_versioned_rows. ``after_flush()`` ^^^^^^^^^^^^^^^^^ @@ -408,7 +401,7 @@ Object Lifecycle Events ----------------------- Another use case for events is to track the lifecycle of objects. This -refers to the states first introduced at :ref:`session_object_states`. +refers to the states first introduced at ref_session_object_states. All the states above can be tracked fully with events. Each event represents a distinct state transition, meaning, the starting state @@ -684,7 +677,7 @@ however, it is often much more convenient to use a "validator" hook, which uses these hooks behind the scenes; see :ref:`simple_validators` for background on this. The attribute events are also behind the mechanics of backreferences. An example illustrating use of attribute events -is in :ref:`examples_instrumentation`. +is in ref_examples_instrumentation. diff --git a/doc/build/orm/session_state_management.rst b/doc/build/orm/session_state_management.rst deleted file mode 100644 index 3538bdc2242..00000000000 --- a/doc/build/orm/session_state_management.rst +++ /dev/null @@ -1,657 +0,0 @@ -State Management -================ - -.. _session_object_states: - -Quickie Intro to Object States ------------------------------- - -It's helpful to know the states which an instance can have within a session: - -* **Transient** - an instance that's not in a session, and is not saved to the - database; i.e. it has no database identity. The only relationship such an - object has to the ORM is that its class has a :class:`_orm.Mapper` associated - with it. - -* **Pending** - when you :meth:`~.Session.add` a transient - instance, it becomes pending. It still wasn't actually flushed to the - database yet, but it will be when the next flush occurs. - -* **Persistent** - An instance which is present in the session and has a record - in the database. You get persistent instances by either flushing so that the - pending instances become persistent, or by querying the database for - existing instances (or moving persistent instances from other sessions into - your local session). - -* **Deleted** - An instance which has been deleted within a flush, but - the transaction has not yet completed. Objects in this state are essentially - in the opposite of "pending" state; when the session's transaction is committed, - the object will move to the detached state. Alternatively, when - the session's transaction is rolled back, a deleted object moves - *back* to the persistent state. - -* **Detached** - an instance which corresponds, or previously corresponded, - to a record in the database, but is not currently in any session. - The detached object will contain a database identity marker, however - because it is not associated with a session, it is unknown whether or not - this database identity actually exists in a target database. Detached - objects are safe to use normally, except that they have no ability to - load unloaded attributes or attributes that were previously marked - as "expired". - -For a deeper dive into all possible state transitions, see the -section :ref:`session_lifecycle_events` which describes each transition -as well as how to programmatically track each one. - -Getting the Current State of an Object -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The actual state of any mapped object can be viewed at any time using -the :func:`_sa.inspect` function on a mapped instance; this function will -return the corresponding :class:`.InstanceState` object which manages the -internal ORM state for the object. :class:`.InstanceState` provides, among -other accessors, boolean attributes indicating the persistence state -of the object, including: - -* :attr:`.InstanceState.transient` -* :attr:`.InstanceState.pending` -* :attr:`.InstanceState.persistent` -* :attr:`.InstanceState.deleted` -* :attr:`.InstanceState.detached` - -E.g.:: - - >>> from sqlalchemy import inspect - >>> insp = inspect(my_object) - >>> insp.persistent - True - -.. seealso:: - - :ref:`orm_mapper_inspection_instancestate` - further examples of - :class:`.InstanceState` - -.. _session_attributes: - -Session Attributes ------------------- - -The :class:`~sqlalchemy.orm.session.Session` itself acts somewhat like a -set-like collection. All items present may be accessed using the iterator -interface:: - - for obj in session: - print(obj) - -And presence may be tested for using regular "contains" semantics:: - - if obj in session: - print("Object is present") - -The session is also keeping track of all newly created (i.e. pending) objects, -all objects which have had changes since they were last loaded or saved (i.e. -"dirty"), and everything that's been marked as deleted:: - - # pending objects recently added to the Session - session.new - - # persistent objects which currently have changes detected - # (this collection is now created on the fly each time the property is called) - session.dirty - - # persistent objects that have been marked as deleted via session.delete(obj) - session.deleted - - # dictionary of all persistent objects, keyed on their - # identity key - session.identity_map - -(Documentation: :attr:`.Session.new`, :attr:`.Session.dirty`, -:attr:`.Session.deleted`, :attr:`.Session.identity_map`). - - -.. _session_referencing_behavior: - -Session Referencing Behavior ----------------------------- - -Objects within the session are *weakly referenced*. This -means that when they are dereferenced in the outside application, they fall -out of scope from within the :class:`~sqlalchemy.orm.session.Session` as well -and are subject to garbage collection by the Python interpreter. The -exceptions to this include objects which are pending, objects which are marked -as deleted, or persistent objects which have pending changes on them. After a -full flush, these collections are all empty, and all objects are again weakly -referenced. - -To cause objects in the :class:`.Session` to remain strongly -referenced, usually a simple approach is all that's needed. Examples -of externally managed strong-referencing behavior include loading -objects into a local dictionary keyed to their primary key, or into -lists or sets for the span of time that they need to remain -referenced. These collections can be associated with a -:class:`.Session`, if desired, by placing them into the -:attr:`.Session.info` dictionary. - -An event based approach is also feasible. A simple recipe that provides -"strong referencing" behavior for all objects as they remain within -the :term:`persistent` state is as follows:: - - from sqlalchemy import event - - - def strong_reference_session(session): - @event.listens_for(session, "pending_to_persistent") - @event.listens_for(session, "deleted_to_persistent") - @event.listens_for(session, "detached_to_persistent") - @event.listens_for(session, "loaded_as_persistent") - def strong_ref_object(sess, instance): - if "refs" not in sess.info: - sess.info["refs"] = refs = set() - else: - refs = sess.info["refs"] - - refs.add(instance) - - @event.listens_for(session, "persistent_to_detached") - @event.listens_for(session, "persistent_to_deleted") - @event.listens_for(session, "persistent_to_transient") - def deref_object(sess, instance): - sess.info["refs"].discard(instance) - -Above, we intercept the :meth:`.SessionEvents.pending_to_persistent`, -:meth:`.SessionEvents.detached_to_persistent`, -:meth:`.SessionEvents.deleted_to_persistent` and -:meth:`.SessionEvents.loaded_as_persistent` event hooks in order to intercept -objects as they enter the :term:`persistent` transition, and the -:meth:`.SessionEvents.persistent_to_detached` and -:meth:`.SessionEvents.persistent_to_deleted` hooks to intercept -objects as they leave the persistent state. - -The above function may be called for any :class:`.Session` in order to -provide strong-referencing behavior on a per-:class:`.Session` basis:: - - from sqlalchemy.orm import Session - - my_session = Session() - strong_reference_session(my_session) - -It may also be called for any :class:`.sessionmaker`:: - - from sqlalchemy.orm import sessionmaker - - maker = sessionmaker() - strong_reference_session(maker) - -.. _unitofwork_merging: - -Merging -------- - -:meth:`~.Session.merge` transfers state from an -outside object into a new or already existing instance within a session. It -also reconciles the incoming data against the state of the -database, producing a history stream which will be applied towards the next -flush, or alternatively can be made to produce a simple "transfer" of -state without producing change history or accessing the database. Usage is as follows:: - - merged_object = session.merge(existing_object) - -When given an instance, it follows these steps: - -* It examines the primary key of the instance. If it's present, it attempts - to locate that instance in the local identity map. If the ``load=True`` - flag is left at its default, it also checks the database for this primary - key if not located locally. -* If the given instance has no primary key, or if no instance can be found - with the primary key given, a new instance is created. -* The state of the given instance is then copied onto the located/newly created - instance. For attribute values which are present on the source instance, the - value is transferred to the target instance. For attribute values that aren't - present on the source instance, the corresponding attribute on the target - instance is :term:`expired` from memory, which discards any locally - present value from the target instance for that attribute, but no - direct modification is made to the database-persisted value for that - attribute. - - If the ``load=True`` flag is left at its default, - this copy process emits events and will load the target object's - unloaded collections for each attribute present on the source object, - so that the incoming state can be reconciled against what's - present in the database. If ``load`` - is passed as ``False``, the incoming data is "stamped" directly without - producing any history. -* The operation is cascaded to related objects and collections, as - indicated by the ``merge`` cascade (see :ref:`unitofwork_cascades`). -* The new instance is returned. - -With :meth:`~.Session.merge`, the given "source" -instance is not modified nor is it associated with the target :class:`.Session`, -and remains available to be merged with any number of other :class:`.Session` -objects. :meth:`~.Session.merge` is useful for -taking the state of any kind of object structure without regard for its -origins or current session associations and copying its state into a -new session. Here's some examples: - -* An application which reads an object structure from a file and wishes to - save it to the database might parse the file, build up the - structure, and then use - :meth:`~.Session.merge` to save it - to the database, ensuring that the data within the file is - used to formulate the primary key of each element of the - structure. Later, when the file has changed, the same - process can be re-run, producing a slightly different - object structure, which can then be ``merged`` in again, - and the :class:`~sqlalchemy.orm.session.Session` will - automatically update the database to reflect those - changes, loading each object from the database by primary key and - then updating its state with the new state given. - -* An application is storing objects in an in-memory cache, shared by - many :class:`.Session` objects simultaneously. :meth:`~.Session.merge` - is used each time an object is retrieved from the cache to create - a local copy of it in each :class:`.Session` which requests it. - The cached object remains detached; only its state is moved into - copies of itself that are local to individual :class:`~.Session` - objects. - - In the caching use case, it's common to use the ``load=False`` - flag to remove the overhead of reconciling the object's state - with the database. There's also a "bulk" version of - :meth:`~.Session.merge` called :meth:`_query.Query.merge_result` - that was designed to work with cache-extended :class:`_query.Query` - objects - see the section :ref:`examples_caching`. - -* An application wants to transfer the state of a series of objects - into a :class:`.Session` maintained by a worker thread or other - concurrent system. :meth:`~.Session.merge` makes a copy of each object - to be placed into this new :class:`.Session`. At the end of the operation, - the parent thread/process maintains the objects it started with, - and the thread/worker can proceed with local copies of those objects. - - In the "transfer between threads/processes" use case, the application - may want to use the ``load=False`` flag as well to avoid overhead and - redundant SQL queries as the data is transferred. - -Merge Tips -~~~~~~~~~~ - -:meth:`~.Session.merge` is an extremely useful method for many purposes. However, -it deals with the intricate border between objects that are transient/detached and -those that are persistent, as well as the automated transference of state. -The wide variety of scenarios that can present themselves here often require a -more careful approach to the state of objects. Common problems with merge usually involve -some unexpected state regarding the object being passed to :meth:`~.Session.merge`. - -Lets use the canonical example of the User and Address objects:: - - class User(Base): - __tablename__ = "user" - - id = mapped_column(Integer, primary_key=True) - name = mapped_column(String(50), nullable=False) - addresses = relationship("Address", backref="user") - - - class Address(Base): - __tablename__ = "address" - - id = mapped_column(Integer, primary_key=True) - email_address = mapped_column(String(50), nullable=False) - user_id = mapped_column(Integer, ForeignKey("user.id"), nullable=False) - -Assume a ``User`` object with one ``Address``, already persistent:: - - >>> u1 = User(name="ed", addresses=[Address(email_address="ed@ed.com")]) - >>> session.add(u1) - >>> session.commit() - -We now create ``a1``, an object outside the session, which we'd like -to merge on top of the existing ``Address``:: - - >>> existing_a1 = u1.addresses[0] - >>> a1 = Address(id=existing_a1.id) - -A surprise would occur if we said this:: - - >>> a1.user = u1 - >>> a1 = session.merge(a1) - >>> session.commit() - sqlalchemy.orm.exc.FlushError: New instance
- with identity key (, (1,)) conflicts with - persistent instance
- -Why is that ? We weren't careful with our cascades. The assignment -of ``a1.user`` to a persistent object cascaded to the backref of ``User.addresses`` -and made our ``a1`` object pending, as though we had added it. Now we have -*two* ``Address`` objects in the session:: - - >>> a1 = Address() - >>> a1.user = u1 - >>> a1 in session - True - >>> existing_a1 in session - True - >>> a1 is existing_a1 - False - -Above, our ``a1`` is already pending in the session. The -subsequent :meth:`~.Session.merge` operation essentially -does nothing. Cascade can be configured via the :paramref:`_orm.relationship.cascade` -option on :func:`_orm.relationship`, although in this case it -would mean removing the ``save-update`` cascade from the -``User.addresses`` relationship - and usually, that behavior -is extremely convenient. The solution here would usually be to not assign -``a1.user`` to an object already persistent in the target -session. - -The ``cascade_backrefs=False`` option of :func:`_orm.relationship` -will also prevent the ``Address`` from -being added to the session via the ``a1.user = u1`` assignment. - -Further detail on cascade operation is at :ref:`unitofwork_cascades`. - -Another example of unexpected state:: - - >>> a1 = Address(id=existing_a1.id, user_id=u1.id) - >>> a1.user = None - >>> a1 = session.merge(a1) - >>> session.commit() - sqlalchemy.exc.IntegrityError: (IntegrityError) address.user_id - may not be NULL - -Above, the assignment of ``user`` takes precedence over the foreign key -assignment of ``user_id``, with the end result that ``None`` is applied -to ``user_id``, causing a failure. - -Most :meth:`~.Session.merge` issues can be examined by first checking - -is the object prematurely in the session ? - -.. sourcecode:: pycon+sql - - >>> a1 = Address(id=existing_a1, user_id=user.id) - >>> assert a1 not in session - >>> a1 = session.merge(a1) - -Or is there state on the object that we don't want ? Examining ``__dict__`` -is a quick way to check:: - - >>> a1 = Address(id=existing_a1, user_id=user.id) - >>> a1.user - >>> a1.__dict__ - {'_sa_instance_state': , - 'user_id': 1, - 'id': 1, - 'user': None} - >>> # we don't want user=None merged, remove it - >>> del a1.user - >>> a1 = session.merge(a1) - >>> # success - >>> session.commit() - -Expunging ---------- - -Expunge removes an object from the Session, sending persistent instances to -the detached state, and pending instances to the transient state: - -.. sourcecode:: python+sql - - session.expunge(obj1) - -To remove all items, call :meth:`~.Session.expunge_all` -(this method was formerly known as ``clear()``). - -.. _session_expire: - -Refreshing / Expiring ---------------------- - -:term:`Expiring` means that the database-persisted data held inside a series -of object attributes is erased, in such a way that when those attributes -are next accessed, a SQL query is emitted which will refresh that data from -the database. - -When we talk about expiration of data we are usually talking about an object -that is in the :term:`persistent` state. For example, if we load an object -as follows:: - - user = session.scalars(select(User).filter_by(name="user1").limit(1)).first() - -The above ``User`` object is persistent, and has a series of attributes -present; if we were to look inside its ``__dict__``, we'd see that state -loaded:: - - >>> user.__dict__ - { - 'id': 1, 'name': u'user1', - '_sa_instance_state': <...>, - } - -where ``id`` and ``name`` refer to those columns in the database. -``_sa_instance_state`` is a non-database-persisted value used by SQLAlchemy -internally (it refers to the :class:`.InstanceState` for the instance. -While not directly relevant to this section, if we want to get at it, -we should use the :func:`_sa.inspect` function to access it). - -At this point, the state in our ``User`` object matches that of the loaded -database row. But upon expiring the object using a method such as -:meth:`.Session.expire`, we see that the state is removed:: - - >>> session.expire(user) - >>> user.__dict__ - {'_sa_instance_state': <...>} - -We see that while the internal "state" still hangs around, the values which -correspond to the ``id`` and ``name`` columns are gone. If we were to access -one of these columns and are watching SQL, we'd see this: - -.. sourcecode:: pycon+sql - - >>> print(user.name) - {execsql}SELECT user.id AS user_id, user.name AS user_name - FROM user - WHERE user.id = ? - (1,) - {stop}user1 - -Above, upon accessing the expired attribute ``user.name``, the ORM initiated -a :term:`lazy load` to retrieve the most recent state from the database, -by emitting a SELECT for the user row to which this user refers. Afterwards, -the ``__dict__`` is again populated:: - - >>> user.__dict__ - { - 'id': 1, 'name': u'user1', - '_sa_instance_state': <...>, - } - -.. note:: While we are peeking inside of ``__dict__`` in order to see a bit - of what SQLAlchemy does with object attributes, we **should not modify** - the contents of ``__dict__`` directly, at least as far as those attributes - which the SQLAlchemy ORM is maintaining (other attributes outside of SQLA's - realm are fine). This is because SQLAlchemy uses :term:`descriptors` in - order to track the changes we make to an object, and when we modify ``__dict__`` - directly, the ORM won't be able to track that we changed something. - -Another key behavior of both :meth:`~.Session.expire` and :meth:`~.Session.refresh` -is that all un-flushed changes on an object are discarded. That is, -if we were to modify an attribute on our ``User``:: - - >>> user.name = "user2" - -but then we call :meth:`~.Session.expire` without first calling :meth:`~.Session.flush`, -our pending value of ``'user2'`` is discarded:: - - >>> session.expire(user) - >>> user.name - 'user1' - -The :meth:`~.Session.expire` method can be used to mark as "expired" all ORM-mapped -attributes for an instance:: - - # expire all ORM-mapped attributes on obj1 - session.expire(obj1) - -it can also be passed a list of string attribute names, referring to specific -attributes to be marked as expired:: - - # expire only attributes obj1.attr1, obj1.attr2 - session.expire(obj1, ["attr1", "attr2"]) - -The :meth:`.Session.expire_all` method allows us to essentially call -:meth:`.Session.expire` on all objects contained within the :class:`.Session` -at once:: - - session.expire_all() - -The :meth:`~.Session.refresh` method has a similar interface, but instead -of expiring, it emits an immediate SELECT for the object's row immediately:: - - # reload all attributes on obj1 - session.refresh(obj1) - -:meth:`~.Session.refresh` also accepts a list of string attribute names, -but unlike :meth:`~.Session.expire`, expects at least one name to -be that of a column-mapped attribute:: - - # reload obj1.attr1, obj1.attr2 - session.refresh(obj1, ["attr1", "attr2"]) - -.. tip:: - - An alternative method of refreshing which is often more flexible is to - use the :ref:`orm_queryguide_populate_existing` feature of the ORM, - available for :term:`2.0 style` queries with :func:`_sql.select` as well - as from the :meth:`_orm.Query.populate_existing` method of :class:`_orm.Query` - within :term:`1.x style` queries. Using this execution option, - all of the ORM objects returned in the result set of the statement - will be refreshed with data from the database:: - - stmt = ( - select(User) - .execution_options(populate_existing=True) - .where((User.name.in_(["a", "b", "c"]))) - ) - for user in session.execute(stmt).scalars(): - print(user) # will be refreshed for those columns that came back from the query - - See :ref:`orm_queryguide_populate_existing` for further detail. - - -What Actually Loads -~~~~~~~~~~~~~~~~~~~ - -The SELECT statement that's emitted when an object marked with :meth:`~.Session.expire` -or loaded with :meth:`~.Session.refresh` varies based on several factors, including: - -* The load of expired attributes is triggered from **column-mapped attributes only**. - While any kind of attribute can be marked as expired, including a - :func:`_orm.relationship` - mapped attribute, accessing an expired :func:`_orm.relationship` - attribute will emit a load only for that attribute, using standard - relationship-oriented lazy loading. Column-oriented attributes, even if - expired, will not load as part of this operation, and instead will load when - any column-oriented attribute is accessed. - -* :func:`_orm.relationship`- mapped attributes will not load in response to - expired column-based attributes being accessed. - -* Regarding relationships, :meth:`~.Session.refresh` is more restrictive than - :meth:`.Session.expire` with regards to attributes that aren't column-mapped. - Calling :meth:`.Session.refresh` and passing a list of names that only includes - relationship-mapped attributes will actually raise an error. - In any case, non-eager-loading :func:`_orm.relationship` attributes will not be - included in any refresh operation. - -* :func:`_orm.relationship` attributes configured as "eager loading" via the - :paramref:`_orm.relationship.lazy` parameter will load in the case of - :meth:`~.Session.refresh`, if either no attribute names are specified, or - if their names are included in the list of attributes to be - refreshed. - -* Attributes that are configured as :func:`.deferred` will not normally load, - during either the expired-attribute load or during a refresh. - An unloaded attribute that's :func:`.deferred` instead loads on its own when directly - accessed, or if part of a "group" of deferred attributes where an unloaded - attribute in that group is accessed. - -* For expired attributes that are loaded on access, a joined-inheritance table - mapping will emit a SELECT that typically only includes those tables for which - unloaded attributes are present. The action here is sophisticated enough - to load only the parent or child table, for example, if the subset of columns - that were originally expired encompass only one or the other of those tables. - -* When :meth:`~.Session.refresh` is used on a joined-inheritance table mapping, - the SELECT emitted will resemble that of when :meth:`.Session.query` is - used on the target object's class. This is typically all those tables that - are set up as part of the mapping. - - -When to Expire or Refresh -~~~~~~~~~~~~~~~~~~~~~~~~~ - -The :class:`.Session` uses the expiration feature automatically whenever -the transaction referred to by the session ends. Meaning, whenever :meth:`.Session.commit` -or :meth:`.Session.rollback` is called, all objects within the :class:`.Session` -are expired, using a feature equivalent to that of the :meth:`.Session.expire_all` -method. The rationale is that the end of a transaction is a -demarcating point at which there is no more context available in order to know -what the current state of the database is, as any number of other transactions -may be affecting it. Only when a new transaction starts can we again have access -to the current state of the database, at which point any number of changes -may have occurred. - -.. sidebar:: Transaction Isolation - - Of course, most databases are capable of handling - multiple transactions at once, even involving the same rows of data. When - a relational database handles multiple transactions involving the same - tables or rows, this is when the :term:`isolation` aspect of the database comes - into play. The isolation behavior of different databases varies considerably - and even on a single database can be configured to behave in different ways - (via the so-called :term:`isolation level` setting). In that sense, the :class:`.Session` - can't fully predict when the same SELECT statement, emitted a second time, - will definitely return the data we already have, or will return new data. - So as a best guess, it assumes that within the scope of a transaction, unless - it is known that a SQL expression has been emitted to modify a particular row, - there's no need to refresh a row unless explicitly told to do so. - -The :meth:`.Session.expire` and :meth:`.Session.refresh` methods are used in -those cases when one wants to force an object to re-load its data from the -database, in those cases when it is known that the current state of data -is possibly stale. Reasons for this might include: - -* some SQL has been emitted within the transaction outside of the - scope of the ORM's object handling, such as if a :meth:`_schema.Table.update` construct - were emitted using the :meth:`.Session.execute` method; - -* if the application - is attempting to acquire data that is known to have been modified in a - concurrent transaction, and it is also known that the isolation rules in effect - allow this data to be visible. - -The second bullet has the important caveat that "it is also known that the isolation rules in effect -allow this data to be visible." This means that it cannot be assumed that an -UPDATE that happened on another database connection will yet be visible here -locally; in many cases, it will not. This is why if one wishes to use -:meth:`.Session.expire` or :meth:`.Session.refresh` in order to view data between ongoing -transactions, an understanding of the isolation behavior in effect is essential. - -.. seealso:: - - :meth:`.Session.expire` - - :meth:`.Session.expire_all` - - :meth:`.Session.refresh` - - :ref:`orm_queryguide_populate_existing` - allows any ORM query - to refresh objects as they would be loaded normally, refreshing - all matching objects in the identity map against the results of a - SELECT statement. - - :term:`isolation` - glossary explanation of isolation which includes links - to Wikipedia. - - `The SQLAlchemy Session In-Depth `_ - a video + slides with an in-depth discussion of the object - lifecycle including the role of data expiration. diff --git a/doc/build/orm/versioning.rst b/doc/build/orm/versioning.rst deleted file mode 100644 index 87865917cdf..00000000000 --- a/doc/build/orm/versioning.rst +++ /dev/null @@ -1,254 +0,0 @@ -.. _mapper_version_counter: - -Configuring a Version Counter -============================= - -The :class:`_orm.Mapper` supports management of a :term:`version id column`, which -is a single table column that increments or otherwise updates its value -each time an ``UPDATE`` to the mapped table occurs. This value is checked each -time the ORM emits an ``UPDATE`` or ``DELETE`` against the row to ensure that -the value held in memory matches the database value. - -.. warning:: - - Because the versioning feature relies upon comparison of the **in memory** - record of an object, the feature only applies to the :meth:`.Session.flush` - process, where the ORM flushes individual in-memory rows to the database. - It does **not** take effect when performing - a multirow UPDATE or DELETE using :meth:`_query.Query.update` or :meth:`_query.Query.delete` - methods, as these methods only emit an UPDATE or DELETE statement but otherwise - do not have direct access to the contents of those rows being affected. - -The purpose of this feature is to detect when two concurrent transactions -are modifying the same row at roughly the same time, or alternatively to provide -a guard against the usage of a "stale" row in a system that might be re-using -data from a previous transaction without refreshing (e.g. if one sets ``expire_on_commit=False`` -with a :class:`.Session`, it is possible to re-use the data from a previous -transaction). - -.. topic:: Concurrent transaction updates - - When detecting concurrent updates within transactions, it is typically the - case that the database's transaction isolation level is below the level of - :term:`repeatable read`; otherwise, the transaction will not be exposed - to a new row value created by a concurrent update which conflicts with - the locally updated value. In this case, the SQLAlchemy versioning - feature will typically not be useful for in-transaction conflict detection, - though it still can be used for cross-transaction staleness detection. - - The database that enforces repeatable reads will typically either have locked the - target row against a concurrent update, or is employing some form - of multi version concurrency control such that it will emit an error - when the transaction is committed. SQLAlchemy's version_id_col is an alternative - which allows version tracking to occur for specific tables within a transaction - that otherwise might not have this isolation level set. - - .. seealso:: - - `Repeatable Read Isolation Level `_ - PostgreSQL's implementation of repeatable read, including a description of the error condition. - -Simple Version Counting ------------------------ - -The most straightforward way to track versions is to add an integer column -to the mapped table, then establish it as the ``version_id_col`` within the -mapper options:: - - class User(Base): - __tablename__ = "user" - - id = mapped_column(Integer, primary_key=True) - version_id = mapped_column(Integer, nullable=False) - name = mapped_column(String(50), nullable=False) - - __mapper_args__ = {"version_id_col": version_id} - -.. note:: It is **strongly recommended** that the ``version_id`` column - be made NOT NULL. The versioning feature **does not support** a NULL - value in the versioning column. - -Above, the ``User`` mapping tracks integer versions using the column -``version_id``. When an object of type ``User`` is first flushed, the -``version_id`` column will be given a value of "1". Then, an UPDATE -of the table later on will always be emitted in a manner similar to the -following: - -.. sourcecode:: sql - - UPDATE user SET version_id=:version_id, name=:name - WHERE user.id = :user_id AND user.version_id = :user_version_id - -- {"name": "new name", "version_id": 2, "user_id": 1, "user_version_id": 1} - -The above UPDATE statement is updating the row that not only matches -``user.id = 1``, it also is requiring that ``user.version_id = 1``, where "1" -is the last version identifier we've been known to use on this object. -If a transaction elsewhere has modified the row independently, this version id -will no longer match, and the UPDATE statement will report that no rows matched; -this is the condition that SQLAlchemy tests, that exactly one row matched our -UPDATE (or DELETE) statement. If zero rows match, that indicates our version -of the data is stale, and a :exc:`.StaleDataError` is raised. - -.. _custom_version_counter: - -Custom Version Counters / Types -------------------------------- - -Other kinds of values or counters can be used for versioning. Common types include -dates and GUIDs. When using an alternate type or counter scheme, SQLAlchemy -provides a hook for this scheme using the ``version_id_generator`` argument, -which accepts a version generation callable. This callable is passed the value of the current -known version, and is expected to return the subsequent version. - -For example, if we wanted to track the versioning of our ``User`` class -using a randomly generated GUID, we could do this (note that some backends -support a native GUID type, but we illustrate here using a simple string):: - - import uuid - - - class User(Base): - __tablename__ = "user" - - id = mapped_column(Integer, primary_key=True) - version_uuid = mapped_column(String(32), nullable=False) - name = mapped_column(String(50), nullable=False) - - __mapper_args__ = { - "version_id_col": version_uuid, - "version_id_generator": lambda version: uuid.uuid4().hex, - } - -The persistence engine will call upon ``uuid.uuid4()`` each time a -``User`` object is subject to an INSERT or an UPDATE. In this case, our -version generation function can disregard the incoming value of ``version``, -as the ``uuid4()`` function -generates identifiers without any prerequisite value. If we were using -a sequential versioning scheme such as numeric or a special character system, -we could make use of the given ``version`` in order to help determine the -subsequent value. - -.. seealso:: - - :ref:`custom_guid_type` - -.. _server_side_version_counter: - -Server Side Version Counters ----------------------------- - -The ``version_id_generator`` can also be configured to rely upon a value -that is generated by the database. In this case, the database would need -some means of generating new identifiers when a row is subject to an INSERT -as well as with an UPDATE. For the UPDATE case, typically an update trigger -is needed, unless the database in question supports some other native -version identifier. The PostgreSQL database in particular supports a system -column called `xmin `_ -which provides UPDATE versioning. We can make use -of the PostgreSQL ``xmin`` column to version our ``User`` -class as follows:: - - from sqlalchemy import FetchedValue - - - class User(Base): - __tablename__ = "user" - - id = mapped_column(Integer, primary_key=True) - name = mapped_column(String(50), nullable=False) - xmin = mapped_column("xmin", String, system=True, server_default=FetchedValue()) - - __mapper_args__ = {"version_id_col": xmin, "version_id_generator": False} - -With the above mapping, the ORM will rely upon the ``xmin`` column for -automatically providing the new value of the version id counter. - -.. topic:: creating tables that refer to system columns - - In the above scenario, as ``xmin`` is a system column provided by PostgreSQL, - we use the ``system=True`` argument to mark it as a system-provided - column, omitted from the ``CREATE TABLE`` statement. The datatype of this - column is an internal PostgreSQL type called ``xid`` which acts mostly - like a string, so we use the :class:`_types.String` datatype. - - -The ORM typically does not actively fetch the values of database-generated -values when it emits an INSERT or UPDATE, instead leaving these columns as -"expired" and to be fetched when they are next accessed, unless the ``eager_defaults`` -:class:`_orm.Mapper` flag is set. However, when a -server side version column is used, the ORM needs to actively fetch the newly -generated value. This is so that the version counter is set up *before* -any concurrent transaction may update it again. This fetching is also -best done simultaneously within the INSERT or UPDATE statement using :term:`RETURNING`, -otherwise if emitting a SELECT statement afterwards, there is still a potential -race condition where the version counter may change before it can be fetched. - -When the target database supports RETURNING, an INSERT statement for our ``User`` class will look -like this: - -.. sourcecode:: sql - - INSERT INTO "user" (name) VALUES (%(name)s) RETURNING "user".id, "user".xmin - -- {'name': 'ed'} - -Where above, the ORM can acquire any newly generated primary key values along -with server-generated version identifiers in one statement. When the backend -does not support RETURNING, an additional SELECT must be emitted for **every** -INSERT and UPDATE, which is much less efficient, and also introduces the possibility of -missed version counters: - -.. sourcecode:: sql - - INSERT INTO "user" (name) VALUES (%(name)s) - -- {'name': 'ed'} - - SELECT "user".version_id AS user_version_id FROM "user" where - "user".id = :param_1 - -- {"param_1": 1} - -It is *strongly recommended* that server side version counters only be used -when absolutely necessary and only on backends that support :term:`RETURNING`, -currently PostgreSQL, Oracle, MariaDB 10.5, SQLite 3.35, and SQL Server. - - -Programmatic or Conditional Version Counters --------------------------------------------- - -When ``version_id_generator`` is set to False, we can also programmatically -(and conditionally) set the version identifier on our object in the same way -we assign any other mapped attribute. Such as if we used our UUID example, but -set ``version_id_generator`` to ``False``, we can set the version identifier -at our choosing:: - - import uuid - - - class User(Base): - __tablename__ = "user" - - id = mapped_column(Integer, primary_key=True) - version_uuid = mapped_column(String(32), nullable=False) - name = mapped_column(String(50), nullable=False) - - __mapper_args__ = {"version_id_col": version_uuid, "version_id_generator": False} - - - u1 = User(name="u1", version_uuid=uuid.uuid4()) - - session.add(u1) - - session.commit() - - u1.name = "u2" - u1.version_uuid = uuid.uuid4() - - session.commit() - -We can update our ``User`` object without incrementing the version counter -as well; the value of the counter will remain unchanged, and the UPDATE -statement will still check against the previous value. This may be useful -for schemes where only certain classes of UPDATE are sensitive to concurrency -issues:: - - # will leave version_uuid unchanged - u1.name = "u3" - session.commit() diff --git a/doc/build/requirements.txt b/doc/build/requirements.txt index 9b9bffd36e5..572512f4243 100644 --- a/doc/build/requirements.txt +++ b/doc/build/requirements.txt @@ -1,6 +1,5 @@ git+https://github.com/sqlalchemyorg/changelog.git#egg=changelog git+https://github.com/sqlalchemyorg/sphinx-paramlinks.git#egg=sphinx-paramlinks -git+https://github.com/sqlalchemyorg/zzzeeksphinx.git#egg=zzzeeksphinx sphinx-copybutton==0.5.1 sphinx-autobuild typing-extensions diff --git a/doc/build/tutorial/data_insert.rst b/doc/build/tutorial/data_insert.rst index d0f6b236d5d..651455e730e 100644 --- a/doc/build/tutorial/data_insert.rst +++ b/doc/build/tutorial/data_insert.rst @@ -1,4 +1,4 @@ -.. highlight:: pycon+sql + highlight:: pycon+sql .. |prev| replace:: :doc:`data` .. |next| replace:: :doc:`data_select` @@ -157,62 +157,6 @@ method in conjunction with the :class:`_sql.Insert` construct, the will be expressed in the VALUES clause of the :class:`_sql.Insert` construct automatically. -.. deepalchemy:: - - Hi, welcome to the first edition of **Deep Alchemy**. The person on the - left is known as **The Alchemist**, and you'll note they are **not** a wizard, - as the pointy hat is not sticking upwards. The Alchemist comes around to - describe things that are generally **more advanced and/or tricky** and - additionally **not usually needed**, but for whatever reason they feel you - should know about this thing that SQLAlchemy can do. - - In this edition, towards the goal of having some interesting data in the - ``address_table`` as well, below is a more advanced example illustrating - how the :meth:`_sql.Insert.values` method may be used explicitly while at - the same time including for additional VALUES generated from the - parameters. A :term:`scalar subquery` is constructed, making use of the - :func:`_sql.select` construct introduced in the next section, and the - parameters used in the subquery are set up using an explicit bound - parameter name, established using the :func:`_sql.bindparam` construct. - - This is some slightly **deeper** alchemy just so that we can add related - rows without fetching the primary key identifiers from the ``user_table`` - operation into the application. Most Alchemists will simply use the ORM - which takes care of things like this for us. - - .. sourcecode:: pycon+sql - - >>> from sqlalchemy import select, bindparam - >>> scalar_subq = ( - ... select(user_table.c.id) - ... .where(user_table.c.name == bindparam("username")) - ... .scalar_subquery() - ... ) - - >>> with engine.connect() as conn: - ... result = conn.execute( - ... insert(address_table).values(user_id=scalar_subq), - ... [ - ... { - ... "username": "spongebob", - ... "email_address": "spongebob@sqlalchemy.org", - ... }, - ... {"username": "sandy", "email_address": "sandy@sqlalchemy.org"}, - ... {"username": "sandy", "email_address": "sandy@squirrelpower.org"}, - ... ], - ... ) - ... conn.commit() - {execsql}BEGIN (implicit) - INSERT INTO address (user_id, email_address) VALUES ((SELECT user_account.id - FROM user_account - WHERE user_account.name = ?), ?) - [...] [('spongebob', 'spongebob@sqlalchemy.org'), ('sandy', 'sandy@sqlalchemy.org'), - ('sandy', 'sandy@squirrelpower.org')] - COMMIT{stop} - - With that, we have some more interesting data in our tables that we will - make use of in the upcoming sections. - .. tip:: A true "empty" INSERT that inserts only the "defaults" for a table without including any explicit values at all is generated if we indicate :meth:`_sql.Insert.values` with no arguments; not every database backend diff --git a/doc/build/tutorial/data_select.rst b/doc/build/tutorial/data_select.rst index 03bc459a1f7..2cc4ee29d93 100644 --- a/doc/build/tutorial/data_select.rst +++ b/doc/build/tutorial/data_select.rst @@ -1683,7 +1683,7 @@ Oracle, and SQL Server. .. seealso:: - :ref:`postgresql_table_valued_overview` - in the :ref:`postgresql_toplevel` documentation. + ref_postgresql_table_valued_overview - in the :ref:`postgresql_toplevel` documentation. While many databases support table valued and other special forms, PostgreSQL tends to be where there is the most demand for these @@ -1719,7 +1719,7 @@ towards as ``value``, and then selected two of its three rows. .. seealso:: - :ref:`postgresql_table_valued` - in the :ref:`postgresql_toplevel` documentation - + ref_postgresql_table_valued - in the :ref:`postgresql_toplevel` documentation - this section will detail additional syntaxes such as special column derivations and "WITH ORDINALITY" that are known to work with PostgreSQL. @@ -1757,7 +1757,7 @@ it is usable for custom SQL functions:: .. seealso:: - :ref:`postgresql_column_valued` - in the :ref:`postgresql_toplevel` documentation. + ref_postgresql_column_valued - in the :ref:`postgresql_toplevel` documentation. .. _tutorial_casts: diff --git a/doc/build/tutorial/data_update.rst b/doc/build/tutorial/data_update.rst index a82f070a3f6..64c8bdd3c26 100644 --- a/doc/build/tutorial/data_update.rst +++ b/doc/build/tutorial/data_update.rst @@ -294,7 +294,7 @@ Facts about :attr:`_engine.CursorResult.rowcount`: * "rowcount" is used by the ORM :term:`unit of work` process to validate that an UPDATE or DELETE statement matched the expected number of rows, and is also essential for the ORM versioning feature documented at - :ref:`mapper_version_counter`. + ref_mapper_version_counter. Using RETURNING with UPDATE, DELETE ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/doc/build/tutorial/index.rst b/doc/build/tutorial/index.rst index ef4bb763457..e9477528feb 100644 --- a/doc/build/tutorial/index.rst +++ b/doc/build/tutorial/index.rst @@ -1,10 +1,6 @@ .. |tutorial_title| replace:: SQLAlchemy Unified Tutorial .. |next| replace:: :doc:`engine` -.. footer_topic:: |tutorial_title| - - Next Section: |next| - .. _unified_tutorial: .. rst-class:: orm_core diff --git a/doc/build/tutorial/metadata.rst b/doc/build/tutorial/metadata.rst index 8ee87480af6..5044e30f8f2 100644 --- a/doc/build/tutorial/metadata.rst +++ b/doc/build/tutorial/metadata.rst @@ -474,14 +474,14 @@ about these classes include: To automatically generate a full-featured ``__init__()`` method which provides for positional arguments as well as arguments with default keyword values, the dataclasses feature introduced at - :ref:`orm_declarative_native_dataclasses` may be used. It's of course + ref_orm_declarative_native_dataclasses may be used. It's of course always an option to use an explicit ``__init__()`` method as well. * The ``__repr__()`` methods are added so that we get a readable string output; there's no requirement for these methods to be here. As is the case with ``__init__()``, a ``__repr__()`` method can be generated automatically by using the - :ref:`dataclasses ` feature. + ref_orm_declarative_native_dataclasses feature. .. topic:: Where'd the old Declarative go? @@ -500,7 +500,7 @@ about these classes include: checkers such as Mypy and Pyright, without the need for plugins. Secondly, deriving the declarations from type annotations is part of SQLAlchemy's integration with Python dataclasses, which can now be - :ref:`generated natively ` from mappings. + ref_orm_declarative_native_dataclasses from mappings. For users who like the "old" way, but still desire their IDEs to not mistakenly report typing errors for their declarative mappings, the @@ -641,7 +641,7 @@ we declared explicitly:: .. seealso:: - Read more about table and schema reflection at :ref:`metadata_reflection_toplevel`. + Read more about table and schema reflection at ref_metadata_reflection_toplevel. For ORM-related variants of table reflection, the section :ref:`orm_declarative_reflected` includes an overview of the available diff --git a/doc/build/tutorial/tutorial_nav_include.rst b/doc/build/tutorial/tutorial_nav_include.rst index c4ee772a873..a6128595add 100644 --- a/doc/build/tutorial/tutorial_nav_include.rst +++ b/doc/build/tutorial/tutorial_nav_include.rst @@ -8,7 +8,4 @@ Previous: |prev| | Next: |next| -.. footer_topic:: |tutorial_title| - - Next Tutorial Section: |next| diff --git a/fix_links.py b/fix_links.py new file mode 100644 index 00000000000..bfb3c5fb707 --- /dev/null +++ b/fix_links.py @@ -0,0 +1,180 @@ +import os +import re +import sys + + +removed = { + "aiomysql", + "aiosqlite", + "associationproxy_toplevel", + "asyncio_events_run_async", + "asyncio_inspector", + "asyncio_install", + "asyncio_orm_avoid_lazyloads", + "asyncio_scoped_session", + "asyncio_toplevel", + "asyncmy", + "asyncpg_prepared_statement_cache", + "asyncpg_prepared_statement_name", + "automap_by_module", + "automap_intercepting_columns", + "automap_toplevel", + "azure_synapse_ignore_no_transaction_on_rollback", + "baked_in", + "baked_toplevel", + "baked_with_before_compile", + "change_3907", + "change_3953", + "change_4109", + "composite_association_proxy", + "composite_operations", + "context_default_functions", + "core_inspection_toplevel", + "custom_version_counter", + "cx_oracle_setinputsizes", + "dataclasses_pydantic", + "declarative_inheritance", + "declarative_many_to_many", + "declarative_mixins", + "declarative_relationship_eval", + "declarative_table_args", + "declarative_toplevel", + "defaults_client_invoked_sql", + "defaults_sequences", + "dialect-postgresql-asyncpg", + "dynamic_relationship", + "examples_adjacencylist", + "examples_asyncio", + "examples_caching", + "examples_inheritance", + "examples_instrumentation", + "examples_performance", + "examples_session_orm_events", + "examples_sharding", + "examples_toplevel", + "examples_versioned_history", + "examples_versioned_rows", + "examples_versioning", + "feature_joins_09", + "generic_functions", + "horizontal_sharding_toplevel", + "hybrid_pep484_naming", + "hybrids_toplevel", + "inspection_toplevel", + "legacy_is_orphan_addition", + "mapper_column_property_sql_expressions", + "mapper_column_property_sql_expressions_composed", + "mapper_composite", + "mapper_sql_expressions", + "mapper_version_counter", + "mapping_columns_toplevel", + "metadata_defaults", + "metadata_defaults_toplevel", + "metadata_reflection", + "metadata_reflection_dbagnostic_types", + "metadata_reflection_schemas", + "metadata_reflection_toplevel", + "migration_2992", + "migration_3061", + "mssql_comment_support", + "mssql_identity", + "mssql_indexes", + "mssql_insert_behavior", + "mssql_isolation_level", + "mssql_pyodbc_access_tokens", + "mssql_pyodbc_fastexecutemany", + "mssql_pyodbc_setinputsizes", + "mssql_reset_on_return", + "mssql_toplevel", + "mssql_triggers", + "multipart_schema_names", + "mutable_toplevel", + "mypy_declarative_mixins", + "mypy_toplevel", + "mysql_indexes", + "mysql_insert_on_duplicate_key_update", + "mysql_isolation_level", + "mysql_sql_mode", + "mysql_storage_engines", + "mysql_timestamp_onupdate", + "mysql_toplevel", + "oracledb", + "oracle_isolation_level", + "oracle_max_identifier_lengths", + "oracle_toplevel", + "orm_declarative_dataclasses", + "orm_declarative_dataclasses_declarative_table", + "orm_declarative_dataclasses_mixin", + "orm_declarative_dc_mixins", + "orm_declarative_native_dataclasses", + "orm_declarative_native_dataclasses_non_mapped_fields", + "orm_imperative_dataclasses", + "postgresql_alternate_search_path", + "postgresql_column_valued", + "postgresql_constraint_options", + "postgresql_indexes", + "postgresql_insert_on_conflict", + "postgresql_isolation_level", + "postgresql_match", + "postgresql_operator_classes", + "postgresql_psycopg", + "postgresql_readonly_deferrable", + "postgresql_reset_on_return", + "postgresql_simple_match", + "postgresql_table_valued", + "postgresql_table_valued_overview", + "proxying_dictionaries", + "psycopg2_executemany_mode", + "pysqlcipher", + "pysqlite_regexp", + "pysqlite_serializable", + "pysqlite_serializable ", + "pysqlite_threading_pooling", + "reflection_overriding_columns", + "relationships_backref", + "server_defaults", + "server_side_version_counter", + "session_expire", + "session_object_states", + "session_referencing_behavior", + "sqlite_autoincrement", + "sqlite_foreign_keys", + "sqlite_include_internal", + "sqlite_isolation_level", + "sqlite_on_conflict_ddl", + "sqlite_on_conflict_insert", + "sqlite_toplevel", + "triggered_columns", + "unitofwork_contextual", + "unitofwork_merging", + "write_only_relationship", +} + + +def fixlink(m): + print(f"MATCH: {m}") + if m.group(1) in removed: + return f"ref_{m.group(1)}" + else: + return m.group(0) + + +def fix(path): + print(f"path: {path}") + with open(path) as file_: + text = file_.read() + text = re.sub(r":ref:`.+ <(.+?)>`", fixlink, text) + print("NEXT PASS") + text = re.sub(r":ref:`(.*?)`", fixlink, text) + + with open(path, "w") as file_: + file_.write(text) + + +for root, dir_, files in os.walk(sys.argv[1]): + for file_ in files: + if ".venv" in root: + continue + if file_.endswith(".rst") or file_.endswith(".py"): + path = os.path.join(root, file_) + fix(path) diff --git a/lib/sqlalchemy/dialects/mysql/aiomysql.py b/lib/sqlalchemy/dialects/mysql/aiomysql.py index bc079ba17ac..19e35224574 100644 --- a/lib/sqlalchemy/dialects/mysql/aiomysql.py +++ b/lib/sqlalchemy/dialects/mysql/aiomysql.py @@ -17,13 +17,13 @@ SQLAlchemy’s continuous integration. As of September, 2021 the driver appears to be unmaintained and no longer functions for Python version 3.10, and additionally depends on a significantly outdated version of PyMySQL. - Please refer to the :ref:`asyncmy` dialect for current MySQL/MariaDB asyncio + Please refer to the ref_asyncmy dialect for current MySQL/MariaDB asyncio functionality. The aiomysql dialect is SQLAlchemy's second Python asyncio dialect. Using a special asyncio mediation layer, the aiomysql dialect is usable -as the backend for the :ref:`SQLAlchemy asyncio ` +as the backend for the ref_asyncio_toplevel extension package. This dialect should normally be used only with the diff --git a/lib/sqlalchemy/dialects/mysql/asyncmy.py b/lib/sqlalchemy/dialects/mysql/asyncmy.py index d3f809e6aed..ce245258b97 100644 --- a/lib/sqlalchemy/dialects/mysql/asyncmy.py +++ b/lib/sqlalchemy/dialects/mysql/asyncmy.py @@ -14,11 +14,11 @@ :url: https://github.com/long2ice/asyncmy .. note:: The asyncmy dialect as of September, 2021 was added to provide - MySQL/MariaDB asyncio compatibility given that the :ref:`aiomysql` database + MySQL/MariaDB asyncio compatibility given that the ref_aiomysql database driver has become unmaintained, however asyncmy is itself very new. Using a special asyncio mediation layer, the asyncmy dialect is usable -as the backend for the :ref:`SQLAlchemy asyncio ` +as the backend for the ref_asyncio_toplevel extension package. This dialect should normally be used only with the diff --git a/lib/sqlalchemy/dialects/mysql/base.py b/lib/sqlalchemy/dialects/mysql/base.py index eb9ccc60645..cd10db18c88 100644 --- a/lib/sqlalchemy/dialects/mysql/base.py +++ b/lib/sqlalchemy/dialects/mysql/base.py @@ -463,7 +463,7 @@ def connect(dbapi_connection, connection_record): available. * INSERT..ON DUPLICATE KEY UPDATE: See - :ref:`mysql_insert_on_duplicate_key_update` + ref_mysql_insert_on_duplicate_key_update * SELECT pragma, use :meth:`_expression.Select.prefix_with` and :meth:`_query.Query.prefix_with`:: @@ -810,7 +810,7 @@ def process(element, compiler, **kw): .. seealso:: - :ref:`mysql_storage_engines` + ref_mysql_storage_engines .. _mysql_unique_constraints: diff --git a/lib/sqlalchemy/dialects/mysql/dml.py b/lib/sqlalchemy/dialects/mysql/dml.py index 7c724c6f12f..cfab91aeefb 100644 --- a/lib/sqlalchemy/dialects/mysql/dml.py +++ b/lib/sqlalchemy/dialects/mysql/dml.py @@ -80,7 +80,7 @@ def inserted(self): .. seealso:: - :ref:`mysql_insert_on_duplicate_key_update` - example of how + ref_mysql_insert_on_duplicate_key_update - example of how to use :attr:`_expression.Insert.inserted` """ @@ -139,7 +139,7 @@ def on_duplicate_key_update(self, *args, **kw) -> Self: .. seealso:: - :ref:`mysql_insert_on_duplicate_key_update` + ref_mysql_insert_on_duplicate_key_update """ if args and kw: diff --git a/lib/sqlalchemy/dialects/postgresql/asyncpg.py b/lib/sqlalchemy/dialects/postgresql/asyncpg.py index c879205e4a1..80ec8f56565 100644 --- a/lib/sqlalchemy/dialects/postgresql/asyncpg.py +++ b/lib/sqlalchemy/dialects/postgresql/asyncpg.py @@ -16,7 +16,7 @@ The asyncpg dialect is SQLAlchemy's first Python asyncio dialect. Using a special asyncio mediation layer, the asyncpg dialect is usable -as the backend for the :ref:`SQLAlchemy asyncio ` +as the backend for the ref_asyncio_toplevel extension package. This dialect should normally be used only with the diff --git a/lib/sqlalchemy/dialects/postgresql/base.py b/lib/sqlalchemy/dialects/postgresql/base.py index ad5e346b76f..c334d12855f 100644 --- a/lib/sqlalchemy/dialects/postgresql/base.py +++ b/lib/sqlalchemy/dialects/postgresql/base.py @@ -198,7 +198,7 @@ def use_identity(element, compiler, **kw): :ref:`dbapi_autocommit` - :ref:`postgresql_readonly_deferrable` + ref_postgresql_readonly_deferrable :ref:`psycopg2_isolation_level` @@ -353,7 +353,7 @@ def set_search_path(dbapi_connection, connection_record): attribute set up. The PostgreSQL dialect can reflect tables from any schema, as outlined in -:ref:`metadata_reflection_schemas`. +ref_metadata_reflection_schemas. With regards to tables which these :class:`_schema.Table` objects refer to via foreign key constraint, a decision must be made as to how @@ -1067,7 +1067,7 @@ def set_search_path(dbapi_connection, connection_record): .. seealso:: - :ref:`postgresql_isolation_level` + ref_postgresql_isolation_level .. _postgresql_index_reflection: diff --git a/lib/sqlalchemy/dialects/postgresql/dml.py b/lib/sqlalchemy/dialects/postgresql/dml.py index e17b87d1e73..190645ec4ad 100644 --- a/lib/sqlalchemy/dialects/postgresql/dml.py +++ b/lib/sqlalchemy/dialects/postgresql/dml.py @@ -77,7 +77,7 @@ def excluded(self): .. seealso:: - :ref:`postgresql_insert_on_conflict` - example of how + ref_postgresql_insert_on_conflict - example of how to use :attr:`_expression.Insert.excluded` """ @@ -149,7 +149,7 @@ def on_conflict_do_update( .. seealso:: - :ref:`postgresql_insert_on_conflict` + ref_postgresql_insert_on_conflict """ self._post_values_clause = OnConflictDoUpdate( @@ -186,7 +186,7 @@ def on_conflict_do_nothing( .. seealso:: - :ref:`postgresql_insert_on_conflict` + ref_postgresql_insert_on_conflict """ self._post_values_clause = OnConflictDoNothing( diff --git a/lib/sqlalchemy/dialects/postgresql/ext.py b/lib/sqlalchemy/dialects/postgresql/ext.py index c79e90034d3..be2b066cd42 100644 --- a/lib/sqlalchemy/dialects/postgresql/ext.py +++ b/lib/sqlalchemy/dialects/postgresql/ext.py @@ -198,14 +198,14 @@ def __init__(self, *elements, **kw): :param ops: Optional dictionary. Used to define operator classes for the elements; works the same way as that of the - :ref:`postgresql_ops ` + ref_postgresql_operator_classes parameter specified to the :class:`_schema.Index` construct. .. versionadded:: 1.3.21 .. seealso:: - :ref:`postgresql_operator_classes` - general description of how + ref_postgresql_operator_classes - general description of how PostgreSQL operator classes are specified. """ diff --git a/lib/sqlalchemy/dialects/postgresql/pg8000.py b/lib/sqlalchemy/dialects/postgresql/pg8000.py index 3f01b00e85d..7eafad3e4ac 100644 --- a/lib/sqlalchemy/dialects/postgresql/pg8000.py +++ b/lib/sqlalchemy/dialects/postgresql/pg8000.py @@ -85,7 +85,7 @@ .. seealso:: - :ref:`postgresql_isolation_level` + ref_postgresql_isolation_level :ref:`psycopg2_isolation_level` diff --git a/lib/sqlalchemy/dialects/postgresql/psycopg2.py b/lib/sqlalchemy/dialects/postgresql/psycopg2.py index e28bd8fdad6..3e758f9896a 100644 --- a/lib/sqlalchemy/dialects/postgresql/psycopg2.py +++ b/lib/sqlalchemy/dialects/postgresql/psycopg2.py @@ -52,7 +52,7 @@ .. seealso:: - :ref:`psycopg2_executemany_mode` + ref_psycopg2_executemany_mode .. tip:: @@ -376,7 +376,7 @@ Psycopg2 Transaction Isolation Level ------------------------------------- -As discussed in :ref:`postgresql_isolation_level`, +As discussed in ref_postgresql_isolation_level, all PostgreSQL dialects support setting of transaction isolation level both via the ``isolation_level`` parameter passed to :func:`_sa.create_engine` , @@ -398,7 +398,7 @@ .. seealso:: - :ref:`postgresql_isolation_level` + ref_postgresql_isolation_level :ref:`pg8000_isolation_level` diff --git a/lib/sqlalchemy/dialects/postgresql/types.py b/lib/sqlalchemy/dialects/postgresql/types.py index edab239357b..fbe54a82928 100644 --- a/lib/sqlalchemy/dialects/postgresql/types.py +++ b/lib/sqlalchemy/dialects/postgresql/types.py @@ -244,7 +244,7 @@ class TSVECTOR(sqltypes.TypeEngine[str]): .. seealso:: - :ref:`postgresql_match` + ref_postgresql_match """ diff --git a/lib/sqlalchemy/dialects/sqlite/aiosqlite.py b/lib/sqlalchemy/dialects/sqlite/aiosqlite.py index f9c60efc17e..a0ca5135d34 100644 --- a/lib/sqlalchemy/dialects/sqlite/aiosqlite.py +++ b/lib/sqlalchemy/dialects/sqlite/aiosqlite.py @@ -24,7 +24,7 @@ interface that's useful for testing and prototyping purposes. Using a special asyncio mediation layer, the aiosqlite dialect is usable -as the backend for the :ref:`SQLAlchemy asyncio ` +as the backend for the ref_asyncio_toplevel extension package. This dialect should normally be used only with the diff --git a/lib/sqlalchemy/dialects/sqlite/base.py b/lib/sqlalchemy/dialects/sqlite/base.py index 6200cee5a32..53ac3ca86ef 100644 --- a/lib/sqlalchemy/dialects/sqlite/base.py +++ b/lib/sqlalchemy/dialects/sqlite/base.py @@ -214,7 +214,7 @@ def bi_c(element, compiler, **kw): SQLite's transactional scope is impacted by unresolved issues in the pysqlite driver, which defers BEGIN statements to a greater - degree than is often feasible. See the section :ref:`pysqlite_serializable` + degree than is often feasible. See the section ref_pysqlite_serializable for techniques to work around this behavior. .. seealso:: @@ -274,7 +274,7 @@ def bi_c(element, compiler, **kw): SQLite's SAVEPOINT feature is impacted by unresolved issues in the pysqlite driver, which defers BEGIN statements to a greater - degree than is often feasible. See the section :ref:`pysqlite_serializable` + degree than is often feasible. See the section ref_pysqlite_serializable for techniques to work around this behavior. Transactional DDL @@ -290,7 +290,7 @@ def bi_c(element, compiler, **kw): SQLite's transactional DDL is impacted by unresolved issues in the pysqlite driver, which fails to emit BEGIN and additionally forces a COMMIT to cancel any transaction when DDL is encountered. - See the section :ref:`pysqlite_serializable` + See the section ref_pysqlite_serializable for techniques to work around this behavior. .. _sqlite_foreign_keys: @@ -349,7 +349,7 @@ def set_sqlite_pragma(dbapi_connection, connection_record): .. seealso:: This section describes the :term:`DDL` version of "ON CONFLICT" for SQLite, which occurs within a CREATE TABLE statement. For "ON CONFLICT" as - applied to an INSERT statement, see :ref:`sqlite_on_conflict_insert`. + applied to an INSERT statement, see ref_sqlite_on_conflict_insert. SQLite supports a non-standard DDL clause known as ON CONFLICT which can be applied to primary key, unique, check, and not null constraints. In DDL, it is @@ -459,7 +459,7 @@ def set_sqlite_pragma(dbapi_connection, connection_record): .. seealso:: This section describes the :term:`DML` version of "ON CONFLICT" for SQLite, which occurs within an INSERT statement. For "ON CONFLICT" as - applied to a CREATE TABLE statement, see :ref:`sqlite_on_conflict_ddl`. + applied to a CREATE TABLE statement, see ref_sqlite_on_conflict_ddl. From version 3.24.0 onwards, SQLite supports "upserts" (update or insert) of rows into a table via the ``ON CONFLICT`` clause of the ``INSERT`` diff --git a/lib/sqlalchemy/dialects/sqlite/dml.py b/lib/sqlalchemy/dialects/sqlite/dml.py index 3f829076be7..296a1d40c55 100644 --- a/lib/sqlalchemy/dialects/sqlite/dml.py +++ b/lib/sqlalchemy/dialects/sqlite/dml.py @@ -53,7 +53,7 @@ class Insert(StandardInsert): .. seealso:: - :ref:`sqlite_on_conflict_insert` + ref_sqlite_on_conflict_insert """ diff --git a/lib/sqlalchemy/dialects/sqlite/pysqlcipher.py b/lib/sqlalchemy/dialects/sqlite/pysqlcipher.py index 28b900ea53d..1eeafd4459d 100644 --- a/lib/sqlalchemy/dialects/sqlite/pysqlcipher.py +++ b/lib/sqlalchemy/dialects/sqlite/pysqlcipher.py @@ -80,7 +80,7 @@ ---------------- The driver makes a change to the default pool behavior of pysqlite -as described in :ref:`pysqlite_threading_pooling`. The pysqlcipher driver +as described in ref_pysqlite_threading_pooling. The pysqlcipher driver has been observed to be significantly slower on connection than the pysqlite driver, most likely due to the encryption overhead, so the dialect here defaults to using the :class:`.SingletonThreadPool` diff --git a/lib/sqlalchemy/engine/cursor.py b/lib/sqlalchemy/engine/cursor.py index aaf2c1918e5..b9cce9edf14 100644 --- a/lib/sqlalchemy/engine/cursor.py +++ b/lib/sqlalchemy/engine/cursor.py @@ -1576,11 +1576,11 @@ def inserted_primary_key_rows(self): .. note:: As indicated below, in current SQLAlchemy versions this accessor is only useful beyond what's already supplied by :attr:`_engine.CursorResult.inserted_primary_key` when using the - :ref:`postgresql_psycopg2` dialect. Future versions hope to + psycopg2 dialect. Future versions hope to generalize this feature to more dialects. This accessor is added to support dialects that offer the feature - that is currently implemented by the :ref:`psycopg2_executemany_mode` + that is currently implemented by the ref_psycopg2_executemany_mode feature, currently **only the psycopg2 dialect**, which provides for many rows to be INSERTed at once while still retaining the behavior of being able to return server-generated primary key values. diff --git a/lib/sqlalchemy/engine/default.py b/lib/sqlalchemy/engine/default.py index 8992334ee6b..6351733923f 100644 --- a/lib/sqlalchemy/engine/default.py +++ b/lib/sqlalchemy/engine/default.py @@ -2150,7 +2150,7 @@ def _exec_default_clause_element(self, column, default, type_): """A dictionary of parameters applied to the current row. This attribute is only available in the context of a user-defined default - generation function, e.g. as described at :ref:`context_default_functions`. + generation function, e.g. as described at ref_context_default_functions. It consists of a dictionary which includes entries for each column/value pair that is to be part of the INSERT or UPDATE statement. The keys of the dictionary will be the key value of each :class:`_schema.Column`, @@ -2167,7 +2167,7 @@ def _exec_default_clause_element(self, column, default, type_): :meth:`.DefaultExecutionContext.get_current_parameters` - :ref:`context_default_functions` + ref_context_default_functions """ @@ -2176,7 +2176,7 @@ def get_current_parameters(self, isolate_multiinsert_groups=True): This method can only be used in the context of a user-defined default generation function, e.g. as described at - :ref:`context_default_functions`. When invoked, a dictionary is + ref_context_default_functions. When invoked, a dictionary is returned which includes entries for each column/value pair that is part of the INSERT or UPDATE statement. The keys of the dictionary will be the key value of each :class:`_schema.Column`, @@ -2201,7 +2201,7 @@ def get_current_parameters(self, isolate_multiinsert_groups=True): :attr:`.DefaultExecutionContext.current_parameters` - :ref:`context_default_functions` + ref_context_default_functions """ try: diff --git a/lib/sqlalchemy/engine/events.py b/lib/sqlalchemy/engine/events.py index c1b182a0ae1..1ce2cfbd07a 100644 --- a/lib/sqlalchemy/engine/events.py +++ b/lib/sqlalchemy/engine/events.py @@ -940,13 +940,13 @@ def do_setinputsizes( .. seealso:: - :ref:`mssql_pyodbc_setinputsizes` + ref_mssql_pyodbc_setinputsizes .. versionadded:: 1.2.9 .. seealso:: - :ref:`cx_oracle_setinputsizes` + ref_cx_oracle_setinputsizes """ pass diff --git a/lib/sqlalchemy/engine/interfaces.py b/lib/sqlalchemy/engine/interfaces.py index 0216c155d1f..74ba176d128 100644 --- a/lib/sqlalchemy/engine/interfaces.py +++ b/lib/sqlalchemy/engine/interfaces.py @@ -3388,7 +3388,7 @@ def register_custom_types(dbapi_connection, ...): .. seealso:: - :ref:`asyncio_events_run_async` + ref_asyncio_events_run_async """ return await_only(fn(self._connection)) diff --git a/lib/sqlalchemy/event/api.py b/lib/sqlalchemy/event/api.py index bb1dbea0fc9..6822cd7e28f 100644 --- a/lib/sqlalchemy/event/api.py +++ b/lib/sqlalchemy/event/api.py @@ -68,7 +68,7 @@ def unique_constraint_name(const, table): internal list upon discovery. This feature is not typically used or recommended by the SQLAlchemy maintainers, but is provided to ensure certain user defined functions can run before others, such as when - :ref:`Changing the sql_mode in MySQL `. + ref_mysql_sql_mode. :param bool named: When using named argument passing, the names listed in the function argument specification will be used as keys in the diff --git a/lib/sqlalchemy/ext/associationproxy.py b/lib/sqlalchemy/ext/associationproxy.py index 243c140f8c6..86361144847 100644 --- a/lib/sqlalchemy/ext/associationproxy.py +++ b/lib/sqlalchemy/ext/associationproxy.py @@ -156,13 +156,13 @@ def association_proxy( :ref:`cascade_scalar_deletes` - complete usage example - :param init: Specific to :ref:`orm_declarative_native_dataclasses`, + :param init: Specific to ref_orm_declarative_native_dataclasses, specifies if the mapped attribute should be part of the ``__init__()`` method as generated by the dataclass process. .. versionadded:: 2.0.0b4 - :param repr: Specific to :ref:`orm_declarative_native_dataclasses`, + :param repr: Specific to ref_orm_declarative_native_dataclasses, specifies if the attribute established by this :class:`.AssociationProxy` should be part of the ``__repr__()`` method as generated by the dataclass process. @@ -170,20 +170,20 @@ def association_proxy( .. versionadded:: 2.0.0b4 :param default_factory: Specific to - :ref:`orm_declarative_native_dataclasses`, specifies a default-value + ref_orm_declarative_native_dataclasses, specifies a default-value generation function that will take place as part of the ``__init__()`` method as generated by the dataclass process. .. versionadded:: 2.0.0b4 :param compare: Specific to - :ref:`orm_declarative_native_dataclasses`, indicates if this field + ref_orm_declarative_native_dataclasses, indicates if this field should be included in comparison operations when generating the ``__eq__()`` and ``__ne__()`` methods for the mapped class. .. versionadded:: 2.0.0b4 - :param kw_only: Specific to :ref:`orm_declarative_native_dataclasses`, + :param kw_only: Specific to ref_orm_declarative_native_dataclasses, indicates if this field should be marked as keyword-only when generating the ``__init__()`` method as generated by the dataclass process. diff --git a/lib/sqlalchemy/ext/asyncio/engine.py b/lib/sqlalchemy/ext/asyncio/engine.py index 6dfca463fb7..2010095100f 100644 --- a/lib/sqlalchemy/ext/asyncio/engine.py +++ b/lib/sqlalchemy/ext/asyncio/engine.py @@ -69,7 +69,7 @@ def create_async_engine(url: Union[str, URL], **kw: Any) -> AsyncEngine: Arguments passed to :func:`_asyncio.create_async_engine` are mostly identical to those passed to the :func:`_sa.create_engine` function. The specified dialect must be an asyncio-compatible dialect - such as :ref:`dialect-postgresql-asyncpg`. + such as ref_dialect-postgresql-asyncpg. .. versionadded:: 1.4 @@ -93,7 +93,7 @@ def async_engine_from_config( This function is analogous to the :func:`_sa.engine_from_config` function in SQLAlchemy Core, except that the requested dialect must be an - asyncio-compatible dialect such as :ref:`dialect-postgresql-asyncpg`. + asyncio-compatible dialect such as ref_dialect-postgresql-asyncpg. The argument signature of the function is identical to that of :func:`_sa.engine_from_config`. @@ -117,7 +117,7 @@ def create_async_pool_from_url(url: Union[str, URL], **kwargs: Any) -> Pool: Arguments passed to :func:`_asyncio.create_async_pool_from_url` are mostly identical to those passed to the :func:`_sa.create_pool_from_url` function. The specified dialect must be an asyncio-compatible dialect - such as :ref:`dialect-postgresql-asyncpg`. + such as ref_dialect-postgresql-asyncpg. .. versionadded:: 2.0.10 diff --git a/lib/sqlalchemy/ext/asyncio/scoping.py b/lib/sqlalchemy/ext/asyncio/scoping.py index 52eeb08281e..f7c50d92620 100644 --- a/lib/sqlalchemy/ext/asyncio/scoping.py +++ b/lib/sqlalchemy/ext/asyncio/scoping.py @@ -109,7 +109,7 @@ class async_scoped_session(Generic[_AS]): """Provides scoped management of :class:`.AsyncSession` objects. - See the section :ref:`asyncio_scoped_session` for usage details. + See the section ref_asyncio_scoped_session for usage details. .. versionadded:: 1.4.19 @@ -610,7 +610,7 @@ def expire( .. seealso:: - :ref:`session_expire` - introductory material + ref_session_expire - introductory material :meth:`.Session.expire` @@ -657,7 +657,7 @@ def expire_all(self) -> None: .. seealso:: - :ref:`session_expire` - introductory material + ref_session_expire - introductory material :meth:`.Session.expire` diff --git a/lib/sqlalchemy/ext/asyncio/session.py b/lib/sqlalchemy/ext/asyncio/session.py index dabe1824e28..f8f071c5a4f 100644 --- a/lib/sqlalchemy/ext/asyncio/session.py +++ b/lib/sqlalchemy/ext/asyncio/session.py @@ -987,7 +987,7 @@ def expire( .. seealso:: - :ref:`session_expire` - introductory material + ref_session_expire - introductory material :meth:`.Session.expire` @@ -1028,7 +1028,7 @@ def expire_all(self) -> None: .. seealso:: - :ref:`session_expire` - introductory material + ref_session_expire - introductory material :meth:`.Session.expire` diff --git a/lib/sqlalchemy/ext/automap.py b/lib/sqlalchemy/ext/automap.py index f378f729a0c..8ca14a3fb99 100644 --- a/lib/sqlalchemy/ext/automap.py +++ b/lib/sqlalchemy/ext/automap.py @@ -19,7 +19,7 @@ a well-integrated approach to the issue of expediently auto-generating ad-hoc mappings. -.. tip:: The :ref:`automap_toplevel` extension is geared towards a +.. tip:: The ref_automap_toplevel extension is geared towards a "zero declaration" approach, where a complete ORM model including classes and pre-named relationships can be generated on the fly from a database schema. For applications that still want to use explicit class declarations @@ -991,7 +991,7 @@ class that is produced by the :func:`.declarative.declarative_base` .. seealso:: - :ref:`automap_toplevel` + ref_automap_toplevel """ @@ -1034,7 +1034,7 @@ class that is produced by the :func:`.declarative.declarative_base` .. seealso:: - :ref:`automap_by_module` + ref_automap_by_module """ @@ -1133,7 +1133,7 @@ def prepare( .. seealso:: - :ref:`automap_by_module` + ref_automap_by_module :param name_for_scalar_relationship: callable function which will be used to produce relationship names for scalar relationships. Defaults @@ -1165,7 +1165,7 @@ def prepare( For an overview of multiple-schema automap including the use of additional naming conventions to resolve table name - conflicts, see the section :ref:`automap_by_module`. + conflicts, see the section ref_automap_by_module. .. versionadded:: 2.0 :meth:`.AutomapBase.prepare` supports being directly invoked any number of times, keeping track of tables diff --git a/lib/sqlalchemy/ext/declarative/extensions.py b/lib/sqlalchemy/ext/declarative/extensions.py index 2cb55a5ae88..b33c8e18d1e 100644 --- a/lib/sqlalchemy/ext/declarative/extensions.py +++ b/lib/sqlalchemy/ext/declarative/extensions.py @@ -190,7 +190,7 @@ class Manager(Employee): Using this approach, we can specify columns and properties that will take place on mapped subclasses, in the way that - we normally do as in :ref:`declarative_mixins`:: + we normally do as in ref_declarative_mixins:: from sqlalchemy.ext.declarative import AbstractConcreteBase diff --git a/lib/sqlalchemy/ext/horizontal_shard.py b/lib/sqlalchemy/ext/horizontal_shard.py index e1741fe526e..fa03d473c73 100644 --- a/lib/sqlalchemy/ext/horizontal_shard.py +++ b/lib/sqlalchemy/ext/horizontal_shard.py @@ -10,7 +10,7 @@ Defines a rudimental 'horizontal sharding' system which allows a Session to distribute queries and persistence operations across multiple databases. -For a usage example, see the :ref:`examples_sharding` example included in +For a usage example, see the ref_examples_sharding example included in the source distribution. .. deepalchemy:: The horizontal sharding extension is an advanced feature, @@ -102,13 +102,7 @@ def __call__( class ShardedQuery(Query[_T]): - """Query class used with :class:`.ShardedSession`. - - .. legacy:: The :class:`.ShardedQuery` is a subclass of the legacy - :class:`.Query` class. The :class:`.ShardedSession` now supports - 2.0 style execution via the :meth:`.ShardedSession.execute` method. - - """ + """Query class used with :class:`.ShardedSession`.""" def __init__(self, *args: Any, **kwargs: Any) -> None: super().__init__(*args, **kwargs) diff --git a/lib/sqlalchemy/ext/hybrid.py b/lib/sqlalchemy/ext/hybrid.py index 4e0fd460499..5b058a4d868 100644 --- a/lib/sqlalchemy/ext/hybrid.py +++ b/lib/sqlalchemy/ext/hybrid.py @@ -171,7 +171,7 @@ def _radius_expression(cls) -> ColumnElement[float]: :meth:`.hybrid_property.expression` modifier should mutate the existing hybrid object at ``Interval.radius`` in place, without creating a new object. Notes on this modifier and its -rationale are discussed in the next section :ref:`hybrid_pep484_naming`. +rationale are discussed in the next section ref_hybrid_pep484_naming. The use of ``@classmethod`` is optional, and is strictly to give typing tools a hint that ``cls`` in this case is expected to be the ``Interval`` class, and not an instance of ``Interval``. @@ -501,7 +501,7 @@ def _balance_expression(cls) -> SQLColumnExpression[Optional[Decimal]]: of joins in favor of the correlated subquery, which can portably be packed into a single column expression. A correlated subquery is more portable, but often performs more poorly at the SQL level. Using the same technique -illustrated at :ref:`mapper_column_property_sql_expressions`, +illustrated at ref_mapper_column_property_sql_expressions, we can adjust our ``SavingsAccount`` example to aggregate the balances for *all* accounts, and use a correlated subquery for the column expression:: @@ -1018,7 +1018,7 @@ def inplace(self) -> Self: .. seealso:: - :ref:`hybrid_pep484_naming` + ref_hybrid_pep484_naming """ return self @@ -1255,7 +1255,7 @@ def _radius_expression(cls) -> ColumnElement[float]: .. seealso:: - :ref:`hybrid_pep484_naming` + ref_hybrid_pep484_naming """ return hybrid_property._InPlace(self) diff --git a/lib/sqlalchemy/ext/instrumentation.py b/lib/sqlalchemy/ext/instrumentation.py index 688c762e72b..22e60b343e4 100644 --- a/lib/sqlalchemy/ext/instrumentation.py +++ b/lib/sqlalchemy/ext/instrumentation.py @@ -20,7 +20,7 @@ their own instrumentation. It is not intended for general use. For examples of how the instrumentation extension is used, -see the example :ref:`examples_instrumentation`. +see the example ref_examples_instrumentation. """ import weakref diff --git a/lib/sqlalchemy/ext/mutable.py b/lib/sqlalchemy/ext/mutable.py index 7d23f9fda7d..b8159714eef 100644 --- a/lib/sqlalchemy/ext/mutable.py +++ b/lib/sqlalchemy/ext/mutable.py @@ -233,14 +233,14 @@ def modified_json(instance, initiator): Composites are a special ORM feature which allow a single scalar attribute to be assigned an object value which represents information "composed" from one or more columns from the underlying mapped table. The usual example is that of -a geometric "point", and is introduced in :ref:`mapper_composite`. +a geometric "point", and is introduced in ref_mapper_composite. As is the case with :class:`.Mutable`, the user-defined composite class subclasses :class:`.MutableComposite` as a mixin, and detects and delivers change events to its parents via the :meth:`.MutableComposite.changed` method. In the case of a composite class, the detection is usually via the usage of the special Python method ``__setattr__()``. In the example below, we expand upon the ``Point`` -class introduced in :ref:`mapper_composite` to include +class introduced in ref_mapper_composite to include :class:`.MutableComposite` in its bases and to route attribute set events via ``__setattr__`` to the :meth:`.MutableComposite.changed` method:: diff --git a/lib/sqlalchemy/ext/serializer.py b/lib/sqlalchemy/ext/serializer.py index 706bff29fb0..159054254c4 100644 --- a/lib/sqlalchemy/ext/serializer.py +++ b/lib/sqlalchemy/ext/serializer.py @@ -9,10 +9,6 @@ """Serializer/Deserializer objects for usage with SQLAlchemy query structures, allowing "contextual" deserialization. -.. legacy:: - - The serializer extension is **legacy** and should not be used for - new development. Any SQLAlchemy query structure, either based on sqlalchemy.sql.* or sqlalchemy.orm.* can be used. The mappers, Tables, Columns, Session diff --git a/lib/sqlalchemy/orm/_orm_constructors.py b/lib/sqlalchemy/orm/_orm_constructors.py index 563fef3c503..55876bb2b6a 100644 --- a/lib/sqlalchemy/orm/_orm_constructors.py +++ b/lib/sqlalchemy/orm/_orm_constructors.py @@ -228,7 +228,7 @@ def mapped_column( :param default: Passed directly to the :paramref:`_schema.Column.default` parameter if the :paramref:`_orm.mapped_column.insert_default` parameter is not present. - Additionally, when used with :ref:`orm_declarative_native_dataclasses`, + Additionally, when used with ref_orm_declarative_native_dataclasses, indicates a default Python value that should be applied to the keyword constructor within the generated ``__init__()`` method. @@ -274,26 +274,26 @@ def mapped_column( .. versionadded:: 2.0.10 - :param init: Specific to :ref:`orm_declarative_native_dataclasses`, + :param init: Specific to ref_orm_declarative_native_dataclasses, specifies if the mapped attribute should be part of the ``__init__()`` method as generated by the dataclass process. - :param repr: Specific to :ref:`orm_declarative_native_dataclasses`, + :param repr: Specific to ref_orm_declarative_native_dataclasses, specifies if the mapped attribute should be part of the ``__repr__()`` method as generated by the dataclass process. :param default_factory: Specific to - :ref:`orm_declarative_native_dataclasses`, + ref_orm_declarative_native_dataclasses, specifies a default-value generation function that will take place as part of the ``__init__()`` method as generated by the dataclass process. :param compare: Specific to - :ref:`orm_declarative_native_dataclasses`, indicates if this field + ref_orm_declarative_native_dataclasses, indicates if this field should be included in comparison operations when generating the ``__eq__()`` and ``__ne__()`` methods for the mapped class. .. versionadded:: 2.0.0b4 :param kw_only: Specific to - :ref:`orm_declarative_native_dataclasses`, indicates if this field + ref_orm_declarative_native_dataclasses, indicates if this field should be marked as keyword-only when generating the ``__init__()``. :param \**kw: All remaining keyword arguments are passed through to the @@ -435,7 +435,7 @@ def column_property( .. seealso:: - :ref:`mapper_column_property_sql_expressions` - general use of + ref_mapper_column_property_sql_expressions - general use of :func:`_orm.column_property` to map SQL expressions :ref:`orm_imperative_table_column_options` - usage of @@ -619,7 +619,7 @@ def composite( ) -> Composite[Any]: r"""Return a composite column-based property for use with a Mapper. - See the mapping documentation section :ref:`mapper_composite` for a + See the mapping documentation section ref_mapper_composite for a full usage example. The :class:`.MapperProperty` returned by :func:`.composite` @@ -664,27 +664,27 @@ def composite( :param info: Optional data dictionary which will be populated into the :attr:`.MapperProperty.info` attribute of this object. - :param init: Specific to :ref:`orm_declarative_native_dataclasses`, + :param init: Specific to ref_orm_declarative_native_dataclasses, specifies if the mapped attribute should be part of the ``__init__()`` method as generated by the dataclass process. - :param repr: Specific to :ref:`orm_declarative_native_dataclasses`, + :param repr: Specific to ref_orm_declarative_native_dataclasses, specifies if the mapped attribute should be part of the ``__repr__()`` method as generated by the dataclass process. :param default_factory: Specific to - :ref:`orm_declarative_native_dataclasses`, + ref_orm_declarative_native_dataclasses, specifies a default-value generation function that will take place as part of the ``__init__()`` method as generated by the dataclass process. :param compare: Specific to - :ref:`orm_declarative_native_dataclasses`, indicates if this field + ref_orm_declarative_native_dataclasses, indicates if this field should be included in comparison operations when generating the ``__eq__()`` and ``__ne__()`` methods for the mapped class. .. versionadded:: 2.0.0b4 :param kw_only: Specific to - :ref:`orm_declarative_native_dataclasses`, indicates if this field + ref_orm_declarative_native_dataclasses, indicates if this field should be marked as keyword-only when generating the ``__init__()``. """ @@ -871,7 +871,7 @@ class of a particular set of mapped classes, to which the rule .. seealso:: - :ref:`examples_session_orm_events` - includes examples of using + ref_examples_session_orm_events - includes examples of using :func:`_orm.with_loader_criteria`. :ref:`do_orm_execute_global_criteria` - basic example on how to @@ -1009,7 +1009,7 @@ class SomeClass(Base): .. warning:: When passed as a Python-evaluable string, the argument is interpreted using Python's ``eval()`` function. **DO NOT PASS UNTRUSTED INPUT TO THIS STRING**. - See :ref:`declarative_relationship_eval` for details on + See ref_declarative_relationship_eval for details on declarative evaluation of :func:`_orm.relationship` arguments. The :paramref:`_orm.relationship.secondary` keyword argument is @@ -1031,7 +1031,7 @@ class SomeClass(Base): :ref:`self_referential_many_to_many` - Specifics on using many-to-many in a self-referential case. - :ref:`declarative_many_to_many` - Additional options when using + ref_declarative_many_to_many - Additional options when using Declarative. :ref:`association_pattern` - an alternative to @@ -1070,7 +1070,7 @@ class SomeClass(Base): .. seealso:: - :ref:`relationships_backref` - notes on using + ref_relationships_backref - notes on using :paramref:`_orm.relationship.backref` :ref:`tutorial_orm_related_objects` - in the :ref:`unified_tutorial`, @@ -1246,7 +1246,7 @@ class that will be synchronized with this one. It is usually .. warning:: When passed as a Python-evaluable string, the argument is interpreted using Python's ``eval()`` function. **DO NOT PASS UNTRUSTED INPUT TO THIS STRING**. - See :ref:`declarative_relationship_eval` for details on + See ref_declarative_relationship_eval for details on declarative evaluation of :func:`_orm.relationship` arguments. .. seealso:: @@ -1331,7 +1331,7 @@ class that will be synchronized with this one. It is usually * ``noload`` - no loading should occur at any time. The related collection will remain empty. The ``noload`` strategy is not recommended for general use. For a general use "never load" - approach, see :ref:`write_only_relationship` + approach, see ref_write_only_relationship * ``raise`` - lazy loading is disallowed; accessing the attribute, if its value were not already loaded via eager @@ -1366,13 +1366,13 @@ class that will be synchronized with this one. It is usually The ``write_only`` loader style is configured automatically when the :class:`_orm.WriteOnlyMapped` annotation is provided on the left hand side within a Declarative mapping. See the section - :ref:`write_only_relationship` for examples. + ref_write_only_relationship for examples. .. versionadded:: 2.0 .. seealso:: - :ref:`write_only_relationship` - in the :ref:`queryguide_toplevel` + ref_write_only_relationship - in the :ref:`queryguide_toplevel` * ``dynamic`` - the attribute will return a pre-configured :class:`_query.Query` object for all read @@ -1382,17 +1382,14 @@ class that will be synchronized with this one. It is usually The ``dynamic`` loader style is configured automatically when the :class:`_orm.DynamicMapped` annotation is provided on the left hand side within a Declarative mapping. See the section - :ref:`dynamic_relationship` for examples. + ref_dynamic_relationship for examples. - .. legacy:: The "dynamic" lazy loader strategy is the legacy form of - what is now the "write_only" strategy described in the section - :ref:`write_only_relationship`. .. seealso:: - :ref:`dynamic_relationship` - in the :ref:`queryguide_toplevel` + ref_dynamic_relationship - in the :ref:`queryguide_toplevel` - :ref:`write_only_relationship` - more generally useful approach + ref_write_only_relationship - more generally useful approach for large collections that should not fully load into memory * True - a synonym for 'select' @@ -1448,7 +1445,7 @@ class that will be synchronized with this one. It is usually .. warning:: When passed as a Python-evaluable string, the argument is interpreted using Python's ``eval()`` function. **DO NOT PASS UNTRUSTED INPUT TO THIS STRING**. - See :ref:`declarative_relationship_eval` for details on + See ref_declarative_relationship_eval for details on declarative evaluation of :func:`_orm.relationship` arguments. :param passive_deletes=False: @@ -1545,7 +1542,7 @@ class that will be synchronized with this one. It is usually .. warning:: When passed as a Python-evaluable string, the argument is interpreted using Python's ``eval()`` function. **DO NOT PASS UNTRUSTED INPUT TO THIS STRING**. - See :ref:`declarative_relationship_eval` for details on + See ref_declarative_relationship_eval for details on declarative evaluation of :func:`_orm.relationship` arguments. .. seealso:: @@ -1564,7 +1561,7 @@ class that will be synchronized with this one. It is usually .. warning:: When passed as a Python-evaluable string, the argument is interpreted using Python's ``eval()`` function. **DO NOT PASS UNTRUSTED INPUT TO THIS STRING**. - See :ref:`declarative_relationship_eval` for details on + See ref_declarative_relationship_eval for details on declarative evaluation of :func:`_orm.relationship` arguments. .. seealso:: @@ -1589,7 +1586,7 @@ class that will be synchronized with this one. It is usually .. seealso:: - :ref:`dynamic_relationship` - Introduction to "dynamic" + ref_dynamic_relationship - Introduction to "dynamic" relationship loaders. :param secondaryjoin: @@ -1606,7 +1603,7 @@ class that will be synchronized with this one. It is usually .. warning:: When passed as a Python-evaluable string, the argument is interpreted using Python's ``eval()`` function. **DO NOT PASS UNTRUSTED INPUT TO THIS STRING**. - See :ref:`declarative_relationship_eval` for details on + See ref_declarative_relationship_eval for details on declarative evaluation of :func:`_orm.relationship` arguments. .. seealso:: @@ -1730,26 +1727,26 @@ class that will be synchronized with this one. It is usually .. versionadded:: 1.3 - :param init: Specific to :ref:`orm_declarative_native_dataclasses`, + :param init: Specific to ref_orm_declarative_native_dataclasses, specifies if the mapped attribute should be part of the ``__init__()`` method as generated by the dataclass process. - :param repr: Specific to :ref:`orm_declarative_native_dataclasses`, + :param repr: Specific to ref_orm_declarative_native_dataclasses, specifies if the mapped attribute should be part of the ``__repr__()`` method as generated by the dataclass process. :param default_factory: Specific to - :ref:`orm_declarative_native_dataclasses`, + ref_orm_declarative_native_dataclasses, specifies a default-value generation function that will take place as part of the ``__init__()`` method as generated by the dataclass process. :param compare: Specific to - :ref:`orm_declarative_native_dataclasses`, indicates if this field + ref_orm_declarative_native_dataclasses, indicates if this field should be included in comparison operations when generating the ``__eq__()`` and ``__ne__()`` methods for the mapped class. .. versionadded:: 2.0.0b4 :param kw_only: Specific to - :ref:`orm_declarative_native_dataclasses`, indicates if this field + ref_orm_declarative_native_dataclasses, indicates if this field should be marked as keyword-only when generating the ``__init__()``. @@ -2001,7 +1998,7 @@ def dynamic_loader( relationship(SomeClass, lazy="dynamic") - See the section :ref:`dynamic_relationship` for more details + See the section ref_dynamic_relationship for more details on dynamic loading. """ @@ -2027,7 +2024,7 @@ def backref(name: str, **kwargs: Any) -> ORMBackrefArgument: .. seealso:: - :ref:`relationships_backref` - background on backrefs + ref_relationships_backref - background on backrefs """ diff --git a/lib/sqlalchemy/orm/base.py b/lib/sqlalchemy/orm/base.py index 5d19cf20ad6..9d75f194ef8 100644 --- a/lib/sqlalchemy/orm/base.py +++ b/lib/sqlalchemy/orm/base.py @@ -798,7 +798,7 @@ class Mapped( The :class:`_orm.Mapped` class represents attributes that are handled directly by the :class:`_orm.Mapper` class. It does not include other Python descriptor classes that are provided as extensions, including - :ref:`hybrids_toplevel` and the :ref:`associationproxy_toplevel`. + ref_hybrids_toplevel and the ref_associationproxy_toplevel. While these systems still make use of ORM-specific superclasses and structures, they are not :term:`instrumented` by the :class:`_orm.Mapper` and instead provide their own functionality @@ -883,9 +883,6 @@ class DynamicMapped(_MappedAnnotationBase[_T]): to indicate that the ``lazy="dynamic"`` loader strategy should be used for a particular :func:`_orm.relationship`. - .. legacy:: The "dynamic" lazy loader strategy is the legacy form of what - is now the "write_only" strategy described in the section - :ref:`write_only_relationship`. E.g.:: @@ -896,13 +893,13 @@ class User(Base): cascade="all,delete-orphan" ) - See the section :ref:`dynamic_relationship` for background. + See the section ref_dynamic_relationship for background. .. versionadded:: 2.0 .. seealso:: - :ref:`dynamic_relationship` - complete background + ref_dynamic_relationship - complete background :class:`.WriteOnlyMapped` - fully 2.0 style version @@ -938,13 +935,13 @@ class User(Base): cascade="all,delete-orphan" ) - See the section :ref:`write_only_relationship` for background. + See the section ref_write_only_relationship for background. .. versionadded:: 2.0 .. seealso:: - :ref:`write_only_relationship` - complete background + ref_write_only_relationship - complete background :class:`.DynamicMapped` - includes legacy :class:`_orm.Query` support diff --git a/lib/sqlalchemy/orm/decl_api.py b/lib/sqlalchemy/orm/decl_api.py index 2f8289acfe7..ecb1993bceb 100644 --- a/lib/sqlalchemy/orm/decl_api.py +++ b/lib/sqlalchemy/orm/decl_api.py @@ -509,7 +509,7 @@ class MyModel(MyMixin, Base): The :func:`_orm.declarative_mixin` decorator currently does not modify the given class in any way; it's current purpose is strictly to assist - the :ref:`Mypy plugin ` in being able to identify + the ref_mypy_toplevel in being able to identify SQLAlchemy declarative mixin classes when no other context is present. .. versionadded:: 1.4.6 @@ -518,8 +518,8 @@ class MyModel(MyMixin, Base): :ref:`orm_mixins_toplevel` - :ref:`mypy_declarative_mixins` - in the - :ref:`Mypy plugin documentation ` + ref_mypy_declarative_mixins - in the + ref_mypy_toplevel """ # noqa: E501 @@ -573,7 +573,7 @@ class MappedAsDataclass(metaclass=DCTransformDeclarative): .. seealso:: - :ref:`orm_declarative_native_dataclasses` - complete background + ref_orm_declarative_native_dataclasses - complete background on SQLAlchemy native dataclass mapping .. versionadded:: 2.0 @@ -1594,7 +1594,7 @@ def mapped_as_dataclass( .. seealso:: - :ref:`orm_declarative_native_dataclasses` - complete background + ref_orm_declarative_native_dataclasses - complete background on SQLAlchemy native dataclass mapping diff --git a/lib/sqlalchemy/orm/descriptor_props.py b/lib/sqlalchemy/orm/descriptor_props.py index 2e57fd0f4eb..af94773d7de 100644 --- a/lib/sqlalchemy/orm/descriptor_props.py +++ b/lib/sqlalchemy/orm/descriptor_props.py @@ -187,7 +187,7 @@ class CompositeProperty( .. seealso:: - :ref:`mapper_composite` + ref_mapper_composite """ @@ -729,7 +729,7 @@ class Comparator(PropComparator[_PT]): """Produce boolean, comparison, and other operators for :class:`.Composite` attributes. - See the example in :ref:`composite_operations` for an overview + See the example in ref_composite_operations for an overview of usage , as well as the documentation for :class:`.PropComparator`. .. seealso:: @@ -858,7 +858,7 @@ class Composite(CompositeProperty[_T], _DeclarativeMapped[_T]): .. seealso:: - :ref:`mapper_composite` + ref_mapper_composite """ diff --git a/lib/sqlalchemy/orm/dynamic.py b/lib/sqlalchemy/orm/dynamic.py index 7514d86cd79..9a067bca27b 100644 --- a/lib/sqlalchemy/orm/dynamic.py +++ b/lib/sqlalchemy/orm/dynamic.py @@ -12,8 +12,6 @@ Dynamic collections act like Query() objects for read operations and support basic add/delete mutation. -.. legacy:: the "dynamic" loader is a legacy feature, superseded by the - "write_only" loader. """ diff --git a/lib/sqlalchemy/orm/events.py b/lib/sqlalchemy/orm/events.py index 9a1fd569a31..c23ef3ffa3b 100644 --- a/lib/sqlalchemy/orm/events.py +++ b/lib/sqlalchemy/orm/events.py @@ -579,7 +579,7 @@ def refresh_flush( :ref:`orm_server_defaults` - :ref:`metadata_defaults_toplevel` + ref_metadata_defaults_toplevel """ @@ -962,7 +962,7 @@ def after_mapper_constructed( .. seealso:: - :ref:`examples_versioning` - an example which illustrates the use + ref_examples_versioning - an example which illustrates the use of the :meth:`_orm.MapperEvents.before_mapper_configured` event to create new mappers to record change-audit histories on objects. @@ -1735,14 +1735,14 @@ def do_orm_execute(self, orm_execute_state: ORMExecuteState) -> None: and parameters as well as an option that allows programmatic invocation of the statement at any point. - :ref:`examples_session_orm_events` - includes examples of using + ref_examples_session_orm_events - includes examples of using :meth:`_orm.SessionEvents.do_orm_execute` - :ref:`examples_caching` - an example of how to integrate + ref_examples_caching - an example of how to integrate Dogpile caching with the ORM :class:`_orm.Session` making use of the :meth:`_orm.SessionEvents.do_orm_execute` event hook. - :ref:`examples_sharding` - the Horizontal Sharding example / + ref_examples_sharding - the Horizontal Sharding example / extension relies upon the :meth:`_orm.SessionEvents.do_orm_execute` event hook to invoke a SQL statement on multiple backends and return a merged result. @@ -2110,13 +2110,6 @@ def after_bulk_update(self, update_context: _O) -> None: """Event for after the legacy :meth:`_orm.Query.update` method has been called. - .. legacy:: The :meth:`_orm.SessionEvents.after_bulk_update` method - is a legacy event hook as of SQLAlchemy 2.0. The event - **does not participate** in :term:`2.0 style` invocations - using :func:`_dml.update` documented at - :ref:`orm_queryguide_update_delete_where`. For 2.0 style use, - the :meth:`_orm.SessionEvents.do_orm_execute` hook will intercept - these calls. :param update_context: an "update context" object which contains details about the update, including these attributes: @@ -2156,13 +2149,6 @@ def after_bulk_delete(self, delete_context: _O) -> None: """Event for after the legacy :meth:`_orm.Query.delete` method has been called. - .. legacy:: The :meth:`_orm.SessionEvents.after_bulk_delete` method - is a legacy event hook as of SQLAlchemy 2.0. The event - **does not participate** in :term:`2.0 style` invocations - using :func:`_dml.delete` documented at - :ref:`orm_queryguide_update_delete_where`. For 2.0 style use, - the :meth:`_orm.SessionEvents.do_orm_execute` hook will intercept - these calls. :param delete_context: a "delete context" object which contains details about the update, including these attributes: @@ -2878,7 +2864,7 @@ def _init_some_attribute(target, dict_, value): we indicate that this value is to be persisted to the database. This supersedes the use of ``SOME_CONSTANT`` in the default generator for the :class:`_schema.Column`. The ``active_column_defaults.py`` - example given at :ref:`examples_instrumentation` illustrates using + example given at ref_examples_instrumentation illustrates using the same approach for a changing default, e.g. a timestamp generator. In this particular example, it is not strictly necessary to do this since ``SOME_CONSTANT`` would be part of the @@ -2939,7 +2925,7 @@ def _init_some_attribute(target, dict_, value): :class:`.AttributeEvents` - background on listener options such as propagation to subclasses. - :ref:`examples_instrumentation` - see the + ref_examples_instrumentation - see the ``active_column_defaults.py`` example. """ @@ -3044,11 +3030,6 @@ class QueryEvents(event.Events[Query[Any]]): """Represent events within the construction of a :class:`_query.Query` object. - .. legacy:: The :class:`_orm.QueryEvents` event methods are legacy - as of SQLAlchemy 2.0, and only apply to direct use of the - :class:`_orm.Query` object. They are not used for :term:`2.0 style` - statements. For events to intercept and modify 2.0 style ORM use, - use the :meth:`_orm.SessionEvents.do_orm_execute` hook. The :class:`_orm.QueryEvents` hooks are now superseded by the @@ -3070,7 +3051,7 @@ def before_compile(self, query: Query[Any]) -> None: the :meth:`_orm.QueryEvents.before_compile` event is **no longer used** for ORM-level attribute loads, such as loads of deferred or expired attributes as well as relationship loaders. See the - new examples in :ref:`examples_session_orm_events` which + new examples in ref_examples_session_orm_events which illustrate new ways of intercepting and modifying ORM queries for the most common purpose of adding arbitrary filter criteria. @@ -3121,7 +3102,7 @@ def my_event(query): :meth:`.QueryEvents.before_compile_delete` - :ref:`baked_with_before_compile` + ref_baked_with_before_compile """ diff --git a/lib/sqlalchemy/orm/mapper.py b/lib/sqlalchemy/orm/mapper.py index 731983ff47d..863c0286c7d 100644 --- a/lib/sqlalchemy/orm/mapper.py +++ b/lib/sqlalchemy/orm/mapper.py @@ -432,7 +432,7 @@ class User(Base): be consistent in more scenarios independently of whether or not an orphan object has been flushed yet or not. - See the change note and example at :ref:`legacy_is_orphan_addition` + See the change note and example at ref_legacy_is_orphan_addition for more detail on this change. :param non_primary: Specify that this :class:`_orm.Mapper` @@ -668,7 +668,7 @@ def set_identity(instance, *arg, **kw): .. seealso:: - :ref:`mapper_version_counter` - discussion of version counting + ref_mapper_version_counter - discussion of version counting and rationale. :param version_id_generator: Define how new version ids should @@ -682,14 +682,14 @@ def generate_version(version): Alternatively, server-side versioning functions such as triggers, or programmatic versioning schemes outside of the version id generator may be used, by specifying the value ``False``. - Please see :ref:`server_side_version_counter` for a discussion + Please see ref_server_side_version_counter for a discussion of important points when using this option. .. seealso:: - :ref:`custom_version_counter` + ref_custom_version_counter - :ref:`server_side_version_counter` + ref_server_side_version_counter :param with_polymorphic: A tuple in the form ``(, diff --git a/lib/sqlalchemy/orm/properties.py b/lib/sqlalchemy/orm/properties.py index 916b9d901d8..97032e07007 100644 --- a/lib/sqlalchemy/orm/properties.py +++ b/lib/sqlalchemy/orm/properties.py @@ -283,7 +283,7 @@ class File(Base): .. seealso:: - :ref:`mapper_column_property_sql_expressions_composed` + ref_mapper_column_property_sql_expressions_composed """ return self.columns[0] diff --git a/lib/sqlalchemy/orm/query.py b/lib/sqlalchemy/orm/query.py index 5cd7cc11704..c28b25d78fa 100644 --- a/lib/sqlalchemy/orm/query.py +++ b/lib/sqlalchemy/orm/query.py @@ -168,10 +168,6 @@ class Query( """ORM-level SQL construction object. - .. legacy:: The ORM :class:`.Query` object is a legacy construct - as of SQLAlchemy 2.0. See the notes at the top of - :ref:`query_api_toplevel` for an overview, including links to migration - documentation. :class:`_query.Query` objects are normally initially generated using the :meth:`~.Session.query` method of :class:`.Session`, and in @@ -1761,7 +1757,7 @@ def execution_options(self, **kwargs: Any) -> Self: statement executions, as the :class:`_orm.Session` will not track objects from different schema translate maps within a single session. For multiple schema translate maps within the scope of a - single :class:`_orm.Session`, see :ref:`examples_sharding`. + single :class:`_orm.Session`, see ref_examples_sharding. .. seealso:: @@ -3003,7 +2999,7 @@ def merge_result( The 'load' argument is the same as that of :meth:`.Session.merge`. For an example of how :meth:`_query.Query.merge_result` is used, see - the source code for the example :ref:`examples_caching`, where + the source code for the example ref_examples_caching, where :meth:`_query.Query.merge_result` is used to efficiently restore state from a cache back into a target :class:`.Session`. diff --git a/lib/sqlalchemy/orm/scoping.py b/lib/sqlalchemy/orm/scoping.py index 19217ec32e7..05ec0299e5d 100644 --- a/lib/sqlalchemy/orm/scoping.py +++ b/lib/sqlalchemy/orm/scoping.py @@ -143,11 +143,11 @@ def __get__(self, instance: Any, owner: Type[_T]) -> Query[_T]: class scoped_session(Generic[_S]): """Provides scoped management of :class:`.Session` objects. - See :ref:`unitofwork_contextual` for a tutorial. + See ref_unitofwork_contextual for a tutorial. .. note:: - When using :ref:`asyncio_toplevel`, the async-compatible + When using ref_asyncio_toplevel, the async-compatible :class:`_asyncio.async_scoped_session` class should be used in place of :class:`.scoped_session`. @@ -266,9 +266,6 @@ def query_property( :class:`_query.Query` object against the class and the current :class:`.Session` when called. - .. legacy:: The :meth:`_orm.scoped_session.query_property` accessor - is specific to the legacy :class:`.Query` object and is not - considered to be part of :term:`2.0-style` ORM use. e.g.:: @@ -463,7 +460,7 @@ def begin_nested(self) -> SessionTransaction: :ref:`session_begin_nested` - :ref:`pysqlite_serializable` - special workarounds required + ref_pysqlite_serializable - special workarounds required with the SQLite driver in order for SAVEPOINT to work correctly. @@ -545,7 +542,7 @@ def commit(self) -> None: :ref:`unitofwork_transaction` - :ref:`asyncio_orm_avoid_lazyloads` + ref_asyncio_orm_avoid_lazyloads """ # noqa: E501 @@ -765,7 +762,7 @@ def expire( .. seealso:: - :ref:`session_expire` - introductory material + ref_session_expire - introductory material :meth:`.Session.expire` @@ -806,7 +803,7 @@ def expire_all(self) -> None: .. seealso:: - :ref:`session_expire` - introductory material + ref_session_expire - introductory material :meth:`.Session.expire` @@ -1189,19 +1186,6 @@ def bulk_save_objects( Proxied for the :class:`_orm.Session` class on behalf of the :class:`_orm.scoping.scoped_session` class. - .. legacy:: - - This method is a legacy feature as of the 2.0 series of - SQLAlchemy. For modern bulk INSERT and UPDATE, see - the sections :ref:`orm_queryguide_bulk_insert` and - :ref:`orm_queryguide_bulk_update`. - - For general INSERT and UPDATE of existing ORM mapped objects, - prefer standard :term:`unit of work` data management patterns, - introduced in the :ref:`unified_tutorial` at - :ref:`tutorial_orm_data_manipulation`. SQLAlchemy 2.0 - now uses :ref:`engine_insertmanyvalues` with modern dialects - which solves previous issues of bulk INSERT slowness. :param objects: a sequence of mapped object instances. The mapped objects are persisted as is, and are **not** associated with the @@ -1275,14 +1259,6 @@ def bulk_insert_mappings( Proxied for the :class:`_orm.Session` class on behalf of the :class:`_orm.scoping.scoped_session` class. - .. legacy:: - - This method is a legacy feature as of the 2.0 series of - SQLAlchemy. For modern bulk INSERT and UPDATE, see - the sections :ref:`orm_queryguide_bulk_insert` and - :ref:`orm_queryguide_bulk_update`. The 2.0 API shares - implementation details with this method and adds new features - as well. :param mapper: a mapped class, or the actual :class:`_orm.Mapper` object, @@ -1358,14 +1334,6 @@ def bulk_update_mappings( Proxied for the :class:`_orm.Session` class on behalf of the :class:`_orm.scoping.scoped_session` class. - .. legacy:: - - This method is a legacy feature as of the 2.0 series of - SQLAlchemy. For modern bulk INSERT and UPDATE, see - the sections :ref:`orm_queryguide_bulk_insert` and - :ref:`orm_queryguide_bulk_update`. The 2.0 API shares - implementation details with this method and adds new features - as well. :param mapper: a mapped class, or the actual :class:`_orm.Mapper` object, @@ -1423,7 +1391,7 @@ def merge( This operation cascades to associated instances if the association is mapped with ``cascade="merge"``. - See :ref:`unitofwork_merging` for a detailed discussion of merging. + See ref_unitofwork_merging for a detailed discussion of merging. :param instance: Instance to be merged. :param load: Boolean, when False, :meth:`.merge` switches into @@ -1658,7 +1626,7 @@ def refresh( .. seealso:: - :ref:`session_expire` - introductory material + ref_session_expire - introductory material :meth:`.Session.expire` diff --git a/lib/sqlalchemy/orm/session.py b/lib/sqlalchemy/orm/session.py index 792b59e817f..353e4a4db88 100644 --- a/lib/sqlalchemy/orm/session.py +++ b/lib/sqlalchemy/orm/session.py @@ -1589,7 +1589,7 @@ def __init__( .. tip:: When using SQLite, the SQLite driver included through Python 3.11 does not handle SAVEPOINTs correctly in all cases without workarounds. See the section - :ref:`pysqlite_serializable` for details on current workarounds. + ref_pysqlite_serializable for details on current workarounds. * ``"control_fully"`` - the :class:`_orm.Session` will take control of the given transaction as its own; @@ -1838,7 +1838,7 @@ def begin_nested(self) -> SessionTransaction: :ref:`session_begin_nested` - :ref:`pysqlite_serializable` - special workarounds required + ref_pysqlite_serializable - special workarounds required with the SQLite driver in order for SAVEPOINT to work correctly. @@ -1895,7 +1895,7 @@ def commit(self) -> None: :ref:`unitofwork_transaction` - :ref:`asyncio_orm_avoid_lazyloads` + ref_asyncio_orm_avoid_lazyloads """ trans = self._transaction @@ -2972,7 +2972,7 @@ def refresh( .. seealso:: - :ref:`session_expire` - introductory material + ref_session_expire - introductory material :meth:`.Session.expire` @@ -3051,7 +3051,7 @@ def expire_all(self) -> None: .. seealso:: - :ref:`session_expire` - introductory material + ref_session_expire - introductory material :meth:`.Session.expire` @@ -3093,7 +3093,7 @@ def expire( .. seealso:: - :ref:`session_expire` - introductory material + ref_session_expire - introductory material :meth:`.Session.expire` @@ -3704,7 +3704,7 @@ def merge( This operation cascades to associated instances if the association is mapped with ``cascade="merge"``. - See :ref:`unitofwork_merging` for a detailed discussion of merging. + See ref_unitofwork_merging for a detailed discussion of merging. :param instance: Instance to be merged. :param load: Boolean, when False, :meth:`.merge` switches into @@ -4298,20 +4298,6 @@ def bulk_save_objects( ) -> None: """Perform a bulk save of the given list of objects. - .. legacy:: - - This method is a legacy feature as of the 2.0 series of - SQLAlchemy. For modern bulk INSERT and UPDATE, see - the sections :ref:`orm_queryguide_bulk_insert` and - :ref:`orm_queryguide_bulk_update`. - - For general INSERT and UPDATE of existing ORM mapped objects, - prefer standard :term:`unit of work` data management patterns, - introduced in the :ref:`unified_tutorial` at - :ref:`tutorial_orm_data_manipulation`. SQLAlchemy 2.0 - now uses :ref:`engine_insertmanyvalues` with modern dialects - which solves previous issues of bulk INSERT slowness. - :param objects: a sequence of mapped object instances. The mapped objects are persisted as is, and are **not** associated with the :class:`.Session` afterwards. @@ -4403,15 +4389,6 @@ def bulk_insert_mappings( ) -> None: """Perform a bulk insert of the given list of mapping dictionaries. - .. legacy:: - - This method is a legacy feature as of the 2.0 series of - SQLAlchemy. For modern bulk INSERT and UPDATE, see - the sections :ref:`orm_queryguide_bulk_insert` and - :ref:`orm_queryguide_bulk_update`. The 2.0 API shares - implementation details with this method and adds new features - as well. - :param mapper: a mapped class, or the actual :class:`_orm.Mapper` object, representing the single kind of object represented within the mapping @@ -4482,14 +4459,6 @@ def bulk_update_mappings( ) -> None: """Perform a bulk update of the given list of mapping dictionaries. - .. legacy:: - - This method is a legacy feature as of the 2.0 series of - SQLAlchemy. For modern bulk INSERT and UPDATE, see - the sections :ref:`orm_queryguide_bulk_insert` and - :ref:`orm_queryguide_bulk_update`. The 2.0 API shares - implementation details with this method and adds new features - as well. :param mapper: a mapped class, or the actual :class:`_orm.Mapper` object, diff --git a/lib/sqlalchemy/orm/state.py b/lib/sqlalchemy/orm/state.py index 70893953132..75ecb2bbeee 100644 --- a/lib/sqlalchemy/orm/state.py +++ b/lib/sqlalchemy/orm/state.py @@ -221,7 +221,7 @@ def transient(self) -> bool: .. seealso:: - :ref:`session_object_states` + ref_session_object_states """ return self.key is None and not self._attached @@ -233,7 +233,7 @@ def pending(self) -> bool: .. seealso:: - :ref:`session_object_states` + ref_session_object_states """ return self.key is None and self._attached @@ -263,7 +263,7 @@ def deleted(self) -> bool: .. seealso:: - :ref:`session_object_states` + ref_session_object_states """ return self.key is not None and self._attached and self._deleted @@ -284,7 +284,7 @@ def was_deleted(self) -> bool: :func:`.orm.util.was_deleted` - standalone function - :ref:`session_object_states` + ref_session_object_states """ return self._deleted @@ -299,7 +299,7 @@ def persistent(self) -> bool: .. seealso:: - :ref:`session_object_states` + ref_session_object_states """ return self.key is not None and self._attached and not self._deleted @@ -310,7 +310,7 @@ def detached(self) -> bool: .. seealso:: - :ref:`session_object_states` + ref_session_object_states """ return self.key is not None and not self._attached @@ -377,7 +377,7 @@ def async_session(self) -> Optional[AsyncSession]: .. seealso:: - :ref:`asyncio_toplevel` + ref_asyncio_toplevel """ if _async_provider is None: diff --git a/lib/sqlalchemy/orm/util.py b/lib/sqlalchemy/orm/util.py index d3e36a4945e..62d800745c5 100644 --- a/lib/sqlalchemy/orm/util.py +++ b/lib/sqlalchemy/orm/util.py @@ -679,7 +679,7 @@ class to an entirely new selectable, provided this selectable is column- same attribute and method interface as the original mapped class, allowing :class:`.AliasedClass` to be compatible with any attribute technique which works on the original class, - including hybrid attributes (see :ref:`hybrids_toplevel`). + including hybrid attributes (see ref_hybrids_toplevel). The :class:`.AliasedClass` can be inspected for its underlying :class:`_orm.Mapper`, aliased selectable, and other information @@ -885,7 +885,7 @@ class AliasedInsp( .. seealso:: - :ref:`inspection_toplevel` + ref_inspection_toplevel """ diff --git a/lib/sqlalchemy/orm/writeonly.py b/lib/sqlalchemy/orm/writeonly.py index a362750f602..9f7de744632 100644 --- a/lib/sqlalchemy/orm/writeonly.py +++ b/lib/sqlalchemy/orm/writeonly.py @@ -500,13 +500,13 @@ class WriteOnlyCollection(AbstractCollectionWriter[_T]): The :class:`.WriteOnlyCollection` is used in a mapping by using the ``"write_only"`` lazy loading strategy with :func:`_orm.relationship`. For background on this configuration, - see :ref:`write_only_relationship`. + see ref_write_only_relationship. .. versionadded:: 2.0 .. seealso:: - :ref:`write_only_relationship` + ref_write_only_relationship """ diff --git a/lib/sqlalchemy/pool/impl.py b/lib/sqlalchemy/pool/impl.py index 99779d54fdc..30efd860295 100644 --- a/lib/sqlalchemy/pool/impl.py +++ b/lib/sqlalchemy/pool/impl.py @@ -327,7 +327,7 @@ class SingletonThreadPool(Pool): :class:`.SingletonThreadPool` is used by the SQLite dialect automatically when a memory-based database is used. - See :ref:`sqlite_toplevel`. + See ref_sqlite_toplevel. """ diff --git a/lib/sqlalchemy/sql/_elements_constructors.py b/lib/sqlalchemy/sql/_elements_constructors.py index 99a839cc909..a2f9564bd13 100644 --- a/lib/sqlalchemy/sql/_elements_constructors.py +++ b/lib/sqlalchemy/sql/_elements_constructors.py @@ -667,7 +667,7 @@ def bindparam( :meth:`.ColumnOperators.in_` - :ref:`baked_in` - with baked queries + ref_baked_in - with baked queries .. note:: The "expanding" feature does not support "executemany"- style parameter sets. diff --git a/lib/sqlalchemy/sql/dml.py b/lib/sqlalchemy/sql/dml.py index 91106164069..1a7b7f21d5e 100644 --- a/lib/sqlalchemy/sql/dml.py +++ b/lib/sqlalchemy/sql/dml.py @@ -461,18 +461,6 @@ def return_defaults( of fetching server-side expressions and defaults, for supporting backends only. - .. deepalchemy:: - - The :meth:`.UpdateBase.return_defaults` method is used by the ORM - for its internal work in fetching newly generated primary key - and server default values, in particular to provide the underyling - implementation of the :paramref:`_orm.Mapper.eager_defaults` - ORM feature as well as to allow RETURNING support with bulk - ORM inserts. Its behavior is fairly idiosyncratic - and is not really intended for general use. End users should - stick with using :meth:`.UpdateBase.returning` in order to - add RETURNING clauses to their INSERT, UPDATE and DELETE - statements. Normally, a single row INSERT statement will automatically populate the :attr:`.CursorResult.inserted_primary_key` attribute when executed, @@ -1257,7 +1245,7 @@ def from_select( correspond. :param include_defaults: if True, non-server default values and SQL expressions as specified on :class:`_schema.Column` objects - (as documented in :ref:`metadata_defaults_toplevel`) not + (as documented in ref_metadata_defaults_toplevel) not otherwise specified in the list of names will be rendered into the INSERT and SELECT statements, so that these values are also included in the data to be inserted. diff --git a/lib/sqlalchemy/sql/events.py b/lib/sqlalchemy/sql/events.py index b34d0741209..1cd0aabdf28 100644 --- a/lib/sqlalchemy/sql/events.py +++ b/lib/sqlalchemy/sql/events.py @@ -446,10 +446,10 @@ def receive_column_reflect(inspector, table, column_info): :ref:`mapper_automated_reflection_schemes` - in the ORM mapping documentation - :ref:`automap_intercepting_columns` - - in the :ref:`automap_toplevel` documentation + ref_automap_intercepting_columns - + in the ref_automap_toplevel documentation - :ref:`metadata_reflection_dbagnostic_types` - in - the :ref:`metadata_reflection_toplevel` documentation + ref_metadata_reflection_dbagnostic_types - in + the ref_metadata_reflection_toplevel documentation """ diff --git a/lib/sqlalchemy/sql/functions.py b/lib/sqlalchemy/sql/functions.py index 30ef0b7e342..0b8fd626cd6 100644 --- a/lib/sqlalchemy/sql/functions.py +++ b/lib/sqlalchemy/sql/functions.py @@ -276,7 +276,7 @@ def table_valued(self, *expr, **kw): :ref:`tutorial_functions_table_valued` - in the :ref:`unified_tutorial` - :ref:`postgresql_table_valued` - in the :ref:`postgresql_toplevel` documentation + ref_postgresql_table_valued - in the :ref:`postgresql_toplevel` documentation :meth:`_functions.FunctionElement.scalar_table_valued` - variant of :meth:`_functions.FunctionElement.table_valued` which delivers the @@ -336,7 +336,7 @@ def column_valued(self, name=None, joins_implicitly=False): :ref:`tutorial_functions_column_valued` - in the :ref:`unified_tutorial` - :ref:`postgresql_column_valued` - in the :ref:`postgresql_toplevel` documentation + ref_postgresql_column_valued - in the :ref:`postgresql_toplevel` documentation :meth:`_functions.FunctionElement.table_valued` @@ -840,7 +840,7 @@ class _FunctionGenerator: Functions which are interpreted as "generic" functions know how to calculate their return type automatically. For a listing of known generic - functions, see :ref:`generic_functions`. + functions, see ref_generic_functions. .. note:: diff --git a/lib/sqlalchemy/sql/operators.py b/lib/sqlalchemy/sql/operators.py index 352e5b62df0..fbc0153f626 100644 --- a/lib/sqlalchemy/sql/operators.py +++ b/lib/sqlalchemy/sql/operators.py @@ -1521,7 +1521,7 @@ def match(self, other: Any, **kwargs: Any) -> ColumnOperators: .. versionchanged:: 2.0 ``plainto_tsquery()`` is used instead of ``to_tsquery()`` for PostgreSQL now; for compatibility with - other forms, see :ref:`postgresql_match`. + other forms, see ref_postgresql_match. * MySQL - renders ``MATCH (x) AGAINST (y IN BOOLEAN MODE)`` diff --git a/lib/sqlalchemy/sql/schema.py b/lib/sqlalchemy/sql/schema.py index 19209646912..fa2110947a9 100644 --- a/lib/sqlalchemy/sql/schema.py +++ b/lib/sqlalchemy/sql/schema.py @@ -590,11 +590,11 @@ def __init__( .. seealso:: - :ref:`metadata_reflection_toplevel` + ref_metadata_reflection_toplevel :meth:`_events.DDLEvents.column_reflect` - :ref:`metadata_reflection_dbagnostic_types` + ref_metadata_reflection_dbagnostic_types :param extend_existing: When ``True``, indicates that if this :class:`_schema.Table` is already present in the given @@ -662,7 +662,7 @@ def __init__( In modern SQLAlchemy there is generally no reason to alter this setting, except for some backend specific cases - (see :ref:`mssql_triggers` in the SQL Server dialect documentation + (see ref_mssql_triggers in the SQL Server dialect documentation for one such example). :param include_columns: A list of strings indicating a subset of @@ -1699,7 +1699,7 @@ def __init__( to render the special SQLite keyword ``AUTOINCREMENT`` is not included as this is unnecessary and not recommended by the database vendor. See the section - :ref:`sqlite_autoincrement` for more background. + ref_sqlite_autoincrement for more background. * Oracle - The Oracle dialect has no default "autoincrement" feature available at this time, instead the :class:`.Identity` construct is recommended to achieve this (the :class:`.Sequence` @@ -1755,7 +1755,7 @@ def __init__( .. seealso:: - :ref:`metadata_defaults_toplevel` + ref_metadata_defaults_toplevel :param doc: optional String that can be used by the ORM or similar to document attributes on the Python side. This attribute does @@ -1852,7 +1852,7 @@ def __init__( .. seealso:: - :ref:`metadata_defaults` - complete discussion of onupdate + ref_metadata_defaults - complete discussion of onupdate :param primary_key: If ``True``, marks this column as a primary key column. Multiple columns can have this flag set to specify @@ -1915,7 +1915,7 @@ def __init__( .. seealso:: - :ref:`server_defaults` - complete discussion of server side + ref_server_defaults - complete discussion of server side defaults :param server_onupdate: A :class:`.FetchedValue` instance @@ -1929,12 +1929,12 @@ def __init__( .. warning:: This directive **does not** currently produce MySQL's "ON UPDATE CURRENT_TIMESTAMP()" clause. See - :ref:`mysql_timestamp_onupdate` for background on how to + ref_mysql_timestamp_onupdate for background on how to produce this clause. .. seealso:: - :ref:`triggered_columns` + ref_triggered_columns :param quote: Force quoting of this column's name on or off, corresponding to ``True`` or ``False``. When left at its default @@ -3681,7 +3681,7 @@ class Sequence(HasSchemaAttr, IdentityOptions, DefaultGenerator): .. seealso:: - :ref:`defaults_sequences` + ref_defaults_sequences :class:`.CreateSequence` @@ -3955,7 +3955,7 @@ class FetchedValue(SchemaEventTarget): .. seealso:: - :ref:`triggered_columns` + ref_triggered_columns """ @@ -5140,13 +5140,13 @@ class Index( :ref:`schema_indexes` - General information on :class:`.Index`. - :ref:`postgresql_indexes` - PostgreSQL-specific options available for + ref_postgresql_indexes - PostgreSQL-specific options available for the :class:`.Index` construct. - :ref:`mysql_indexes` - MySQL-specific options available for the + ref_mysql_indexes - MySQL-specific options available for the :class:`.Index` construct. - :ref:`mssql_indexes` - MSSQL-specific options available for the + ref_mssql_indexes - MSSQL-specific options available for the :class:`.Index` construct. """ @@ -5676,13 +5676,13 @@ def reflect( .. seealso:: - :ref:`metadata_reflection_toplevel` + ref_metadata_reflection_toplevel :meth:`_events.DDLEvents.column_reflect` - Event used to customize the reflected columns. Usually used to generalize the types using :meth:`_types.TypeEngine.as_generic` - :ref:`metadata_reflection_dbagnostic_types` - describes how to + ref_metadata_reflection_dbagnostic_types - describes how to reflect tables using general types. """ diff --git a/lib/sqlalchemy/sql/sqltypes.py b/lib/sqlalchemy/sql/sqltypes.py index 4e7514e38da..60ca40a50b1 100644 --- a/lib/sqlalchemy/sql/sqltypes.py +++ b/lib/sqlalchemy/sql/sqltypes.py @@ -1775,7 +1775,7 @@ class PickleType(TypeDecorator[object]): a serialized binary field. To allow ORM change events to propagate for elements associated - with :class:`.PickleType`, see :ref:`mutable_toplevel`. + with :class:`.PickleType`, see ref_mutable_toplevel. """ diff --git a/lib/sqlalchemy/sql/type_api.py b/lib/sqlalchemy/sql/type_api.py index 5af12cb934f..e031c5efa4c 100644 --- a/lib/sqlalchemy/sql/type_api.py +++ b/lib/sqlalchemy/sql/type_api.py @@ -863,7 +863,7 @@ def as_generic(self, allow_nulltype: bool = False) -> TypeEngine[_T]: .. seealso:: - :ref:`metadata_reflection_dbagnostic_types` - describes the + ref_metadata_reflection_dbagnostic_types - describes the use of :meth:`_types.TypeEngine.as_generic` in conjunction with the :meth:`_sql.DDLEvents.column_reflect` event, which is its intended use.