Modernize the SQLAlchemy dialect #8
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ility - Add comprehensive compat.py module for version detection and compatibility utilities - Enhance base.py with statement caching, server-side cursors, and connection pool improvements - Update dtypes.py with cache key support for VECTOR and JSON types - Create comprehensive test suite with 6 test modules covering all implemented features - Add proper skip decorators for tests requiring specific SQLAlchemy versions or real connections - Remove hardcoded environment variables - tests now rely on user-set PYTHONPATH and SINGLESTOREDB_URL - All tests pass or are properly skipped with explanatory messages 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Create comprehensive pytest fixture system in conftest.py - Convert test_integration_features.py from unittest to pytest style - Add pytest-compatible error handling tests - Fix JSON type constructor to properly call parent __init__ - Add database isolation with random test database names - Implement automatic table cleanup after tests - Add test documentation with usage examples - Update test dependencies and fix pre-commit issues 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Replace nested conn.begin() calls with conn.commit() to fix "already initialized a Transaction" errors. SQLAlchemy 2.x automatically starts transactions on connection, so explicit begin() was causing conflicts. Fixed tests: - test_statement_caching_behavior - test_json_type_functionality - test_execution_options_streaming 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Update get_connection_url to handle HTTP/HTTPS URLs with proper singlestoredb+ prefix and improve error handling with ValueError instead of pytest.skip for missing environment variable. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Add proper pytest configuration in conftest.py - Fix import issues in test files - Improve test database setup and teardown 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Add Docker container support to pytest fixtures - Tests automatically start SingleStoreDB container if SINGLESTOREDB_URL not set - Container health check ensures database is ready before tests run - Convert test_basics.py from unittest to pytest fixtures for Docker compatibility - Update test-requirements.txt to include docker dependency - Maintain backward compatibility - existing SINGLESTOREDB_URL still works - Session-scoped container lifecycle for optimal performance 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Change test database fixture from function to session scope - Use single shared database for all tests in a session instead of per-test databases - Add table_name_prefix fixture to generate unique table names per test - Update all tests to use random table prefixes for isolation - Modify clean_tables fixture to only remove tables created by each test - Convert test_data_loaded to use randomized table names from test.sql Performance improvement: ~95% reduction in test setup overhead - 12 tests now run in ~25s vs ~240s+ with individual databases - Maintains full test isolation while sharing database connection - Optimizes Docker container usage with minimal database operations 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Add explicit transaction blocks around DDL operations in test_integration_features.py - Fix clean_tables fixture to use single transaction for SELECT and DROP operations - Update CI workflow to test both SQLAlchemy 1.4 and 2.0 compatibility - All tests now pass with SQLALCHEMY_WARN_20=1 without deprecation warnings 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Update test imports to use singlestoredb driver instead of PyMySQL, ensuring tests can run properly with the actual SingleStore driver. Improve Docker container integration for automated testing. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
…n tests Updates test_modulus_with_percent_signs to use the recommended exec_driver_sql() method instead of the deprecated text() constructor for direct SQL execution with parameters. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Ensures proper database connection URL handling when using Docker containers by always using the standard connection URL for database creation operations. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Remove unittest.TestCase inheritance and setUp method - Replace unittest assertions with pytest assertions - Update method signatures to use pytest fixtures (dialect, test_connection) - Convert @unittest.skip to @pytest.mark.skip decorators - Remove self.subTest() context managers (not needed in pytest) - Fix the 3 failing tests that required base_connection_url parameter - Update __main__ section to use pytest instead of unittest runner This resolves the TypeError issues where unittest methods expected pytest fixture parameters that couldn't be injected. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Fix conftest.py base_connection_url fixture to properly strip URL schemes - Update test_basics.py with improved formatting from linter - Clean up README_pytest_fixtures.md formatting These changes improve the reliability of test database connections and ensure consistent fixture behavior across different environments. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Detect if connection URL specifies a database and use it directly - Skip database creation/deletion when using existing database - Add proper regex parsing of database name from connection URL - Only create/cleanup test databases when necessary 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Allows specifying a different URL for database initialization, useful when test server setup requires different credentials or connection parameters than runtime tests. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Add SINGLESTOREDB_INIT_DB_URL support for separate initialization credentials - Extract URL validation logic into reusable helper functions - Improve URL parsing and database name extraction - Better separation of concerns between connection setup and database creation 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Add engine.dispose() calls to properly close connections - Add debug logging for engine URL in test_engine fixture - Ensure clean connection management between test setup and teardown
…tion - Created singlestoredb-docker.py utility to start/stop SingleStore containers - Updated Makefile to automatically start container if SINGLESTOREDB_URL not set - Uses dynamic port allocation to avoid conflicts - Container is automatically cleaned up after doc generation 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Add _escape_column_name() utility function to properly escape column names containing special characters (spaces, hyphens, backticks) with backticks in all DDL compilation functions. Changes: - Added _escape_column_name() utility function in ddlelement.py - Updated compile_shard_key() to escape column names - Updated compile_sort_key() to escape column names - Updated compile_vector_key() to escape column names - Updated compile_multi_value_index() to escape column names - Updated compile_fulltext_index() to escape column names - Added comprehensive tests for special column names in all DDL test files Fixes DDL generation errors when using column names with spaces, hyphens, or other special characters that require backtick escaping in SQL. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
BREAKING CHANGE: Updated all DDL element classes to use variable positional
arguments (*columns) instead of list-based parameters. This modernizes the API
while maintaining functionality but intentionally breaks backward compatibility.
Key Changes:
- ShardKey: Changed from ShardKey(['col1', 'col2']) to ShardKey('col1', 'col2')
- SortKey: Changed from SortKey(['col1']) to SortKey('col1')
- VectorKey: Changed from VectorKey(['col1']) to VectorKey('col1')
- FullTextIndex: Added ASC/DESC support and modernized API
- MultiValueIndex: Changed from MultiValueIndex('col') to MultiValueIndex('col1', 'col2')
All keyword arguments are now keyword-only for improved API clarity.
Updated 9 files including all examples, tests, and documentation.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <[email protected]>
…g strategies Enhanced ShardKey to support column-level ASC/DESC directions, matching SortKey functionality and enabling more sophisticated sharding strategies for performance optimization. Key changes: - ShardKey now accepts Union[str, Tuple[str, str]] column specifications - Added ShardKey.asc() and ShardKey.desc() static helper methods for cleaner API - Updated SQL compilation to properly handle ASC/DESC directions in SHARD KEY clauses - Changed internal storage from Tuple[str, ...] to List[Tuple[str, str]] for direction tracking - Enhanced repr() to show explicit DESC directions while keeping ASC implicit for readability Comprehensive test coverage includes: - Direction-based API functionality with mixed ASC/DESC columns - Static helper method usage patterns - SQL compilation with proper direction handling - Error validation for invalid directions - Backward compatibility preservation - Integration with table constructor and reflection All 212 tests pass with full compatibility maintained. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Remove deprecated MetaData(bind=engine) usage in test files to eliminate SQLAlchemy 2.0 warnings. Updated test setup in: - test_shard_key.py - test_sort_key.py - test_vector_key.py All tests continue to pass with clean output. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Enhanced SingleStore table options to accept Python booleans for improved API usability: - Add boolean support for AUTOSTATS_ENABLED (True/False -> TRUE/FALSE) - Add boolean support for AUTOSTATS_SAMPLING (True/False -> ON/OFF) - Add boolean False support for AUTOSTATS_CARDINALITY_MODE and AUTOSTATS_HISTOGRAM_MODE (False -> OFF) - Maintain full backward compatibility with existing string-based values - Add comprehensive validation with descriptive error messages for invalid values - Add extensive test coverage for all boolean conversion scenarios This enhancement makes the API more Pythonic by allowing native Python boolean values instead of requiring string literals for boolean-like table options, while maintaining backward compatibility. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Implement comprehensive ColumnGroup support for SingleStoreDB tables: - Add ColumnGroup DDL element class with optional keyword-only name parameter - Implement SQL compilation generating COLUMN GROUP (*) syntax - Integrate with Table constructor via singlestoredb_column_group parameter - Support both named and auto-generated column groups - Add complete test coverage with 14 test cases - Include column name escaping for special characters - Export ColumnGroup in module public API Column groups create materialized row copies as separate indexes for columnstore tables, improving full-row retrieval/update performance on wide tables while using less RAM than rowstore equivalents. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
…and ColumnStore Introduces a complete table type system for SingleStore with support for: - TableType base class with mutually exclusive modifier validation - RowStore class for in-memory OLTP tables with ROWSTORE prefix - ColumnStore class for disk-based analytical tables (default, no prefix) - Table modifiers: reference, temporary, and global_temporary - Integration with existing DDL compiler and SingleStore features Key features: - RowStore always emits ROWSTORE prefix for clarity - ColumnStore uses no prefix (SingleStore default behavior) - Comprehensive validation prevents conflicting modifiers - Proper prefix handling preserves existing table prefixes - Full integration with SHARD KEY, SORT KEY, and other DDL elements - 20 comprehensive test cases covering all scenarios and edge cases Changes: - Add TableType, RowStore, ColumnStore to ddlelement.py - Update SingleStoreDBDDLCompiler.visit_create_table for table type handling - Export new classes in __init__.py maintaining alphabetical order - Add test_table_types.py with comprehensive validation and compilation tests 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
…ex_options - Add Dict[str, Any] support for index_options parameter in VectorKey and MultiValueIndex - Automatically serialize dictionary index_options to JSON strings - Maintain backward compatibility with string-based index_options - Add comprehensive docstring examples for both string and dict usage - Add proper type annotations and validation - Update imports to include json module and Dict type 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
…ures - Add FullTextIndex reflection tests in test_advanced_indexes.py - Add ColumnGroup reflection tests in test_column_group.py - Add TableType (RowStore/ColumnStore) reflection tests in test_table_types.py - Add Table Options (AUTOSTATS, COMPRESSION) reflection tests in test_table_options.py - Fix transaction error in temporary table reflection test - Fix warnings for SingleStore table options in reflection.py 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
…table type support - Add comprehensive declarative reflection tests covering ShardKey, SortKey, and VectorKey - Update reflection parser to properly handle ASC/DESC directions in CREATE TABLE statements - Enhance base.py with get_table_options method for converting parsed features to dialect options - Fix parser handling of new (column_name, direction) tuple format for key specifications - Update test expectations in test_shard_key.py and test_sort_key.py for new tuple format - Add support for table type reflection (RowStore, ColumnStore) with modifiers - Improve CREATE TABLE parsing with metadata_only, named keys, and direction parsing - Ensure full round-trip compatibility between table creation and reflection 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Change table options separator from space to comma in DDL compiler - Add support for COLUMN GROUP (*) syntax in reflection parser - Enhance key/index parsing to handle ALL COLUMNS and quoted/unquoted names - Update tests to use COLUMN GROUP (*) instead of KEY ALL COLUMNS - Improve error handling in advanced index tests - Add docker to dev dependencies for testing 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.