ShardingSphere 5.3.2 Release: Apollo Integration, Better Data Migration, and Stronger Encryption

Apache ShardingSphere version 5.3.2, released after two months of development, brings significant improvements in functionality, performance, testing, documentation, and examples. With 797 PRs from teams and individuals worldwide, users can enjoy an enhanced and problem-solving version. Notably, the ShardingSphere Driver now supports Apollo, allowing for configuration reading from the Apollo configuration center. Other updates include support for migrating tables with any index structure and the CONCAT function for data encryption fuzzy query.

The JDBC URL is written as follows:

# Read the configuration file from the absolute path
jdbc:shardingsphere:absolutepath:/path/to/config.yml


# Read the configuration file from the classpath
jdbc:shardingsphere:classpath:config/shardingsphere/config.yml


# Read the configuration file from the Apollo configuration center
jdbc:shardingsphere:apollo:apollo.meta?appid=ss-test&namespace=config

The provision of an SPI interface allows developers to tailor the configuration reading logic to their specific scenarios, such as enabling configuration reading from other registration centers. The ShardingSphere community encourages developers to contribute to the shared configuration implementation to extend its benefits to a broader user base.

Migrate Tables with Any Index Structure, Including Tables Without an Index

ShardingSphere's latest update now allows for the migration of tables with any index structure, including those without an index, providing significant benefits to users with complex application scenarios. The updated data migration support considers scenarios where tables have composite primary keys or unique indexes, or no primary keys or indexes at all. This improvement gives users greater flexibility in managing their data, making it easier to ensure smooth application operations.

Relevant Links

🔗 Download Link

🔗 Release Notes

🔗 Project Address

🔗 ShardingSphere-on-Cloud

The Data Encryption Fuzzy Query Now Supports the CONCAT Function

After receiving user feedback on the encrypted column fuzzy query feature in ShardingSphere 5.3.0, it was noted that the LIKE fuzzy query was often used in combination with the wildcard % through the CONCAT function in SQL. However, ShardingSphere was limited to completing wildcard concatenation on parameter values only. In version 5.3.2, ShardingSphere has made improvements to support functions in encryption and now allows for the use of wildcards directly in SQL in the following format:

select * from t where user_name like CONCAT('%', ?, '%')

Release Notes

API Changes

• Proxy: Added the property system-log-level to support dynamic changes to the log level.
• DistSQL: Removed the Hint-related DistSQL; users can now use SQL Hint instead.

New Features

• Scaling: Pipeline job now supports tables with any index structure.

Enhancements

• Scaling: Uses JDBC stream query for inventory dump and data consistency check.
• Scaling: Pipeline jobs are now compatible with tables that have MySQL binary-type fields as primary keys.
• Scaling: The migration and other jobs are now implemented in a blocking manner, making it easier for ejob to obtain the real lifecycle of the jobs and support failover.
• Scaling: Improved large table records count estimation.
• Scaling: Support proxy sharding rule absent for migration jobs, including migration to a single table or pure encryption.
• Scaling: Added useServerPrepStmts=false to avoid exceeding the limit of precompiled statements on the MySQL server.
• Scaling: MySQL binlog now returns accurate data types when parsing datetime/time instead of strings.
• Scaling: Added global status for the pipeline preparation stage.
• Scaling: Added netTimeoutForStreamingResults to prevent MySQL streaming data consumption from being too slow and causing the connection to be closed by the server.
• Authority: Supports specifying password authentication method.
• Authority: Added md5 authentication support for openGauss protocol.
• Agent: Added more monitoring metrics for JDBC.
• Kernel: ShardingSphere Driver configuration is now based on SPI and supports Apollo.
• Kernel: Adjusted SKIP_ENCRYPT_REWRITE SQL Hint to SKIP_SQL_REWRITEto support more scenarios.
• Kernel: Support openGauss EXPLAIN PERFORMANCE syntax.
• Encrypt: LIKE now supports CONCAT function.

Bug Fixes

• Scaling: Fixed the issue where TestDecodingPlugin parsed Unicode characters incorrectly.
• Scaling: PostgreSQL migration now supports fields related to JSON.
• DistSQL: CREATE SHARDING TABLE RULE now supports NONE strategy.
• Kernel: Fixed the issue of an error when using the system database with an empty data source.
• Kernel: Fixed the issue where setting worker-id does not take effect with Standalone mode.
• Kernel: Cleared storage node information when deleting read/write-splitting and database discovery rules.
• Kernel: Fixed the abnormal problem of Column index out of range in single table complex query.
• Kernel: Fixed the issue where PostgreSQL LIKE queries (using lowercase) failed.
• Kernel: Fixed the exception of built-in metabase data collection when the front and back database types were inconsistent.
• Kernel: Fixed the problem of routing error reporting under certain table names.
• Kernel: Fixed the issue where parsing errors occurred when creating MySQL stored procedures.
• Kernel: Fixed NPE caused by incorrect extraction of table names in union statements.
• Kernel: Fixed the issue where uppercase table name constraints were not rewritten.
• Kernel: Fixed the issue where PostgreSQL/OpenGauss SQL containing money type caused parsing errors.
• Kernel: Fixed the issue where using placeholders and type conversions simultaneously in PostgreSQL/OpenGauss caused SQL parsing errors.
• Kernel: Fixed the issue where syntax errors occurred when parsing constant type conversions in PostgreSQL/OpenGauss.
• Kernel: Fixed Chinese character encoding exception when executing select with SQL Federation engine.
• Kernel: Fixed IndexOutOfBoundsException when executing set variable statement in JDBC adapter.
• Kernel: Fixed index does not exist exception when executing drop index statement.
• Proxy: Properly handled number signs in MySQL Proxy binary protocol.
• Proxy: Fixed the issue where PostgreSQL/openGauss Proxy failed to handle bytea data type.
• Proxy: Fixed the issue where PostgreSQL Proxy failed to parse time value with microseconds.
• Proxy: Fixed the exception when the client passes quoted charset to PostgreSQL/openGauss Proxy.

In summary, the latest release of Apache ShardingSphere offers many enhancements and functionalities that can improve problem-solving for users. The community has been responsive to user feedback and has improved the overall user experience. This release incorporates the work of 37 contributors who made 797 merged PRs. We thank them for their dedication.

Relevant Links

🔗 Download Link

🔗 Release Notes

🔗 Project Address

🔗 ShardingSphere-on-Cloud