Grails Shards Plugin
Dependency:
compile "org.grails.plugins:sharding:1.0"
Summary
Supports sharding of data
Description
The Sharding plugin allows application programmers to define multiple database shards to horizontally scale their user's data across multiple database schema and servers. The programmer can easily change the database the set of Domain Classes are targeting with a simple service call and objects used to segment the data are assigned a shard based on all shards usage and capacity. This is not available to grails programs out of the box as currently there is only one DataSource and SessionFactory. The original blog post covering the plugin and a sample application can be found here.I have created a very basic test application that has the Index database and two Shard databases. This example applcation and instructions for using can be found here.
Dependancies
- grails-datasources plugin
- hibernate plugin
Shards DSL
The Sharding plugin follows the convention over configuration philosiphy of grails but in some cases needs the programmer to provide some configuration. This configuration is contained in grails-app/conf/Shards.groovy. Here is a description of the DSL used to configure the sharding plugin.- index - This section contains data about the Index Database
- domainClass - The name of the domain class used to represent the object used to segment the data.
- shardNameFieldName - The name of the domain class field that will hold the shard the object is assigned to.
- name - The name used to represent the SessionFactory and DataSource for this Index Database
- user - The username used to connect to the Index database
- password - The password used to connect to the Index database
- driverClass - The name of the driver class used to talk to the Index Database
- jdbcUrl - The jdbc connection string used to connect to the Index database
- dialect - The hibernate dialect used to talk to the Index database
- shards - This section has a set of entries for each shard in the system
- shard_XX - Container for a shard, XX should be a two digit number that increments and is unique
- name - The string used to represent this shard.
- user - The username used to connect to the shard database
- password - The password used to connect to the shard database
- driverClass - The name of the driver class used to talk to the shard Database
- jdbcUrl - The jdbc connection string used to connect to the shard database
- capacity - The number of objecs that can be assigned to the shard database before it is considered full, please note this is only used to calculate a percentage full. This does not stop objects being assigned if the percentage exceeds 100%.
index = { domainClass('UserIndex') shardNameFieldName('shard') name('shardINDEX') user('root') password('PASSWORD') driverClass('com.mysql.jdbc.Driver') jdbcUrl('jdbc:mysql://localhost:3306/shardINDEX') dialect(org.hibernate.dialect.MySQL5InnoDBDialect) } shards = { shard_01 { name('shard1001') user('root') password('PASSWORD') driverClass('com.mysql.jdbc.Driver') capacity(1000) jdbcUrl('jdbc:mysql://localhost:3306/shard1001') } shard_02 { name('shard1002') user('root') password('PASSWORD') driverClass('com.mysql.jdbc.Driver') capacity(1000) jdbcUrl('jdbc:mysql://localhost:3306/shard1002') } }
Shard Domain Class
The plugin creates a Domain Class called Shard which contains a record for every shard in the system. Included in this Domain Class is the capacity of the shard and the number of objects assigned to the shard. This class is queried every time a new object (typically a user) is created within the system to assign a shard based on the usage to capacity ratio. This allows for shards to be located on different types of hardware that should take either a more or less objects.Usage
To install the plugin in your application just run the 'install-plugin' Grails script, e.g.grails install-plugin sharding
Authors
Jeff Rick [email protected]Robert WoollamPlease report any issues to the Grails User mailing list and/or write up an issue in JIRA at http://jira.codehaus.org/browse/GRAILSPLUGINS under the Grails-Sharding component.History
- July, 7 2010
- released initial version 0.2