Grails Shards Plugin

  • Tags: database, datasource, sharding
  • Latest: 1.0
  • Last Updated: 03 April 2013
  • Grails version: 2.0.0 > *
0 vote
Dependency: 
compile ":sharding:1.0"

 Documentation  Source

Summary

Supports sharding of data

Description

The Sharding plugin allows application programmers to define multiple database shards to horizontally scale their user's data across multiple database schema and servers. The programmer can easily change the database the set of Domain Classes are targeting with a simple service call and objects used to segment the data are assigned a shard based on all shards usage and capacity. This is not available to grails programs out of the box as currently there is only one DataSource and SessionFactory. The original blog post covering the plugin and a sample application can be found here.

Dependancies

Shards DSL

The Sharding plugin follows the convention over configuration philosiphy of grails but in some cases needs the programmer to provide some configuration. This configuration is contained in grails-app/conf/Shards.groovy. Here is a description of the DSL used to configure the sharding plugin.

  • index - This section contains data about the Index Database
    • domainClass - The name of the domain class used to represent the object used to segment the data.
    • shardNameFieldName - The name of the domain class field that will hold the shard the object is assigned to.
    • name - The name used to represent the SessionFactory and DataSource for this Index Database
    • user - The username used to connect to the Index database
    • password - The password used to connect to the Index database
    • driverClass - The name of the driver class used to talk to the Index Database
    • jdbcUrl - The jdbc connection string used to connect to the Index database
    • dialect - The hibernate dialect used to talk to the Index database
  • shards - This section has a set of entries for each shard in the system
    • shard_XX - Container for a shard, XX should be a two digit number that increments and is unique
      • name - The string used to represent this shard.
      • user - The username used to connect to the shard database
      • password - The password used to connect to the shard database
      • driverClass - The name of the driver class used to talk to the shard Database
      • jdbcUrl - The jdbc connection string used to connect to the shard database
      • capacity - The number of objecs that can be assigned to the shard database before it is considered full, please note this is only used to calculate a percentage full. This does not stop objects being assigned if the percentage exceeds 100%.
Here is an example Shards.groovy

index = {
  domainClass('UserIndex')
  shardNameFieldName('shard')
  name('shardINDEX')
  user('root')
  password('PASSWORD')
  driverClass('com.mysql.jdbc.Driver')
  jdbcUrl('jdbc:mysql://localhost:3306/shardINDEX')
  dialect(org.hibernate.dialect.MySQL5InnoDBDialect)
}
shards = {
  shard_01 {
    name('shard1001')
    user('root')
    password('PASSWORD')
    driverClass('com.mysql.jdbc.Driver')
    capacity(1000)
    jdbcUrl('jdbc:mysql://localhost:3306/shard1001')
  }
  shard_02 {
    name('shard1002')
    user('root')
    password('PASSWORD')
    driverClass('com.mysql.jdbc.Driver')
    capacity(1000)
    jdbcUrl('jdbc:mysql://localhost:3306/shard1002')
  }
}

Shard Domain Class

The plugin creates a Domain Class called Shard which contains a record for every shard in the system. Included in this Domain Class is the capacity of the shard and the number of objects assigned to the shard. This class is queried every time a new object (typically a user) is created within the system to assign a shard based on the usage to capacity ratio. This allows for shards to be located on different types of hardware that should take either a more or less objects.

Usage

To install the plugin in your application just run the 'install-plugin' Grails script, e.g.

grails install-plugin sharding

I have created a very basic test application that has the Index database and two Shard databases. This example applcation and instructions for using can be found here.

Authors

Jeff Rick jeffrick@gmail.com

Robert Woollam

Please report any issues to the Grails User mailing list and/or write up an issue in JIRA at http://jira.codehaus.org/browse/GRAILSPLUGINS under the Grails-Sharding component.

History

  • July, 7 2010
    • released initial version 0.2