Integrating Cloud Foundry with Apache Brooklyn Part 3: Managing Services

In Part 2, we demonstrated how to create services with the Brooklyn Plugin for Cloud Foundry and briefly introduced some of the tools you can use to manage your services.

We are now going to create a more complicated service involving MongoDB and sharding then show you how you can use effectors to scale this service out or in.

MongoDB offers some pretty advanced options for scaling, both horizontally and vertically. So, we’re going to look at how to create a Sharded deployment (like that described in the figure above) with the Brooklyn plugin.

Create a sharded deployment service

First, taking advantage of our new ability to specify blueprints in the application manifest, under the brooklyn section, create a MongoDBShardedDeployment service:

applications:
- name: my-app
    memory: 512M
    brooklyn:
    - name: Sharded MongoDB
        location: localhost
        services:
        - type: brooklyn.entity.nosql.mongodb.sharding.MongoDBShardedDeployment
            id: mongo
            name: Mongo DB Backend
            provisioning.properties:
                minRam: 16gb
            brooklyn.config:
                initialRouterClusterSize: 1
                initialShardClusterSize: 5
                shardReplicaSetSize: 3

Notice with the blueprint, you can also specify provisioning properties and configure the sharded deployment. We have specified a minimum of 16GB of RAM with one Router, five Shards and three nodes per shard in this blueprint. Simply push this blueprint:

$ cf brooklyn push
Running the brooklyn command
Enter broker: brooklyn
Enter username: user
Enter password: simple-password

...
OK

and you have a sharded deployment of MongoDB.

Examine the sensors

As we saw in Part 2, you can view the sensors associated with a running service with the sensors command, so let’s do that with this service:

$ cf brooklyn sensors brooklyn user simple-password "Sharded MongoDB"
Sharded MongoDB
---------------
Entity:Mongo DB Backend
    service.state : RUNNING
    ...
        Cluster of MongoDBConfigServer (LocalhostMachineProvisioningLocation{id=Bg7x8Kpi, name=localhost})
            service.state : RUNNING
            mongodb.config.server.addresses : [127.0.0.1:27019 127.0.0.1:27020 127.0.0.1:27021]
            ...
                MongoDBConfigServer:buuD
                    service.state : RUNNING
                    host.address : 127.0.0.1
                    mongodb.server.port : 27019
                    ...
                MongoDBConfigServer:v3Uq
                    service.state : RUNNING
                    host.address : 127.0.0.1
                    mongodb.server.port : 27020
                    ...
                MongoDBConfigServer:rHuG
                    host.address : 127.0.0.1
                    service.state : RUNNING
                    mongodb.server.port : 27021
                    ...
                quarantine
                    group.members : []
                    group.members.count : 0
                    ...
        Cluster of MongoDBRouter (LocalhostMachineProvisioningLocation{id=Bg7x8Kpi, name=localhost})
            service.state : RUNNING
            ...
                MongoDBRouter:JScw
                    service.state : RUNNING
                    mongodb.router.config.shard.count : 5
                    host.address : 127.0.0.1
                    mongodb.server.port : 27018
                    ...
                quarantine
                    group.members : []
                    group.members.count : 0
                    ...
        Cluster of MongoDBReplicaSet (LocalhostMachineProvisioningLocation{id=Bg7x8Kpi, name=localhost})
            group.members.count : 5
            service.state : RUNNING
            ...
                Cluster of MongoDBServer (LocalhostMachineProvisioningLocation{id=Bg7x8Kpi, name=localhost})
                    service.state : RUNNING
                    mongodb.server.network.bytesIn : 318844
                    group.members.count : 3
                    ...
                        MongoDBServer:GApX
                            mongodb.server.network.bytesIn : 145177
                            host.address : 127.0.0.1
                            mongodb.server.network.bytesOut : 883410
                            mongodb.server.port : 27027
                            service.state : RUNNING
                            ...
                        MongoDBServer:Ep4N
                            mongodb.server.network.bytesOut : 761446
                            mongodb.server.network.bytesIn : 87014
                            mongodb.server.port : 27026
                            host.address : 127.0.0.1
                            service.state : RUNNING
                            ...
                        MongoDBServer:EqDn
                            mongodb.server.port : 27035
                            service.state : RUNNING
                            mongodb.server.network.bytesIn : 86653
                            mongodb.server.network.bytesOut : 763923
                            host.address : 127.0.0.1
                            ...
                        quarantine
                            group.members : []
                            group.members.count : 0
                            ...
                        ...
                        quarantine
                            group.members : []
                            group.members.count : 0
                            ...

OK

In this example, there are lots of entities and many more sensors, but a few are worth pointing out:service.state shows whether the entity is running; host.address and mongodb.server.port show how to connect to the individual MongoDB servers, should you need to; and mongodb.server.network.bytesInand mongodb.server.network.bytesOut show how much traffic is coming in and out.

Scale it back using effectors

So, looking at the sensors, suppose we decide that we would like to scale back the number of shards to one. To find out which effector to use for that we use the effectors command:

$ cf brooklyn effectors brooklyn user simple-password "Sharded MongoDB"
Sharded MongoDB
---------------
Application:m6obgkNz
    m6obgkNz:start Start the process/service represented by an entity
        parameters:
        locations The location or locations to start in, as a string, a location object, a list of strings, or a list of location objects
    m6obgkNz:restart Restart the process/service represented by an entity
    m6obgkNz:stop Stop the process/service represented by an entity
    Mongo DB Backend
        htgPyEms:restart Restart the process/service represented by an entity
        htgPyEms:start Start the process/service represented by an entity
        parameters:
        locations The location or locations to start in, as a string, a location object, a list of strings, or a list of location objects
        htgPyEms:stop Stop the process/service represented by an entity
        Cluster of MongoDBReplicaSet (LocalhostMachineProvisioningLocation{id=Bg7x8Kpi, name=localhost})
            cfgFazAk:restart Restart the process/service represented by an entity
            cfgFazAk:start Start the process/service represented by an entity
                parameters:
                locations The location or locations to start in, as a string, a location object, a list of strings, or a list of location objects
            cfgFazAk:stop Stop the process/service represented by an entity
            cfgFazAk:resizeByDelta Changes the size of the cluster.
                parameters:
                delta The change in number of nodes
            cfgFazAk:resize Changes the size of the entity (e.g. the number of nodes in a cluster)
                parameters:
                desiredSize The new size of the cluster
            cfgFazAk:replaceMember Replaces the entity with the given ID, if it is a member; first adds a new member, then removes this one. Returns id of the new entity; or throws exception if couldn't be replaced.
                parameters:
                memberId The entity id of a member to be replaced
...
OK

The one we need is cfgFazAk:resize so let’s invoke that:

$ cf brooklyn invoke brooklyn user simple-password "Sharded MongoDB" cfgFazAk:resize --desiredSize 1
Invoking effector cfgFazAk:resize
1
OK

Scale it back out

And if we need to scale back out again we simple run the same command again but with the parameter changed:

$ cf brooklyn invoke brooklyn user simple-password "Sharded MongoDB" cfgFazAk:resize --desiredSize 3
Invoking effector cfgFazAk:resize
3
OK

What about increased replication? Just look at the effectors to find the cluster you want to increase and run the same command

$ cf brooklyn invoke brooklyn user simple-password "Sharded MongoDB" RaArKduP:resize --desiredSize 5
Invoking effector RaArKduP:resize
5
OK

Video

To wrap up this post once again here is a video of our plugin in action.

Next time

In part 1 and part 2, we saw the basics of using the Cloud Foundry Service Broker and Plugin to manage Brooklyn services, from creating services in the catalog to adding new items to the catalog; from specifying catalog items in the manifest to specifying new types of services in the catalog. And now we’ve seen a bigger service that we can control using sensors and effectors.

But Brooklyn has more, and in the next Part we are going to show you how to deploy an application to Cloud Foundry with Multi-Cloud, Polyglot persistence provided by Apache Brooklyn. See you then.