Integrating Cloud Foundry with Apache Brooklyn Part 2: Brooklyn Plugin

In Part 1 of this series, we showed how the Brooklyn Service Broker for Cloud Foundry lets you to create and use services for Cloud Foundry using Brooklyn.

We now take a look at how the Brooklyn Plugin for Cloud Foundry lets you create and manage these services directly using cf.

The Brooklyn Plugin allows you to

  • add new Brooklyn blueprints to the service catalog
  • create new services as part of your application’s manifest
  • get information and invoke effectors on running service instances

If you are using CLI version 6.10+ you can install the plugin using Plugin Discovery with the community repository:

$ cf add-plugin-repo community http://plugins.cloudfoundry.org/
$ cf install-plugin Brooklyn -r community

Otherwise, you can get a binary with your web browser from the community site and follow the post-build install instructions below.

You can also compile from the source code. To begin, set up your environment to develop Cloud Foundry plugins as described here then build the plugin:

$ git clone git@github.com:cloudfoundry-community/brooklyn-plugin.git
$ cd brooklyn-plugin
$ go build 

Next install the plugin in Cloud Foundry:

$ cf install-plugin brooklyn-plugin 
Installing plugin ./brooklyn-plugin...
OK
Plugin BrooklynPlugin successfully installed.

$ cf plugins
Listing Installed Plugins...
OK

Plugin Name      Command Name   Command Help   
BrooklynPlugin   brooklyn       Brooklyn plugin command's help text

Good! Now let’s do something cool.

Adding catalog items

Recall that in Part 1 we added Catalog items to Brooklyn through the Brooklyn web console and then instantiated them through Cloud Foundry, but with our plugin we can submit catalog items through the Broker using cf. So, let’s make a Brooklyn catalog blueprint for MongoDB and save it in, say, catalog.yml:

brooklyn.catalog:
    id: my-mongo
    version: 1.0
    iconUrl: classpath://mongodb.png
    description: My MongoDB

name: MongoDB
services:
    - type: brooklyn.entity.basic.BasicApplication
    brooklyn.children:
        - type: brooklyn.entity.nosql.mongodb.MongoDBServer
        id: myMongoServer
        name: mongod

And add it with the add-catalog command:

$ cf brooklyn add-catalog brooklyn broker-user broker-password catalog.yml 
Adding Brooklyn catalog item...
OK
Catalog item sucessfully added.

Good, now when we refresh the Brooklyn Service Broker the service will be available:

$ cf update-service-broker brooklyn broker-user broker-password http://broker-host:8080/
Updating service broker brooklyn as admin...
OK

$ cf service-access
Getting service access as admin...
broker: brooklyn
service                     plan                             access   orgs          
MySQL Database              localhost                        all
MongoDB                     localhost                        none

Nice. All you need to do is enable it as before.

To delete catalog items, simply use delete-catalog:

$ cf brooklyn delete-catalog brooklyn broker-user broker-password MongoDB

Specifying services in the application's manifest

Until now, creating services and binding them, even with the Brooklyn Service Broker, is still way too manual, right? Some services are an integral part of your application; for these, we can automate creating a service and binding by just specifying it in the the manifest.yml file:

applications:
    - name: my-app
    memory: 512M
    hosts: 
    - my-app1
    brooklyn:
        - name: mongodb
        location: localhost
        service: MongoDB

With this manifest, Cloud Foundry with Brooklyn will create a new MongoDB instance and bind it with the instance id mongodb as if it had been created prior and added to the services section of a traditional manifest.

$ cf brooklyn push
Running the brooklyn command
Enter broker: brooklyn
Enter username: broker-user 
Enter password: broker-password

...            
OK

We can do even better; we can automate adding to the broker’s catalog, too, by specifying Brooklyn blueprints directly in the manifest:

applications:
- name: my-app
brooklyn:
- name: mongodb
    location: localhost
    services: 
    - type: brooklyn.entity.nosql.mongodb.MongoDBServer

That’s pretty cool, we no longer need to have the service broker advertising its catalog, just give it a blueprint for what you want, Brooklyn will create it, and the service broker will bind it to your application.

N.B. When you push your app in this way, it can occur that the binding is performed before the service is ready to use. This is a known issue we are working on. In the meantime, if this happens you can unbind the service and push again so that your application can access the updated VCAP_SERVICES variable which contains sensor data that your application may need.

$ cf unbind-service my-app mongodb
$ cf brooklyn push

How do we know when our service is ready to bind? We can look at the sensor output using the plugin.

Viewing sensors

You can find out about your running services by getting sensor data from brooklyn with the following command:

$ cf brooklyn sensors brooklyn broker-user broker-password mongodb
mongodb
-------
Entity:MongoDBServer:kBqU
    host.address : 127.0.0.1
    softwareprocess.pid.file : 
    mongodb.server.opcounters.query : 3
    mongodb.server.replicaSet.isSecondary : false
    host.name : 127.0.0.1
    mongodb.server.network.bytesIn : 9383
    mongodb.server.port : 27018

    ...

    service.process.isRunning : true
    service.state.expected : running @ 1424168919348 / Tue Feb 17 10:28:39 GMT 2015
OK

There are a number of sensors that are unique to the type of service deployed, but, in this case, we are mostly interested in service.process.isRunning. Your application will also have used host.address andmongodb.server.port which will have become available once the service is running.

Effectors

Likewise you can view effectors:

$ cf brooklyn effectors brooklyn broker-user broker-password mongodb
mongodb
-------
Application:e8pchN1J
    e8pchN1J:start      Start the process/service represented by an entity
        parameters: 
        locations         The location or locations to start in, as a string, a location object, a list of strings, or a list of location objects
    e8pchN1J:restart    Restart the process/service represented by an entity
    e8pchN1J:stop       Stop the process/service represented by an entity
    MongoDBServer:kBqU
        kBqUsoNj:restart    Restart the process/service represented by an entity
            parameters: 
            restartChildren   Whether to restart children; default false
            restartMachine    Whether to restart/replace the machine provisioned for this entity:  'true', 'false', or 'auto' are supported, with the default being 'auto' which means to restart or reprovision the machine if there is no simpler way known to restart the entity (for example, if the machine is unhealthy, it would not be possible to restart the process, not even via a stop-then-start sequence); if the machine was not provisioned for this entity, this parameter has no effect
        kBqUsoNj:start      Start the process/service represented by an entity
            parameters: 
            locations         The location or locations to start in, as a string, a location object, a list of strings, or a list of location objects
        kBqUsoNj:stop       Stop the process/service represented by an entity
            parameters: 
            stopMachineMode   When to stop the machine with regard to the entity state. ALWAYS will try to stop the machine even if the entity is already stopped, IF_NOT_STOPPED stops the machine only if the entity is not already stopped, NEVER doesn't stop the machine.
            stopProcessMode   When to stop the process with regard to the entity state
OK

Parameters can be passed simply by prepending “–” to each parameter name followed by the desired parameter value.

Video

To wrap up this post once again here is a video of our plugin in action.

Sneak peak

Running and restarting a single MongoDB instance isn’t that exciting, but the exact same process works for any blueprint, from a Hadoop-Storm-Kafka platform to a set of Couchbase clusters replicated across multiple clouds.

In Part 3 we will demonstrate much more interesting service topologies using the Service Broker and Plugin building blocks you’ve just installed and discuss next steps for this work.