Overview

Cloud Preferences provides a RESTful API and is designed to be work with OpenStack and Rackspace Cloud. It let’s you store and manage data associated with users, devices and more.

Cloud Preferences is built with Python and uses Tornado for the web framework providing asynchronous networking and Motor for non-blocking MongoDB support.

Installation

Install the Cloud Preferences API, Supervisor, MongoDB and Memcached:

apt-get install supervisor mongodb-server nginx python-pip python-dev libmemcached-dev zlib1g-dev git
pip install git+https://github.com/treytabner/cloudprefs.git

Configuration

The examples below are assuming you are deploying to a server with 8 cores. If you have more or less, please make sure to put the appropriate value for numprocs and update the Nginx upstream configuration to match.

Configuration is done at runtime or with the Supervisor configuration file. Create /etc/supervisor/conf.d/cloudprefs.conf with the following content:

[program:cloudprefs]
process_name=cloudprefs:%(process_num)s
command=python -m cloudprefs --port=%(process_num)s --mongodb=127.0.0.1:27017
stdout_logfile=/var/log/cloudprefs.log
stderr_logfile=/var/log/cloudprefs.log
user=nobody
group=nogroup
autostart=true
autorestart=true
startsecs=10
stopwaitsecs=10
directory=/tmp
numprocs=8
numprocs_start=8001

To support a cluster of MongoDB hosts, use something like:

python -m cloudprefs --mongodb=10.0.0.1:27017,10.0.0.2:27017,10.0.0.3:27017

Configure Nginx by creating /etc/nginx/sites-enabled/default with the following configuration:

upstream cloudprefs {
    server 127.0.0.1:8001 max_fails=3 fail_timeout=1s;
    server 127.0.0.1:8002 max_fails=3 fail_timeout=1s;
    server 127.0.0.1:8003 max_fails=3 fail_timeout=1s;
    server 127.0.0.1:8004 max_fails=3 fail_timeout=1s;
    server 127.0.0.1:8005 max_fails=3 fail_timeout=1s;
    server 127.0.0.1:8006 max_fails=3 fail_timeout=1s;
    server 127.0.0.1:8007 max_fails=3 fail_timeout=1s;
    server 127.0.0.1:8008 max_fails=3 fail_timeout=1s;
}

server {
    listen 0.0.0.0:80;
    server_name localhost;
    access_log /var/log/nginx/cloudprefs.log main;
    location / {
        proxy_set_header Host $host;
        proxy_set_header X-Real-Ip $remote_addr;
        proxy_pass http://cloudprefs/;
    }
}

You may also want to adjust worker_connections in /etc/nginx/nginx.conf to something higher.

Starting services

Once you have the configuration in place, add it to Supervisor:

service nginx restart
supervisorctl update
supervisorctl status
tail -f /var/log/cloudprefs.log

Or you can start it up manually:

python -m cloudprefs --help
python -m cloudprefs

API documentation

The Cloud Preferences API uses a concept of organizing data into specific containers. Within each container you can store any data you want, in any structure you want, provided that it’s JSON encodable. Each user can have an almost unlimited number of containers–you are only limited by the number of documents that you may store in a single collection.

Authentication

There are plans to integrate the OpenStack Identity Service but you probably want to handle this through a proxy such as Repose instead.

In addition to whatever headers you send for authentication (such as X-Auth-Token), you will want to supply the X-User-Id header. This is used to associate the data with the proper collection in MongoDB.

Endpoint

Assume your endpoint is running at http://localhost:8888/ unless you have specifically configured something else. Normally you would want to deploy at something other than the root (/) so that you support multiple API versions in the future.

HTTP Actions

All examples below use HTTPie. Install it with pip install httpie or apt-get install httpie.

JSON output is returned only for HTTP GET requests, not HTTP POST or DELETE requests.

Possible HTTP status codes include:

• 200: Successful request with content
• 204: Successful request with no content (from a POST request)
• 400: Bad request, possible invalid JSON or combination of data and key provided
• 401: Forbidden, make sure you are passing the proper headers as described above
• 404: Category or key not found from a GET request
• 409: Problem storing data, conflict with a key detected
• 500: Internal server error, please submit a bug if this happens
• 504: Gateway timeout, the load balancer, proxy or Nginx instance may be having a problem

Storing (POST)

The action for HTTP POST is to create or update. If the container doesn’t already exist, it’s created, otherwise it will be updated.

To create a container, send an HTTP POST request to /<container>. For example, to create an empty “defaults” container:

http POST http://localhost:8888/defaults X-User-Id:username

Note: HTTPie defaults to the GET method unless you pipe in any data, in which case it uses the POST method.

You can also specify a valid JSON document to populate a container. If the container does not already exist, it will be created:

echo '{"build_config": ["monitoring", "backup"]}' | http http://localhost:8888/defaults X-User-Id:username

You can also simplify the request by specifying the key to store data in directly with the URL. If data already exists for the specified key, it will be replaced. For example:

echo '["monitoring", "backup"]' | http http://localhost:8888/defaults/build_config X-User-Id:username

echo '"secretpassword"' | http http://localhost:8888/defaults/passwords/root X-User-Id:username

Retrieving (GET)

To retrieve a list of any containers you may have, send an HTTP GET request to the root of the endpoint itself, for example:

http http://localhost:8888 X-User-Id:username

Response:

[
    "defaults"
]

To retrieve details from a container, send an HTTP GET request to /<container>, for example, to retrieve data from the “defaults” container:

http http://localhost:8888/defaults X-User-Id:username

Response:

{
    "build_config": [
        "monitoring",
        "backup"
    ],
    "passwords": {
        "root": "secretpassword"
    }
}

To retrieve specific details from a container, you can also specify a key. For example:

http http://localhost:8888/defaults/build_config X-User-Id:username

Response:

[
    "monitoring",
    "backup"
]

You can also specify a path to the key:

http http://localhost:8888/defaults/passwords/root X-User-Id:username

Response:

"secretpassword"

Removing (DELETE)

To delete all of your containers and their data, send an HTTP DELETE request to the root of the endpoint itself, for example:

http DELETE http://localhost:8888 X-User-Id:username

To delete a specific container and it’s data, send an HTTP DELETE request to the specific container. To remove the “defaults” container that was created above, do:

http DELETE http://localhost:8888/defaults X-User-Id:username

You can also delete a specific key from a container:

http DELETE http://localhost:8888/defaults/build_options X-User-Id:username

http DELETE http://localhost:8888/defaults/ssh_keys/root X-User-Id:username