Simple server that records submitted data to S3
Перейти к файлу
Kyle Lahnakoski 37b62d876c update from machine 2020-08-11 07:13:02 -04:00
modatasubmission better logging 2016-06-07 18:32:17 -04:00
pyLibrary lib update 2016-06-07 18:11:40 -04:00
resources update from machine 2020-08-11 07:13:02 -04:00
tests better logging 2016-06-07 18:32:17 -04:00
.gitignore fix ignore file to include more 2016-01-09 08:06:24 -05:00
LICENSE Initial commit 2016-01-08 11:29:11 -05:00
README.md lib update 2016-06-07 18:11:40 -04:00
requirements.txt admin 2016-01-08 22:48:41 -05:00

README.md

MoDataSubmission

A service that accepts HTTP POST requests of JSON, stores them in S3, and returns a link to its location. Requests are authenticated using Hawk.

Requirements

  • Python27, including Pip
  • Server with write access to Amazon S3

Installation

There is an installation script in resources/scripts/install.sh which can be used as a guide for installing the server.

Setup

The server requires a configuration file. An example configuration file is at tests/resources/config/server.json

	{
		"flask": {
			"host": "0.0.0.0",
			"port": 5000,
			"debug": true,
			"threaded": true,
			"processes": 1
		},
		"aws": {
	        "aws_access_key_id": "sometring",
	        "aws_secret_access_key" : "somebase64",
	        "region": "us-west-2"
	    },
		"users": [
			{
				"hawk": {
					"id": "kyle@example.com",
					"key": "secret",
					"algorithm": "sha256"
				},
				"resources": [
					"testing"
				]
			}
		],
		"debug": {
			"trace": true,
			"log": [
				{
					"log_type": "console"
				}
			]
		}
	}

###Setup Properties

  • flask - properties delivered to the Flask.run() method. Be sure you have debug=false in production!
  • aws - your AWS permissions. These should NOT be in this configuration file, but rather in a separate key file. See the server.jsonfile for an example
  • users - An array of users and permissions. The hawk credentials are used to identify the POST requests. And resources is a list of bucket names that user is allowed to POST to.
  • debug - Logging configuration, which lacks good documentation.

Running

The run script details the steps to run the server. Do not use the sudo -i step if you are not listening on port 80.

Testing

There is a simple client in the test code to save some JSON in S3. The only important line is the one that sends the data; here is that same line, expanded with parameters matching the server configuration above:

	link, id = Client(
		url = "http://example.com:80/testing",
		hawk_credentials = {
			"id": "kyle@example.com",
			"key": "secret",
			"algorithm": "sha256"
		}
	).send(data)