Merge branch 'master' into review_app

This commit is contained in:
Lucie 2018-02-13 12:30:23 +01:00 коммит произвёл GitHub
Родитель 5f556b8b4f a15b5733b3
Коммит fb2b54c1c7
Не найден ключ, соответствующий данной подписи
Идентификатор ключа GPG: 4AEE18F83AFDEB23
20 изменённых файлов: 946 добавлений и 114 удалений

307
README.md
Просмотреть файл

@ -4,30 +4,75 @@
This is the REST API server for the Mozilla Network Pulse project.
- [Current API end points](#current-api-end-points)
All API routes are prefixed with `/api/pulse/`. The "pulse" might seem redundant, but this lets us move the API to different domains and fold it into other API servers without namespace conflicts in the future.
---
# API documentation
- [General Routes](#general-routes)
- [Content-specific routes](#content-specific-routes)
- [Creators](#creators)
- [Entries](#entries)
- [Help Types](#help-types)
- [Issues](#issues)
- [Profiles](#profiles)
- [Tags](#tags)
- [Syndication](#syndication)
---
# Developer information
- [Getting up and running for local development](#getting-up-and-running-for-local-development)
- [Setting up your superuser](#setting-up-your-superuser)
- [Using a localhost rebinding to a "real" domain](#important-using-a-localhost-rebinding-to-a-real-domain)
- [Running the server](#running-the-server)
- [Environment variables](#environment-variables)
- [Deploying to Heroku](#deploying-to-heroku)
- [Debugging all the things](#debugging-all-the-things)
- [Migrating data](#migrating-data)
- [Debugging](#debugging-all-the-things)
- [Resetting your database](#resetting-your-database-because-of-incompatible-model-changes)
- [Migrating data from Google sheets](#migrating-data-from-google-sheets)
## Resetting your database because of incompatible model changes
---
## Current API end points
# General Routes
All API routes are prefixed with `/api/pulse/`. The "pulse" might seem redundant, but this lets us move the API to different domains and fold it into other API servers without namespace conflicts.
## Login routes
### `GET /api/pulse/entries/` with optional `?format=json`
### `GET /api/pulse/login?original_url=<url>`
This retrieves the full list of entries as stored in the database. As a base URL call this returns an HTML page with formatted results, as url with `?format=json` suffix this results a JSON object for use as data input to applications, webpages, etc.
This will kick off a Google OAuth2 login process. This process is entirely based on browser redirects, and once this process completes the user will be redirect to `original_url` with an additional url query argument `loggedin=True` or `loggedin=False` depending on whether the login attemp succeeded or not.
This route takes a swathe of optional arguments for filtering the entry set, visit this route in the browser for more detailed information on all available query arguments.
### `GET /logout`
#### Filters
This will log out a user if they have an authenticated session going. Note that this route does not have a redirect path associated with it: simply calling `/logout` with an XHR or Fetch operation is enough to immediately log the user out and invalidate their session at the API server. The only acknowledgement that callers will receive around this operation succeeding is that if an HTTP 200 status code is received, logout succeeded.
Please run the server and see [http://localhost:8000/entries](http://localhost:8000/entries) for all supported filters.
### `GET /oauth2callback`
This is the route that oauth2 login systems must point to in order to complete an oauth2 login process with in-browser callback URL redirection.
### `GET /api/pulse/userstatus/`
This gets the current user's session information in the form of their full name and email address.
The call response is a JSON object of the following form:
```
{
username: <string: the user's full name according to Google>
profileid: <string: the user's profile id>
customname: <string: the user's custom name as set in their profile>
email: <string: the user's google-login-associated email address>
loggedin: <boolean: whether this user is logged in or not>
moderator: <boolean: whether this logged-in user has moderation rights>
}
```
If a user is authenticated, all three fields will be present. If a user is not authenticated, the response object will only contain the `loggedin` key, with value `false`.
**This data should never be cached persistently**. Do not store this in localStorage, cookies, or any other persistent data store. When the user terminates their client, or logs out, this information should immediately be lost. Also do not store this in a global namespace like `window` or `document`, or in anything that isn't protected by a closure.
## POST protection
### `GET /api/pulse/nonce/`
@ -46,25 +91,9 @@ The call response is a 403 for not authenticated users, or a JSON object when au
Also note that "the page itself" counts as global scope, so you generally don't want to put these values on the page as `<form>` elements. Instead, a form submission should be intercepted, and an in-memory form should be created with all the information of the original form, but with the nonce and csrf values copied. The submission can then use this in-memory form as basis for its POST payload instead.
### `GET /api/pulse/userstatus/`
# Content-specific routes
This gets the current user's session information in the form of their full name and email address.
The call response is a JSON object of the following form:
```
{
username: <string: the user's full name according to Google>
customname: <string: the user's custom name as set in their profile>
email: <string: the user's google-login-associated email address>
loggedin: <boolean: whether this user is logged in or not>
moderator: <boolean: whether this logged-in user has moderation rights>
}
```
If a user is authenticated, all three fields will be present. If a user is not authenticated, the response object will only contain the `loggedin` key, with value `false`.
**This data should never be cached persistently**. Do not store this in localStorage, cookies, or any other persistent data store. When the user terminates their client, or logs out, this information should immediately be lost. Also do not store this in a global namespace like `window` or `document`, or in anything that isn't protected by a closure.
## Creators
### `GET /api/pulse/creators/?name=...`
@ -93,35 +122,22 @@ In this response:
- `next` is a URL if there are more results than fit in a single result set (set to `null` if there are no additional pages of results).
- `results` points to an array of creator records, where each creator has a `name` (string data), a `creator_id` (integer) as well as a `profile_id` (which is either an integer if the creator has an associated profile, or `false` if the creator does not have an associated profile). By default, this array will contain 6 objects, but this number can be increased (to a maximum of 20) by adding `&page_size=...` to the query with the desired results-per-page number.
### `GET /api/pulse/issues/`
## Entries
Gets the list of internet health issues that entries can be related to. This route yields a documentation page unless the request mimetype is set to `application/json`, or the `?format=json` query argument is passed. When requesting JSON, this route yields an object of the form:
### `GET /api/pulse/entries/` with optional `?format=json`
```
[{
name: "issue name",
description: "issue description"
},{
name: ...
description: ...
},
...]
```
This retrieves the full list of entries as stored in the database. As a base URL call this returns an HTML page with formatted results, as url with `?format=json` suffix this results a JSON object for use as data input to applications, webpages, etc.
### `GET /api/pulse/helptypes/`
This route takes a swathe of optional arguments for filtering the entry set, visit this route in the browser for more detailed information on all available query arguments.
Gets the list of help types that are used by entries to indicate how people can get involved. This route yields a documentation page unless the request mimetype is set to `application/json`, or the `?format=json` query argument is passed. When requesting JSON, this route yields an object of the form:
#### Filters
Please run the server and see [http://localhost:8000/entries](http://localhost:8000/entries) for all supported filters.
### `GET /api/pulse/entries/<id=number>/` with optional `?format=json`
This retrieves a single entry with the indicated `id` as stored in the database. As a base URL call this returns an HTML page with formatted results, as url with `?format=json` suffix this results a JSON object for use as data input to applications, webpages, etc.
```
[{
name: "help type name",
description: "help type description"
},{
name: ...
description: ...
},
...]
```
### `POST /api/pulse/entries/`
@ -169,6 +185,40 @@ A successful post will yield a JSON object:
A failed post will yield an HTTP 400 response.
## Entry Moderation
### Moderation state
#### `GET /api/pulse/entries/moderation-states/` with optional `?format=json`
This retrieves the list of moderation states that are used for entry moderation. As a base URL call this returns an HTML page with formatted results, as url with `?format=json` suffix this results a JSON object for use as data input to applications, webpages, etc.
The result is of the format:
```
[
{
id: "id number as string",
name: "human-readable name for this moderation state"
},
{...},
...
]
```
#### `PUT /api/pulse/entries/<id=number>/moderate/<id=number>` with optional `?format=json`
This changes the moderation state for an entry to the passed moderations state. Note that the moderation state is indicated by `id` number, **not** by moderation state name.
### Featured Entries
#### `PUT /api/pulse/entries/<id=number>/feature` with optional `?format=json`
This *toggles* the featured state for an entry if called by a user with moderation rights. An entry that was not featured will become featured, and already featured entries will become unfeatured when this route is called.
## Entry Bookmarking
### `POST /api/pulse/entries/bookmarks/ids=<a comma-separated list of integer ids>`
POSTing to bookmark a list of entries requires sending the following payload object:
@ -197,29 +247,6 @@ A failed post will yield
- an HTTP 400 response if any entry id passed is invalid
- an HTTP 403 response if the current user is not authenticated
### `GET /api/pulse/entries/moderation-states/` with optional `?format=json`
This retrieves the list of moderation states that are used for entry moderation. As a base URL call this returns an HTML page with formatted results, as url with `?format=json` suffix this results a JSON object for use as data input to applications, webpages, etc.
The result is of the format:
```
[
{
id: "id number as string",
name: "human-readable name for this moderation state"
},
{...},
...
]
```
### `GET /api/pulse/entries/<id=number>/` with optional `?format=json`
This retrieves a single entry with the indicated `id` as stored in the database. As a base URL call this returns an HTML page with formatted results, as url with `?format=json` suffix this results a JSON object for use as data input to applications, webpages, etc.
### `PUT /api/pulse/entries/<id=number>/moderate/<id=number>` with optional `?format=json`
This changes the moderation state for an entry to the passed moderations state. Note that the moderation state is indicated by `id` number, **not** by moderation state name.
### `PUT /api/pulse/entries/<id=number>/bookmark`
@ -237,10 +264,62 @@ This operation requires a payload of the following form:
Get the list of all entries that have been bookmarked by the currently authenticated user. Calling this as anonymous user yields an object with property `count` equals to `0`. As a base URL call this returns an HTML page with formatted result, as url with `?format=json` suffix this results a JSON object for use as data input to applications, webpages, etc.
## Help Types
### `GET /api/pulse/helptypes/`
Gets the list of help types that are used by entries to indicate how people can get involved. This route yields a documentation page unless the request mimetype is set to `application/json`, or the `?format=json` query argument is passed. When requesting JSON, this route yields an object of the form:
```
[{
name: "help type name",
description: "help type description"
},{
name: ...
description: ...
},
...]
```
## Issues
### `GET /api/pulse/issues/`
Gets the list of internet health issues that entries can be related to. This route yields a documentation page unless the request mimetype is set to `application/json`, or the `?format=json` query argument is passed. When requesting JSON, this route yields an object of the form:
```
[{
name: "issue name",
description: "issue description"
},{
name: ...
description: ...
},
...]
```
### `GET /api/pulse/issues/<Issue Name>`
Fetches the same data as above, but restricted to an individual issue queried for. Note that this is a URL query, not a URL argument query, so to see the data for an issue named "Security and Privacy" for example, the corresponding URL will be `/api/pulse/issues/Security and Privacy`.
## Profiles
### `GET /api/pulse/profiles/<id=number>/` with optional `?format=json`
This retrieves a single user profile with the indicated `id` as stored in the database. Any profile can be retrieved using this route even without being authenticated. The payload returned by this route also includes an array of entries published (`published_entries`) by the user owning this profile and an array of entries created (`created_entries`) by this profile (as defined by other users when creating entries). As a base URL call this returns an HTML page with formatted results, as url with `?format=json` suffix this results a JSON object for use as data input to applications, webpages, etc.
### `GET /api/pulse/profiles/?...` with a filter arguments, and optional `format=json`
The list of profiles known to the system can be queried, but **only** in conjunction with one or more of three query arguments:
- `profile_type`: filter the list by profile types `board member`, `fellow`, `grantee`, `plain`, or `staff`.
- `program_type`: filter the list by program types `media fellow`, `open web fellow`, `science fellow`, `senior fellow`, or `tech policy fellow`.
- `program_year`: filter the list by program year in the range 2015-2019 (inclusive).
### `GET /api/pulse/myprofile/` with optional `?format=json`
This retrieves the **editable** user profile for the currently authenticated user as stored in the database. An unauthenticated user will receive an HTTP 403 Forbidden response if they try to access this route. As a base URL call this returns an HTML page with formatted results, as url with `?format=json` suffix this results a JSON object for use as data input to applications, webpages, etc.
@ -270,15 +349,68 @@ Also note that this PUT **must** be accompanied by the following header:
X-CSRFToken: required csrf token string obtained from [GET /nonce]
```
### `GET /api/pulse/login?original_url=<url>`
#### Updating extended profile information
This will kick off a Google OAuth2 login process. This process is entirely based on browser redirects, and once this process completes the user will be redirect to `original_url` with an additional url query argument `loggedin=True` or `loggedin=False` depending on whether the login attemp succeeded or not.
If a user's profile has the `enable_extended_information` flag set to `True`, then there are additional fields that can be updated by this user:
### `GET /logout`
```
{
...
program_type: one of the above-listed program types
program_year: one of the above-listed program years
affiliation: the name of the primary organisation associated with the program the user is part of
user_bio_long: a long-form text for the user biography information (4096 character limit)
}
```
This will log out a user if they have an authenticated session going. Note that this route does not have a redirect path associated with it: simply calling `/logout` with an XHR or Fetch operation is enough to immediately log the user out and invalidate their session at the API server. The only acknowledgement that callers will receive around this operation succeeding is that if an HTTP 200 status code is received, logout succeeded.
## Tags
## Getting up and running for local development
### `GET /api/pulse/tags/` with optional `?format=json`
This retrieves the list of all tags known to the system. When requesting JSON, this route yields an object of the form:
```
[
"tag 1",
"tag 2",
"tag 3",
...
]
```
### Filtering
The above route can also be passed a `?search=...` query argument, which will filter the tag list based on `starts-with` logic, such that searching on `?search=abc` will find all tags that start with the partial string `abc`.
# Syndication
There are several syndication routes for RSS/Atom feeds available - these do not use the `/api/pulse` prefix:
## RSS
### `GET /rss/latest`
Replies with an RSS feed consisting of (a subset of) the latest entries that were published to Mozilla Pulse.
### `GET /rss/featured`
Replies with an RSS feed consisting of (a subset of) only those entries that are currently considered featured content.
## Atom
### `GET /atom/latest`
Replies with an Atom feed consisting of (a subset of) the latest entries that were published to Mozilla Pulse.
### `GET /atom/featured`
Replies with an Atom feed consisting of (a subset of) only those entries that are currently considered featured content.
---
# Getting up and running for local development
You'll need `python` (v3) with `pip` (latest) and optionally `virtualenv` (python3 comes with a way to build virtual environments, but you can also install `virtualenv` as a dedicated library if you prefer)
@ -315,11 +447,11 @@ As a Django server, this API server is run like any other Django server:
- `python manage.py runserver`
## Testing the API using the "3rd party library" test file
### Testing the API using the "3rd party library" test file
Fire up a localhost server with port 8080 pointing at the `public` directory (some localhost servers like [http-server](https://npmjs.com/package/http-server) do this automatically for you) and point your browser to [http://localhost:8080](http://localhost:8080). If all went well (but read this README.md to the end, first) you should be able to post to the API server running "on" http://test.example.com:8000
## **Important**: using a localhost rebinding to a "real" domain
### **Important**: using a localhost rebinding to a "real" domain
Google Auth does not like oauth2 to `localhost`, so you will need to set up a host binding such that 127.0.0.1 looks like a real domain. You can do this by editing your `hosts` file (in `/etc/hosts` on most unix-like systems, or `Windows\System32\Drivers\etc\hosts` in Windows). Add the following rule:
@ -327,7 +459,7 @@ Google Auth does not like oauth2 to `localhost`, so you will need to set up a ho
and then use `http://test.example.com:8000` instead of `http://localhost:8000` everywhere. Google Auth should now be perfectly happy.
### Why "test.example.com"?
#### Why "test.example.com"?
Example.com and example.org are "special" domains in that they *cannot* resolve to a real domain as part of the policy we, as the internet-connected world, agreed on. This means that if you forget to set that `hosts` binding, visiting test.example.com will be a guaranteed failure. Any other domain may in fact exist, and you don't want to be hitting a real website when you're doing login and authentication.
@ -343,6 +475,7 @@ The following environment variables are used in this codebase
- `TOKEN_URI`: optional, defaults to 'https://accounts.google.com/o/oauth2/token' and there is no reason to change it.
- `SSL_PROTECTION`: Defaults to `False` to make development easier, but if you're deploying you probably want this to be `True`. This sets a slew of security-related variables in `settings.py` that you can override individually if desired.
Heroku provisions some environmnets on its own, like a `PORT` and `DATABASE_URL` variable, which this codebase will make use of if it sees them, but these values are only really relevant to Heroku deployments and not something you need to mess with for local development purposes.
- `PULSE_FRONTEND_HOSTNAME`: Defaults to `localhost:3000`. Used by the RSS and Atom feed views to create entry URLs that link to the network pulse frontend rather than the JSON API.
## Deploying to Heroku
@ -361,9 +494,9 @@ When working across multiple branches with multiple model changes, it sometimes
Simply run `python reset_database.py` and the steps mentioned above will be run automatically.
**Note:** This does wipe *everything* so you will still need to call `python manage.py createsuperuser` to make sure you have a super user set up again.
**Note:** This does wipe *everything* so you will still need to call `python manage.py createsuperuser` afterwards to make sure you have a super user set up again.
## Migrating data
## Migrating data from Google sheets
To migrate data, export JSON from the Google Sheets db, and save it in the root directory as `migrationData.json`. Then run `python migrate.py`. This generates `massagedData.json`.
In `public/migrate.html`, update the endpoint to be the address of the one you're trying to migrate data into. If it's a local db, leave as is.

Просмотреть файл

@ -33,7 +33,7 @@ def get_default_moderation_state():
"""
states = ModerationState.objects.all()
if (len(states) == 0):
if len(states) == 0:
return -1
default_state = states[0].id
@ -165,6 +165,9 @@ class Entry(models.Model):
objects = EntryQuerySet.as_manager()
def frontend_entry_url(self):
return '{frontend_url}/entry/{pk}'.format(frontend_url=settings.PULSE_FRONTEND_HOSTNAME, pk=self.id)
class Meta:
"""
Make plural not be wrong

Просмотреть файл

@ -1,7 +1,14 @@
from django.contrib import admin
from django.utils.html import format_html
from pulseapi.utility.get_admin_url import get_admin_url
from .models import Location, UserProfile, UserBookmarks
from .models import (
Location,
ProfileType,
ProgramType,
ProgramYear,
UserProfile,
UserBookmarks,
)
class LocationInline(admin.TabularInline):
@ -33,6 +40,12 @@ class UserProfileAdmin(admin.ModelAdmin):
'linkedin',
'github',
'website',
'enable_extended_information',
'profile_type',
'program_type',
'program_year',
'affiliation',
'user_bio_long',
)
readonly_fields = (
@ -72,3 +85,7 @@ class UserBookmarksAdmin(admin.ModelAdmin):
admin.site.register(UserProfile, UserProfileAdmin)
admin.site.register(UserBookmarks, UserBookmarksAdmin)
admin.site.register(ProfileType)
admin.site.register(ProgramType)
admin.site.register(ProgramYear)

Просмотреть файл

@ -0,0 +1,72 @@
# -*- coding: utf-8 -*-
# Generated by Django 1.10.3 on 2018-01-30 17:53
from __future__ import unicode_literals
from django.db import migrations, models
import django.db.models.deletion
class Migration(migrations.Migration):
dependencies = [
('profiles', '0012_remove_userprofile_user'),
]
operations = [
migrations.CreateModel(
name='ProfileType',
fields=[
('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('value', models.CharField(max_length=50)),
],
),
migrations.CreateModel(
name='ProgramType',
fields=[
('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('value', models.CharField(max_length=150)),
],
),
migrations.CreateModel(
name='ProgramYear',
fields=[
('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('value', models.CharField(max_length=25)),
],
),
migrations.AddField(
model_name='userprofile',
name='affiliation',
field=models.CharField(blank=True, max_length=200),
),
migrations.AddField(
model_name='userprofile',
name='enable_extended_information',
field=models.BooleanField(default=False),
),
migrations.AddField(
model_name='userprofile',
name='user_bio_long',
field=models.CharField(blank=True, max_length=4096),
),
migrations.AlterField(
model_name='userprofile',
name='user_bio',
field=models.CharField(blank=True, max_length=212),
),
migrations.AddField(
model_name='userprofile',
name='profile_type',
field=models.ForeignKey(null=True, on_delete=django.db.models.deletion.CASCADE, to='profiles.ProfileType'),
),
migrations.AddField(
model_name='userprofile',
name='program_type',
field=models.ForeignKey(null=True, on_delete=django.db.models.deletion.CASCADE, to='profiles.ProgramType'),
),
migrations.AddField(
model_name='userprofile',
name='program_year',
field=models.ForeignKey(null=True, on_delete=django.db.models.deletion.CASCADE, to='profiles.ProgramYear'),
),
]

Просмотреть файл

@ -0,0 +1,39 @@
# -*- coding: utf-8 -*-
# Generated by Django 1.10.3 on 2018-01-30 01:08
from __future__ import unicode_literals
from django.db import migrations
def setup_default_values(apps, schema_editor):
ProfileType = apps.get_model('profiles', 'ProfileType')
ProfileType.objects.get_or_create(value='plain')
ProfileType.objects.get_or_create(value='staff')
ProfileType.objects.get_or_create(value='fellow')
ProfileType.objects.get_or_create(value='board member')
ProfileType.objects.get_or_create(value='grantee')
ProgramType = apps.get_model('profiles', 'ProgramType')
ProgramType.objects.get_or_create(value='senior fellow')
ProgramType.objects.get_or_create(value='science fellow')
ProgramType.objects.get_or_create(value='open web fellow')
ProgramType.objects.get_or_create(value='tech policy fellow')
ProgramType.objects.get_or_create(value='media fellow')
ProgramYear = apps.get_model('profiles', 'ProgramYear')
ProgramYear.objects.get_or_create(value='2015')
ProgramYear.objects.get_or_create(value='2016')
ProgramYear.objects.get_or_create(value='2017')
ProgramYear.objects.get_or_create(value='2018')
ProgramYear.objects.get_or_create(value='2019')
class Migration(migrations.Migration):
dependencies = [
('profiles', '0013_auto_20180130_0953'),
]
operations = [
migrations.RunPython(setup_default_values),
]

Просмотреть файл

@ -0,0 +1,46 @@
# -*- coding: utf-8 -*-
# Generated by Django 1.10.3 on 2018-02-01 19:52
from __future__ import unicode_literals
from django.db import migrations, models
import django.db.models.deletion
class Migration(migrations.Migration):
dependencies = [
('profiles', '0014_bootstrap_relations'),
]
operations = [
migrations.AlterField(
model_name='profiletype',
name='value',
field=models.CharField(max_length=50, unique=True),
),
migrations.AlterField(
model_name='programtype',
name='value',
field=models.CharField(max_length=150, unique=True),
),
migrations.AlterField(
model_name='programyear',
name='value',
field=models.CharField(max_length=25, unique=True),
),
migrations.AlterField(
model_name='userprofile',
name='profile_type',
field=models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.CASCADE, to='profiles.ProfileType'),
),
migrations.AlterField(
model_name='userprofile',
name='program_type',
field=models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.CASCADE, to='profiles.ProgramType'),
),
migrations.AlterField(
model_name='userprofile',
name='program_year',
field=models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.CASCADE, to='profiles.ProgramYear'),
),
]

Просмотреть файл

@ -0,0 +1,31 @@
# -*- coding: utf-8 -*-
# Generated by Django 1.10.3 on 2018-02-01 20:57
from __future__ import unicode_literals
from django.db import migrations, models
import django.db.models.deletion
class Migration(migrations.Migration):
dependencies = [
('profiles', '0015_auto_20180201_1152'),
]
operations = [
migrations.AlterField(
model_name='userprofile',
name='profile_type',
field=models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.SET_NULL, to='profiles.ProfileType'),
),
migrations.AlterField(
model_name='userprofile',
name='program_type',
field=models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.SET_NULL, to='profiles.ProgramType'),
),
migrations.AlterField(
model_name='userprofile',
name='program_year',
field=models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.SET_NULL, to='profiles.ProgramYear'),
),
]

Просмотреть файл

@ -59,6 +59,66 @@ class Location(models.Model):
)
class ProfileType(models.Model):
"""
See https://github.com/mozilla/network-pulse/issues/657
Values that should exist (handled via migration):
- plain
- staff
- fellow
- board member
- grantee
"""
value = models.CharField(
max_length=50,
unique=True
)
def get_default_profile_type():
(default, _) = ProfileType.objects.get_or_create(value='plain')
return default
def __str__(self):
return self.value
class ProgramType(models.Model):
"""
See https://github.com/mozilla/network-pulse/issues/657
These values are determined by pulse API administrators
(tech policy fellowship, mozfest speaker, etc)
"""
value = models.CharField(
max_length=150,
unique=True
)
def __str__(self):
return self.value
class ProgramYear(models.Model):
"""
See https://github.com/mozilla/network-pulse/issues/657
You'd think this would be 4 characters, but a "year" is
not a calendar year, so the year could just as easily
be "summer 2017" or "Q2 2016 - Q1 2018", so this is the
same kind of simple value model that the profile and
program types use.
"""
value = models.CharField(
max_length=25,
unique=True
)
def __str__(self):
return self.value
class UserProfile(models.Model):
"""
This class houses all user profile information,
@ -74,12 +134,6 @@ class UserProfile(models.Model):
default=False
)
# A tweet-style user bio
user_bio = models.CharField(
max_length=140,
blank=True
)
# "user X bookmarked entry Y" is a many to many relation,
# for which we also want to know *when* a user bookmarked
# a specific entry. As such, we use a helper class that
@ -119,10 +173,11 @@ class UserProfile(models.Model):
# We provide an easy accessor to the profile's user because
# accessing the reverse relation (using related_name) can throw
# a RelatedObjectDoesNotExist exception for orphan profiles. This
# allows us to return None instead.
# We however cannot use this accessor as a lookup field in querysets
# because it is not an actual field.
# a RelatedObjectDoesNotExist exception for orphan profiles.
# This allows us to return None instead.
#
# Note: we cannot use this accessor as a lookup field in querysets
# because it is not an actual field.
@property
def user(self):
# We do not import EmailUser directly so that we don't end up with
@ -207,6 +262,57 @@ class UserProfile(models.Model):
blank=True
)
# --- extended information ---
enable_extended_information = models.BooleanField(
default=False
)
profile_type = models.ForeignKey(
'profiles.ProfileType',
null=True,
blank=True,
on_delete=models.SET_NULL
# default is handled in save()
)
program_type = models.ForeignKey(
'profiles.ProgramType',
null=True,
blank=True,
on_delete=models.SET_NULL
)
program_year = models.ForeignKey(
'profiles.ProgramYear',
null=True,
blank=True,
on_delete=models.SET_NULL
)
# Free form affiliation information
affiliation = models.CharField(
max_length=200,
blank=True
)
# A tweet-style user bio
user_bio = models.CharField(
max_length=212,
blank=True
)
# A long-form user bio
user_bio_long = models.CharField(
max_length=4096,
blank=True
)
def save(self, *args, **kwargs):
if self.profile_type is None:
self.profile_type = ProfileType.get_default_profile_type()
super(UserProfile, self).save(*args, **kwargs)
def __str__(self):
if self.user is None:
return 'orphan profile'

Просмотреть файл

@ -9,6 +9,15 @@ from pulseapi.creators.models import OrderedCreatorRecord
from pulseapi.entries.serializers import EntrySerializer
# Helper function to remove a value from a dictionary
# by key, removing the key itself as well.
def remove_key(data, key):
try:
del data[key]
except:
pass
class UserBookmarksSerializer(serializers.ModelSerializer):
"""
Serializes a {user,entry,when} bookmark.
@ -24,12 +33,30 @@ class UserBookmarksSerializer(serializers.ModelSerializer):
class UserProfileSerializer(serializers.ModelSerializer):
"""
Serializes a user profile.
Note that the following fields should only show up when
the 'enable_extended_information' flag is set to True:
- user_bio_long
- program_type
- program_year
- affiliation
"""
user_bio = serializers.CharField(
max_length=140,
required=False,
allow_blank=True,
)
def __init__(self, instance=None, *args, **kwargs):
super().__init__(instance, *args, **kwargs)
if instance is not None and type(instance) is UserProfile:
# We type-check to prevent this from kicking in for entire
# QuerySet objects, rather than just UserProfile objects.
if instance.enable_extended_information is False:
self.fields.pop('user_bio_long')
self.fields.pop('program_type')
self.fields.pop('program_year')
self.fields.pop('affiliation')
# Whether this flag is set or not, it should not
# end up in the actual serialized profile data.
self.fields.pop('enable_extended_information')
custom_name = serializers.CharField(
max_length=70,
required=False,
@ -66,6 +93,25 @@ class UserProfileSerializer(serializers.ModelSerializer):
allow_blank=True,
)
user_bio = serializers.CharField(
max_length=140,
required=False,
allow_blank=True,
)
profile_type = serializers.StringRelatedField()
program_type = serializers.StringRelatedField()
program_year = serializers.StringRelatedField()
def update(self, instance, validated_data):
if instance.enable_extended_information is False:
remove_key(validated_data, 'user_bio_long')
remove_key(validated_data, 'program_type')
remove_key(validated_data, 'program_year')
remove_key(validated_data, 'affiliation')
remove_key(validated_data, 'enable_extended_information')
return super(UserProfileSerializer, self).update(instance, validated_data)
class Meta:
"""
Meta class. Because

Просмотреть файл

@ -2,6 +2,8 @@ import json
from django.core.urlresolvers import reverse
from .models import UserProfile, ProfileType, ProgramType, ProgramYear
from pulseapi.tests import PulseMemberTestCase
from pulseapi.entries.serializers import EntrySerializer
from pulseapi.creators.models import OrderedCreatorRecord
@ -30,5 +32,138 @@ class TestProfileView(PulseMemberTestCase):
)
created_entries = [EntrySerializer(x.entry).data for x in entry_creators]
self.assertEqual(entriesjson['created_entries'], created_entries)
# make sure extended profile data does not show
self.assertEqual('program_type' in entriesjson, False)
def test_extended_profile_data(self):
(profile, created) = UserProfile.objects.get_or_create(related_user=self.user)
profile.enable_extended_information = True
test_program = ProgramType.objects.all().first()
profile.program_type = test_program
profile.save()
profile_url = reverse('profile', kwargs={'pk': profile.id})
# extended profile data should show in API responses
response = self.client.get(profile_url)
entriesjson = json.loads(str(response.content, 'utf-8'))
self.assertEqual('program_type' in entriesjson, True)
self.assertEqual(entriesjson['program_type'], test_program.value)
def test_updating_extended_profile_data(self):
(profile, created) = UserProfile.objects.get_or_create(related_user=self.user)
profile.enable_extended_information = True
profile.program_type = ProgramType.objects.all().first()
profile.save()
profile_url = reverse('myprofile')
# authentication is absolutely required
self.client.logout()
response = self.client.put(profile_url, json.dumps({'affiliation': 'Mozilla'}))
self.assertEqual(response.status_code, 403)
# with authentication, updates should work
self.client.force_login(user=self.user)
response = self.client.put(profile_url, json.dumps({'affiliation': 'Mozilla'}))
profile.refresh_from_db()
self.assertEqual(profile.affiliation, 'Mozilla')
response = self.client.get(profile_url)
entriesjson = json.loads(str(response.content, 'utf-8'))
self.assertEqual('affiliation' in entriesjson, True)
self.assertEqual(entriesjson['affiliation'], 'Mozilla')
def test_updating_disabled_extended_profile_data(self):
(profile, created) = UserProfile.objects.get_or_create(related_user=self.user)
profile.enable_extended_information = False
profile.affiliation = 'untouched'
profile.save()
profile_url = reverse('myprofile')
# With authentication, "updates" should work, but
# enable_extened_information=False should prevent
# an update from occurring.
self.client.put(profile_url, {'affiliation': 'Mozilla'})
profile.refresh_from_db()
self.assertEqual(profile.affiliation, 'untouched')
def test_profile_type_uniqueness(self):
# as found in the bootstrap migration:
(profile, created) = ProfileType.objects.get_or_create(value='plain')
self.assertEqual(created, False)
def test_program_type_uniqueness(self):
# as found in the bootstrap migration:
(profile, created) = ProgramType.objects.get_or_create(value='senior fellow')
self.assertEqual(created, False)
def test_program_year_uniqueness(self):
# as found in the bootstrap migration:
(profile, created) = ProgramYear.objects.get_or_create(value='2018')
self.assertEqual(created, False)
def test_profile_listing(self):
profile_types = ['a', 'b', 'c']
program_types = ['a', 'b']
program_years = ['a', 'b']
for v1 in profile_types:
(profile_type, _) = ProfileType.objects.get_or_create(value=v1)
for v2 in program_types:
(program_type, _) = ProgramType.objects.get_or_create(value=v2)
for v3 in program_years:
(program_year, _) = ProgramYear.objects.get_or_create(value=v3)
profile = UserProfile.objects.create()
profile.enable_extended_information = True
profile.profile_type = profile_type
profile.program_type = program_type
profile.program_year = program_year
profile.save()
profile_url = reverse('profile_list')
# There should be four results for each profile type
for profile_type in profile_types:
url = ('{url}?profile_type={type}').format(url=profile_url, type=profile_type)
response = self.client.get(url)
entriesjson = json.loads(str(response.content, 'utf-8'))
self.assertEqual(len(entriesjson), 4)
# There should be six results for each program type
for program_type in program_types:
url = ('{url}?program_type={type}').format(url=profile_url, type=program_type)
response = self.client.get(url)
entriesjson = json.loads(str(response.content, 'utf-8'))
self.assertEqual(len(entriesjson), 6)
# There should be six results for each program year
for program_year in program_years:
url = ('{url}?program_year={type}').format(url=profile_url, type=program_year)
response = self.client.get(url)
entriesjson = json.loads(str(response.content, 'utf-8'))
self.assertEqual(len(entriesjson), 6)
# There should only be one result for each unique combination
for v1 in profile_types:
for v2 in program_types:
for v3 in program_years:
url = ('{url}?profile_type={v1}&program_type={v2}&program_year={v3}').format(
url=profile_url,
v1=profile_type,
v2=program_type,
v3=program_year
)
response = self.client.get(url)
entriesjson = json.loads(str(response.content, 'utf-8'))
self.assertEqual(len(entriesjson), 1)
def test_invalid_profile_listing_argument(self):
profile_url = reverse('profile_list')
url = ('{url}?unsupported_arg=should_be_empty_response').format(url=profile_url)
response = self.client.get(url)
entriesjson = json.loads(str(response.content, 'utf-8'))
self.assertEqual(len(entriesjson), 0)

Просмотреть файл

@ -1,6 +1,8 @@
from django.conf.urls import url
from pulseapi.profiles.views import (
# UserProfileAPIView, # see note below.
UserProfileListAPIView,
UserProfilePublicAPIView,
UserProfilePublicSelfAPIView,
)
@ -15,5 +17,13 @@ urlpatterns = [
r'^me/',
UserProfilePublicSelfAPIView.as_view(),
name='profile_self',
)
),
# note that there is also a /myprofile route
# defined in the root urls.py which connects
# to the UserProfileAPIView class.
url(
r'^$',
UserProfileListAPIView.as_view(),
name='profile_list',
),
]

Просмотреть файл

@ -1,10 +1,11 @@
import base64
import django_filters
from django.core.files.base import ContentFile
from django.shortcuts import get_object_or_404
from rest_framework import permissions
from rest_framework.decorators import detail_route
from rest_framework.generics import RetrieveAPIView, RetrieveUpdateAPIView
from rest_framework import filters, permissions
from rest_framework.generics import RetrieveAPIView, RetrieveUpdateAPIView, ListAPIView
from pulseapi.profiles.models import UserProfile
from pulseapi.profiles.serializers import (
@ -34,13 +35,13 @@ class UserProfileAPIView(RetrieveUpdateAPIView):
permissions.IsAuthenticated,
IsProfileOwner
)
serializer_class = UserProfileSerializer
def get_object(self):
user = self.request.user
return get_object_or_404(UserProfile, related_user=user)
@detail_route(methods=['put'])
def put(self, request, *args, **kwargs):
'''
If there is a thumbnail, and it was sent as part of an
@ -52,15 +53,90 @@ class UserProfileAPIView(RetrieveUpdateAPIView):
much mutually exclusive patterns. A try/pass make far more sense.
'''
payload = request.data
try:
thumbnail = request.data['thumbnail']
thumbnail = payload['thumbnail']
# do we actually need to repack as ContentFile?
if thumbnail['name'] and thumbnail['base64']:
name = thumbnail['name']
encdata = thumbnail['base64']
proxy = ContentFile(base64.b64decode(encdata), name=name)
request.data['thumbnail'] = proxy
payload['thumbnail'] = proxy
except:
pass
return super(UserProfileAPIView, self).put(request, *args, **kwargs)
# NOTE: DRF has deprecated the FilterSet class in favor of
# django_filters.rest_framework.FilterSet in v3.7.x, which
# we aren't far from upgrading to.
# SEE: https://github.com/mozilla/network-pulse-api/issues/288
class ProfileCustomFilter(filters.FilterSet):
"""
We add custom filtering to allow you to filter by:
* Profile type - pass the `?profile_type=` query parameter
* Program type - pass the `?program_type=` query parameter
* Program year - pass the `?program_year=` query parameter
"""
profile_type = django_filters.CharFilter(
name='profile_type__value',
lookup_expr='iexact',
)
program_type = django_filters.CharFilter(
name='program_type__value',
lookup_expr='iexact',
)
program_year = django_filters.CharFilter(
name='program_year__value',
lookup_expr='iexact',
)
@property
def qs(self):
"""
Ensure that if the filter route is called without
a legal filtering argument, we return an empty
queryset, rather than every profile in existence.
"""
empty_set = UserProfile.objects.none()
request = self.request
if request is None:
return empty_set
queries = self.request.GET
if queries is None:
return empty_set
fields = ProfileCustomFilter.get_fields()
for key in fields:
if key in queries:
return super(ProfileCustomFilter, self).qs
return empty_set
class Meta:
"""
Required Meta class
"""
model = UserProfile
fields = [
'profile_type',
'program_type',
'program_year',
]
class UserProfileListAPIView(ListAPIView):
serializer_class = UserProfilePublicSerializer
filter_backends = (
filters.DjangoFilterBackend,
)
filter_class = ProfileCustomFilter
queryset = UserProfile.objects.all()

Просмотреть файл

@ -27,6 +27,7 @@ env = environ.Env(
SSL_PROTECTION=(bool, False),
CORS_REGEX_WHITELIST=(tuple, ()),
HEROKU_APP_NAME=(str, ''),
PULSE_FRONTEND_HOSTNAME=(str, ''),
)
SSL_PROTECTION = env('SSL_PROTECTION')
@ -214,6 +215,8 @@ if SSL_PROTECTION is True:
X_FRAME_OPTIONS = "DENY"
# Frontend URL is required for the RSS and Atom feeds
PULSE_FRONTEND_HOSTNAME = env('PULSE_FRONTEND_HOSTNAME')
USE_S3 = env('USE_S3')

Просмотреть файл

@ -102,6 +102,17 @@ class JSONDefaultClient(Client):
**extra
)
def put(self, path, data=None, content_type=CONTENT_TYPE_JSON,
follow=False, secure=False, **extra):
return super(JSONDefaultClient, self).put(
path,
data=data,
content_type=content_type,
follow=follow,
secure=secure,
**extra
)
def create_logged_in_user(test, name, email, password="password1234"):
test.name = name

Просмотреть файл

@ -19,6 +19,12 @@ from django.contrib import admin
from django.conf.urls.static import static
from pulseapi.profiles.views import UserProfileAPIView
from pulseapi.utility.syndication import (
RSSFeedLatestFromPulse,
AtomFeedLatestFromPulse,
RSSFeedFeaturedFromPulse,
AtomFeedFeaturedFromPulse
)
urlpatterns = [
# admin patterns
@ -42,7 +48,13 @@ urlpatterns = [
r'^api/pulse/myprofile/',
UserProfileAPIView.as_view(),
name='myprofile'
)
),
# Syndication
url(r'^rss/latest', RSSFeedLatestFromPulse()),
url(r'^rss/featured', RSSFeedFeaturedFromPulse()),
url(r'^atom/latest', AtomFeedLatestFromPulse()),
url(r'^atom/featured', AtomFeedFeaturedFromPulse()),
]
if settings.USE_S3 is not True:

Просмотреть файл

@ -37,7 +37,7 @@ class TestUserViews(PulseMemberTestCase):
def test_user_status_true(self):
"""
Assert that an authenticated user calling /userstatus is 200
with response { loggedin: true, username: "...", email: "..." }
with response { loggedin: true, username: "...", email: "...", profileid: "..." }
"""
response = self.client.get('/api/pulse/userstatus/')
self.assertEqual(response.status_code, 200)
@ -46,4 +46,6 @@ class TestUserViews(PulseMemberTestCase):
json_obj = json.loads(string)
self.assertEqual(json_obj['loggedin'], True)
self.assertEqual(json_obj['email'], 'test@example.org')
self.assertTrue(json_obj['profileid'])
self.assertEqual(type(json_obj['profileid']), str)
self.assertEqual(json_obj['username'], 'plain user')

Просмотреть файл

@ -87,6 +87,7 @@ def userstatus(request):
cached by applications, for obvious reasons.
"""
username = False
profileid = False
customname = False
email = False
@ -103,11 +104,13 @@ def userstatus(request):
if loggedin:
username = user.name
profileid = user.profile.id
customname = user.profile.custom_name
email = user.email
return render(request, 'users/userstatus.json', {
'username': username,
'profileid': profileid,
'customname': customname,
'email': email,
'loggedin': loggedin,

Просмотреть файл

@ -0,0 +1,85 @@
"""
Provide RSS and Atom feeds for Pulse.
"""
from django.contrib.syndication.views import Feed
from django.utils.feedgenerator import Atom1Feed
from pulseapi import settings
from pulseapi.entries.models import Entry
# The creator(s) name can be found in the `OrderedCreatorRecord` class from the `creators` models.
def get_entry_creators(entry):
# Since `creators` is an optional field and can be empty, we return the publisher name instead.
if entry.related_creators.count() >= 1:
return ', '.join(
creator_record.creator.creator_name
for creator_record
in entry.related_creators.all()
)
else:
return entry.published_by.name
# Generic class for RSS feeds
class RSSFeedFromPulse(Feed):
def item_author_name(self, entry):
return get_entry_creators(entry)
def item_title(self, entry):
return entry.title
def item_pubdate(self, entry):
return entry.created
def item_description(self, entry):
return entry.description
def item_enclosure_url(self, entry):
if entry.thumbnail:
if settings.USE_S3:
return entry.thumbnail.url
# Provide an absolute URL for local dev purposes
else:
return settings.ALLOWED_HOSTS[0] + ':8000' + entry.thumbnail.url
def item_enclosure_length(self, entry):
return ''
def item_enclosure_mime_type(self, entry):
return 'image/jpeg'
def item_link(self, entry):
return entry.frontend_entry_url()
# RSS feed for latest entries
class RSSFeedLatestFromPulse(RSSFeedFromPulse):
title = 'Latest from Mozilla Pulse'
link = '{frontend_url}/latest'.format(frontend_url=settings.PULSE_FRONTEND_HOSTNAME)
description = 'Subscribe to get the latest entries from Mozilla Pulse.'
def items(self):
return Entry.objects.order_by('-created')
# RSS feed for featured entries
class RSSFeedFeaturedFromPulse(RSSFeedFromPulse):
title = 'Latest from Mozilla Pulse'
link = '{frontend_url}/featured'.format(frontend_url=settings.PULSE_FRONTEND_HOSTNAME)
description = 'Subscribe to get the latest featured entries from Mozilla Pulse.'
def items(self):
return Entry.objects.filter(featured=True).order_by('-created')
# Atom feed for latest entries
class AtomFeedLatestFromPulse(RSSFeedLatestFromPulse):
feed_type = Atom1Feed
subtitle = RSSFeedLatestFromPulse.description
# Atom feed for featured entries
class AtomFeedFeaturedFromPulse(RSSFeedFeaturedFromPulse):
feed_type = Atom1Feed
subtitle = RSSFeedLatestFromPulse.description

Просмотреть файл

@ -1,3 +1,4 @@
DEBUG=True
REDIRECT_URIS=http://test.example.com:8000/api/pulse/oauth2callback
SSL_PROTECTION=False
PULSE_FRONTEND_HOSTNAME=localhost:3000

Просмотреть файл

@ -1,5 +1,6 @@
{
{% if loggedin %} "username": "{{ username }}",
{% endif %}{% if profileid %} "profileid": "{{ profileid }}",
{% endif %}{% if customname %} "customname": "{{ customname }}",
{% endif %}{% if loggedin %} "email": "{{ email }}",
{% endif %} "loggedin": {% if loggedin %}true{% else %}false