зеркало из https://github.com/getsops/sops.git
Julien Vehent
Коммит
a5e0899de8
138
README.rst
138
README.rst
|
|
@ -1,2 +1,140 @@
|
|||
SOPS: Secrets OPerationS
|
||||
========================
|
||||
`sops` is a cli that encrypt values of yaml, json or text files using AWS KMS.
|
||||
|
||||
Usage
|
||||
-----
|
||||
|
||||
Editing
|
||||
~~~~~~~
|
||||
|
||||
`sops` encrypted file contain the necessary KMS information to decrypt their
|
||||
content. All a user of `sops` need is valid AWS credentials and the necessary
|
||||
permissions on KMS keys.
|
||||
|
||||
Given that, the only command a `sops` user need is:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sops <file>
|
||||
|
||||
`<file>` will be opened, decrypted, passed to a text editor (vim by default),
|
||||
encrypted if modified, and save back to its original location. All of these
|
||||
steps, apart from the actual editing, are transparent to the user.
|
||||
|
||||
Creating
|
||||
~~~~~~~~
|
||||
|
||||
In order to create a file, the KMS ARN must be provided to `sops`, either on the
|
||||
command line in the `-k` flag, or in the environment variable **SOPS_KMS_ARN**.
|
||||
|
||||
`sops` automatically create a file if the given path doesn't exist (it will not
|
||||
create folders, however).
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sops newfile.yaml -k arn:aws:kms:us-east-1:656532927350:key/920aff2e-c5f1-4040-943a-047fa387b27e
|
||||
newfile.yaml doesn't exist, creating it.
|
||||
new data key generated from kms: CiC6yCOtzsnFhkfdIs...
|
||||
file written to newfile.yaml
|
||||
|
||||
Input some cleartext yaml:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
myapp1: t00m4nys3cr3tz
|
||||
app2:
|
||||
db:
|
||||
user: bob
|
||||
password: c4r1b0u
|
||||
# private key for secret operations in app2
|
||||
key: |
|
||||
-----BEGIN RSA PRIVATE KEY-----
|
||||
MIIBPAIBAAJBAPTMNIyHuZtpLYc7VsHQtwOkWYobkUblmHWRmbXzlAX6K8tMf3Wf
|
||||
Erb0xAEyVV7e8J0CIQC8VBY8f8yg+Y7Kxbw4zDYGyb3KkXL10YorpeuZR4LuQQIg
|
||||
bKGPkMM4w5blyE1tqGN0T7sJwEx+EUOgacRNqM2ljVA=
|
||||
-----END RSA PRIVATE KEY-----
|
||||
an_array:
|
||||
- secretuser1 # a super secret user
|
||||
- secretuser2
|
||||
sops:
|
||||
kms:
|
||||
enc: CiC6yCOtzsnFhkfdIslYZ0bAf//gYLYCmIu87B3sy/5yYxKnAQEBAQB4usg......
|
||||
enc_ts: 1439587921.752637
|
||||
arn: arn:aws:kms:us-east-1:656532927350:key/920aff2e-c5f1-4040-943a-047fa387b27e
|
||||
|
||||
After saving the file and exiting, it is automatically encrypted. Keys are
|
||||
still in cleartext, but value are now unreadable.
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
myapp1: ENC[AES256_GCM,data=s4mlbkPqyk+GFDluAHY=,iv=7c9X8CwZyK5PsRRmUpzxL4CeQmp7+ry6mVemJtmpR7U=,aad=CFVNHUiz8xupOCMNYUlF4l+TcCjGaxayiknL9tQtolw=,tag=5ecBRedoXPJJ3uBjaj7J1w==]
|
||||
app2:
|
||||
db:
|
||||
user: ENC[AES256_GCM,data=CmkT,iv=xnUTxXU4g5lKEqetiZrM2s+m20idUUt9xGU6XitsIic=,aad=KidFJD6ioPXKqz+BYVYXtHk8Dd6e1yvhPx6kO5BOJTs=,tag=7WDZbBf2oqMuXi3YH4m2Ig==]
|
||||
password: ENC[AES256_GCM,data=zw2yh6Oz8Q==,iv=Apme9l8h+OwdwgbozsuXa1mVK+b821eoQNEBBSF6Ihs=,aad=SZFoaQDlNe0SkRaX65zB7E8SDyhkr9uVBI+3GWUBKsQ=,tag=We5dwW455S1M4ob1HzAu7Q==]
|
||||
# private key for secret operations in app2
|
||||
key: |-
|
||||
ENC[AES256_GCM,data=feo0o1qW8p4Nw2tN9/QAt0zoeGZHgolORWXH+7hk4Oc5nQcA/Ve3mYQ9TKSZAtzsYr+OEnEVUAg/RzvXy40F9dXsv2ugux+DLS1SIWddKRAdeL283vjnsDtydc3+AP+UuEuCyIVHqT8uKcqQnenzzu/yx07scIwcMQ6Vs2RnQ3WwrOrkBsbPQ4PpuPsrlck7EbcKkMnoIe09AMN3/J3A+mlmOGBxAio1ahFXpeBzzYeoRkjffojvigT2ULZy92Kx1afRSnWXNmUtMKqbJDIIvYulWHW5efAnulk/nHZ0Bhy+wxV0jqXAp7mKiIlGuydxRZ6DPon7jhABWV5d93EZdZJ+/33sUiOyQKIEukqae17C2Hrt0QoGg7OhG/O0oyTKiql0Nj6KC3bFbkdM7sSFbsIbv/of0P5Kb2zr3VYAJriJqUWMKzj3i7M6z9+wxrTVxuMQ4Lvzw3aHDjNOgJobkfjxxMYUvF5l2OWRFrdxtY9WxBYAcDzkJnBYtkPnUzlEc/8ieypefqOBlcphOvzl+EjM1I0N4OGG5ij5nNsHQ/MSoM3FJpjROQKclhz8ZN5CH41LUemP3AdddPpoUuwzHCxR8NskUhyHBlep0iZL9xGFLL7SwYEACKxk2BCwHMWeNmXfKo6co+wjCmn+un3FANE=,iv=NworRcR7VnLgW30c4W9OmVgBaY7tA1fd090JQpBM5ho=,aad=sbwFbTuEr9FbPd/ofR7BL9NORUpfmNd+X3Q+tJqmj8g=,tag=wc7RWWBArQrTMt3AAbSwZQ==]
|
||||
an_array:
|
||||
- ENC[AES256_GCM,data=L3Y0Bzn2M6yERcU=,iv=FslXY0z783MXhjCaz9ZZTqNaEwBWZkspNHAtHJaENH0=,aad=x0x9+PnDW81oLbYufq72RmaRZB29IPCALCL94KtmsvQ=,tag=qPyqJ3I9JM6wIJDOmgmJkQ==]
|
||||
- ENC[AES256_GCM,data=To5dwUDJi4Mh3hc=,iv=03vcf/AJaUKcHKEnGPq7ih8/xaKHewYiFkQcWOsh7So=,aad=nxUVG7rA+TjyK9BrzVtDGbCp7Iu7BCRLjYvZSnI5iCI=,tag=41ExX9KH+jRYvn51aaP6OA==]
|
||||
sops:
|
||||
kms:
|
||||
enc: CiC6yCOtzsnFhkfdIslYZ0bAf//gYLYCmIu87B3sy/5yYxKnAQEBAQB4usgjrc7JxYZH3SLJWGdGwH//4GC2ApiLvOwd7Mv+cmMAAAB+MHwGCSqGSIb3DQEHBqBvMG0CAQAwaAYJKoZIhvcNAQcBMB4GCWCGSAFlAwQBLjARBAwkRAZG5vQyIKvIKPwCARCAO9zQ43qeQ8loKu0HzXRnpqi6MK/+TpbO22sH0NkVXddXNTl7lfPjKc6gJynrEVdu6aCslUYIid+3FONY
|
||||
enc_ts: 1439587921.752637
|
||||
arn: arn:aws:kms:us-east-1:656532927350:key/920aff2e-c5f1-4040-943a-047fa387b27e
|
||||
|
||||
To decrypt, using flag `-d`.
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sops -d newfile.yaml
|
||||
myapp1: t00m4nys3cr3tz
|
||||
app2:
|
||||
db:
|
||||
user: bob
|
||||
[...]
|
||||
|
||||
Set the env variable **SOPS_KMS_ARN** to your KMS ARN value to avoid
|
||||
needing to set the `-k` flag every time you create a file.
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ export SOPS_KMS_ARN="arn:aws:kms:us-east-1:656532927350:key/920aff2e-c5f1-4040-943a-047fa387b27e"
|
||||
$ sops newfile.yaml
|
||||
|
||||
Requirements
|
||||
------------
|
||||
* `boto3 <https://pypi.python.org/pypi/boto3/1.1.1>`_
|
||||
* `ruamel.yaml <https://pypi.python.org/pypi/ruamel.yaml>`_; requires
|
||||
libyaml-devel and python-devel prior to `pip install`-ing it.
|
||||
|
||||
.. code::
|
||||
|
||||
sudo yum install libyaml-devel python-devel
|
||||
sudo pip install ruamel.yaml
|
||||
|
||||
* `cryptography <https://pypi.python.org/pypi/cryptography>`_; requires
|
||||
libffi-devel prior to `pip install`-ing it.
|
||||
|
||||
.. code::
|
||||
|
||||
sudo yum install libffi-devel
|
||||
sudo pip install cryptography
|
||||
|
||||
License
|
||||
-------
|
||||
Mozilla Public License Version 2.0
|
||||
|
||||
Authors
|
||||
-------
|
||||
* Julien Vehent
|
||||
|
||||
Credits
|
||||
-------
|
||||
|
||||
`sops` is inspired by projects like `hiera-eyaml
|
||||
<https://github.com/TomPoulton/hiera-eyaml>`_, `credstash
|
||||
<https://github.com/LuminalOSS/credstash>`_ and `sneaker
|
||||
<https://github.com/codahale/sneaker>`_.
|
||||
|
|
|
|||
|
|
@ -0,0 +1,5 @@
|
|||
ENC[AES256_GCM,data=2CC7cVfI1TwHUdVoyEtXuyS+h5ZeFRQ7RzTxsUBsXjj1WqXoVxVDqSl8prx/C2wsjW5OBB/YaZr2huUtbAlPxbW2B1B8d6u/0NBZBAC3pJTRYnYF9oOyLoKRHkMkEYhRHjs9yUx9IZvPAY6tnHO8LBh7V+qzFpZ/ZSbbvyIlNEEVymEn7VNNjsozOFiDyaPh7KVK/u6KsLqcYSElVgmVen3IoPimPoD6WytJYvovkoKmbTz+gMpQACrrDk98wRmbl8isMGT/+QhShoYDLqgxqLIzT9Yqf+9H9xnKPMuqITsaC0oq1t4E5Uh653tTgpxUvsIbNzye2bMcKHslldMtcthnQas0QgCSzN0f82WSNTn4v9bf914NFdc1uF+JHfKUnped0u9X0APbiVdxokPKcXr4KQIItfQMKlzoa3D4cjWpx2Lz2l/0pZYrlmuciCznEJXSwcIE/8RhoCvQbR/3ckw/3fhFoFRIshAD6WFSXfm4EUHWbIvEPJ3SfhFAVBws0O5LZEqXryVYEkHteqCfBW5Kh88LSuiWZ5H/3/gDmwpM4xv7Hrxo5I4e/2ZogyySzh71C84ngVUt9jlmDvdo7tzmlEeG6tVktS0HFQaVyCL5tHDYY6Xh7Wg=,iv=IJXjsaYvvEYC6pX4Wec9CS29xHoZ84Y/n0rm5CQJXZY=,aad=7TFp7NRJtNlM10CuQC0RpeULfLxONWGUSiR6tvO8EdY=,tag=SaOeqojH9lHk+XOqyuF6UQ==]
|
||||
# --- sops encryption info. do not edit. ---
|
||||
SOPS_KMS_ARN=arn:aws:kms:us-east-1:656532927350:key/920aff2e-c5f1-4040-943a-047fa387b27e
|
||||
SOPS_KMS_ENC=CiC6yCOtzsnFhkfdIslYZ0bAf//gYLYCmIu87B3sy/5yYxKnAQEBAQB4usgjrc7JxYZH3SLJWGdGwH//4GC2ApiLvOwd7Mv+cmMAAAB+MHwGCSqGSIb3DQEHBqBvMG0CAQAwaAYJKoZIhvcNAQcBMB4GCWCGSAFlAwQBLjARBAwGvcLMWiThqNayEncCARCAO2XIzjbSX1e8HjKcRkSXpZldSkigmO/WHg++c5ee1YarV9zaQB46Vd95baJZxQrGWW89bRKT53Nb/IsN
|
||||
SOPS_KMS_ENCTS=1439584577.42
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
myapp1: ENC[AES256_GCM,data=Tr7oc19nc6t1m9OrUeo=,iv=1vzlPZLfy6wa14/x17P8Ix8wEGDeY0v2dIboZmmwpww=,aad=NpobRzMzpDOkqijzONm8KglltzG+aBV7BJAxtm77veo=,tag=kaYqRgGGBhXhODSSmIZwyA==]
|
||||
app2:
|
||||
db:
|
||||
user: ENC[AES256_GCM,data=CwE4O1s=,iv=S0fozGAOxNma/pWDUuk1iEaYw0wlba0VOLHjPxIok2k=,aad=nEVizsMMyBXOxySnOHw/trTFBSW72nh+Q80YU7TPgIo=,tag=XaGsYaL9LCkLWJI0uxnTYw==]
|
||||
password: ENC[AES256_GCM,data=p673JCgHYw==,iv=EOOeivCp/Fd80xFdMYX0QeZn6orGTK8CeckmipjKqYY=,aad=UAhi/SHK0aCzptnFkFG4dW8Vv1ASg7TDHD6lui9mmKQ=,tag=QE6uuhRx+cGInwSVdmxXzA==]
|
||||
# private key for secret operations in app2
|
||||
key: |-
|
||||
ENC[AES256_GCM,data=Ea3zTFSOlg1PDZmBa1U2dtKl3pO4nTmaFswJx41fPfq3u8O2/Bq1UVfXn2SrO13obfr6xH4zuUceCDTvW2qvphlan5ir609EXt4dE2TEEcjVKhmAHf4LMwlZVAbvTJtlsnvo/aYJH95uctjsSX5h8pBlLaTGBGYwMrZuMyRU6vdcMWyha+piJckUc9sq7fevy1TSqIxf1Usbn/0NEklWm2VSNzQ2Urqtny6EXar+xU7NfYSRJ3mqmcJZ14oIeXPdpk962RwMEFWdYrbE7D59kWU2BgMjDxYJD5KXpWiw2YCrA/wsATxVCbZlwqC+TJFA5WAUZX756mFhV/t2Li3zQyDNUe6KkMXV9qwf/oV1j5sVRVFsKDYIBqhi3qWBVA+SO9RloQMjhru+IsdbQcS4LKq/1DrBENeZuJ0djUAxKLVfJzMGUf89ju3m9IEPovW8mfF0RbfAGRwFHMO9nEXCxrTLERf3owdR3u4j5/rNBpIvvy1z+2dy6sAx/eyNdS+cn5qO9BPAxsXpSwkaI96rlBagwH1Pfxus0x/D00j93OpE+M8MgQ/9LA68FlCFU4OAQlvw8f7MPoxnq+/+gFTS/qqjTR6EoUuX5NH2WY93YCC5TCbe4GOXyP0H05PbIWq55UMVLNcpAyac3gO4kL5O5U8=,iv=Dl61tsemKH0fdmNul/PmEEsRYFAh8GorR8GRupus/EM=,aad=Ft2aSYYukD1x8pMj1WvmodLjJV6waPy5FqdlImWyQKA=,tag=EPg4KpWqni/buCFjFL857A==]
|
||||
an_array:
|
||||
- ENC[AES256_GCM,data=v8dfh92oL8IcgjQ=,iv=HgNNPlQh9GNdE+YPvG4Ufpb2I0sIlEpCsOW3lJA1uBE=,aad=21GroP5gb9sCTxZIahN1NhMGqRPQZZksAr5Q7eCeHRc=,tag=gLsjVqot9+Pqck9LJC+bVA==]
|
||||
- ENC[AES256_GCM,data=X1LMy27AE9SI4h0=,iv=oA1kSg9esGxAvi3qhpcM6Ewrh+p0CFV5cgf6jSPpM08=,aad=CZ7FGJNko6367sd6PwbrIgN/V7Rly4TptbQ1gVsXT1Q=,tag=HerE4nTstX2QZhMn3CPZcw==]
|
||||
- ENC[AES256_GCM,data=KNkH9iI0bSyvcP3E+BRbqfcPUv3YBbCmtvbK1y+sHMI6Z1kXnkX4RoyYiZZXrM680Nh/p0TxNOdNsA==,iv=1h3KbThwTsRaVF+k+dnSwfocSEoyT00X279Dg1Wro60=,aad=foCwpM862VeAD2/7bHRJHAYISneTUJweoSRl2oAdsI4=,tag=tNuCjsNqIy5FVDRu39dQcw==]
|
||||
sops:
|
||||
kms:
|
||||
enc: CiC6yCOtzsnFhkfdIslYZ0bAf//gYLYCmIu87B3sy/5yYxKnAQEBAQB4usgjrc7JxYZH3SLJWGdGwH//4GC2ApiLvOwd7Mv+cmMAAAB+MHwGCSqGSIb3DQEHBqBvMG0CAQAwaAYJKoZIhvcNAQcBMB4GCWCGSAFlAwQBLjARBAyGdRODuYMHbA8Ozj8CARCAO7opMolPJUmBXd39Zlp0L2H9fzMKidHm1vvaF6nNFq0ClRY7FlIZmTm4JfnOebPseffiXFn9tG8cq7oi
|
||||
enc_ts: 1439568549.245995
|
||||
arn: arn:aws:kms:us-east-1:656532927350:key/920aff2e-c5f1-4040-943a-047fa387b27e
|
||||
|
|
@ -0,0 +1,424 @@
|
|||
#!/usr/bin/env python
|
||||
# This Source Code Form is subject to the terms of the Mozilla Public
|
||||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
#
|
||||
# Contributor: Julien Vehent jvehent@mozilla.com [:ulfr]
|
||||
|
||||
from __future__ import print_function
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import argparse
|
||||
from base64 import b64encode, b64decode
|
||||
from ruamel.yaml.comments import CommentedMap
|
||||
from textwrap import dedent
|
||||
import ruamel.yaml
|
||||
import json
|
||||
import boto3
|
||||
from cryptography.hazmat.primitives.ciphers import Cipher, modes, algorithms
|
||||
from cryptography.hazmat.backends import default_backend
|
||||
import time
|
||||
import subprocess
|
||||
import random
|
||||
import re
|
||||
|
||||
|
||||
DESC = """
|
||||
`sops` encrypts and decrypts secrets using AWS KMS. It requires access
|
||||
to AWS uising credentials in ~/.aws/credentials .
|
||||
|
||||
The ARN of the KMS Key used to encrypt/decrypt a document must be specified
|
||||
in a top-level key of the document. For example:
|
||||
YAML
|
||||
sops:
|
||||
kms:
|
||||
arn: arn:aws:kms:us-east-1:656532927350:key/305caadb
|
||||
JSON
|
||||
{"sops":{"kms":{"arn": "arn:aws:kms:us-east-1:650:key/305caadb"}}}
|
||||
TEXT
|
||||
SOPS_KMS_ARN="arn:aws:kms:us-east-1:6565350:key/30db-b886-4e12-8d"
|
||||
|
||||
Alternatively, the KMS Key ID can be defined on the command line (-k flag) and
|
||||
in the environment variable $SOPS_KMS_ARN. The order of preference to select
|
||||
which KMS ID to use is: 1) file, 2) -k flag, 3) environment variable.
|
||||
|
||||
By default, editing is done in vim. Set the env variable $EDITOR to use
|
||||
a different editor.
|
||||
|
||||
Mozilla Services - ulfr, relud - 2015
|
||||
"""
|
||||
SOPS_KMS_ARN = ""
|
||||
SOPS_FOOTER = "# --- sops encryption info. do not edit. ---"
|
||||
|
||||
def main():
|
||||
argparser = argparse.ArgumentParser(
|
||||
usage='sops <file>',
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
description='Encrypted secrets editor',
|
||||
epilog=dedent(DESC))
|
||||
argparser.add_argument('file',
|
||||
help="file to edit; create it if it doesn't exist")
|
||||
argparser.add_argument('-k', '--kms', dest='kmsarn',
|
||||
help="ARN of KMS key used for (en|de)cryption, "
|
||||
"if none is defined in <file>.")
|
||||
argparser.add_argument('-d', '--decrypt', action='store_true',
|
||||
dest='decrypt',
|
||||
help="Decrypt <file> and print it to stdout")
|
||||
argparser.add_argument('-e', '--encrypt', action='store_true',
|
||||
dest='encrypt',
|
||||
help="encrypt <file> and print it to stdout")
|
||||
argparser.add_argument('--input-type', dest='input_type',
|
||||
help="input type (yaml, json, ...). "
|
||||
"If undef, use file extension.")
|
||||
argparser.add_argument('--output-type', dest='output_type',
|
||||
help="output type (yaml, json, ...). "
|
||||
"If undef, use input type.")
|
||||
args = argparser.parse_args()
|
||||
|
||||
global SOPS_KMS_ARN
|
||||
if args.kmsarn:
|
||||
SOPS_KMS_ARN = args.kmsarn
|
||||
elif 'SOPS_KMS_ARN' in os.environ:
|
||||
SOPS_KMS_ARN = os.environ['SOPS_KMS_ARN']
|
||||
|
||||
if args.input_type:
|
||||
itype = args.input_type
|
||||
else:
|
||||
itype = detect_filetype(args.file)
|
||||
|
||||
if args.output_type:
|
||||
otype = args.output_type
|
||||
else:
|
||||
otype = itype
|
||||
|
||||
try:
|
||||
fstat = os.stat(args.file)
|
||||
# read the encrypted file from disk
|
||||
tree = load_tree(args.file, itype)
|
||||
except:
|
||||
if args.encrypt or args.decrypt:
|
||||
panic("cannot operate on non-existent file")
|
||||
print("%s doesn't exist, creating it." % args.file)
|
||||
tree = dict()
|
||||
|
||||
if args.encrypt:
|
||||
# Encrypt mode: encrypt, display and exit
|
||||
key = get_kms_data_key(tree)
|
||||
tree = walk_and_encrypt(tree, key)
|
||||
|
||||
elif args.decrypt:
|
||||
# Decrypt mode: decrypt, display and exit
|
||||
key = get_kms_data_key(tree)
|
||||
tree = walk_and_decrypt(tree, key)
|
||||
|
||||
else:
|
||||
# EDIT Mode: decrypt, edit, encrypt and save
|
||||
key = get_kms_data_key(tree)
|
||||
|
||||
# we need a stash to save the IV and AAD and reuse them
|
||||
# if a given value has not changed during editing
|
||||
stash = {'sops': {'has_stash': True}}
|
||||
tree = walk_and_decrypt(tree, key, stash=stash)
|
||||
|
||||
# the decrypted tree is written to a tempfile and an editor
|
||||
# is opened on the file
|
||||
tmppath = write_file(tree, filetype=otype)
|
||||
tmpstamp = os.stat(tmppath)
|
||||
run_editor(tmppath)
|
||||
|
||||
# verify if file has been modified, and if not, just exit
|
||||
tmpstamp2 = os.stat(tmppath)
|
||||
if tmpstamp == tmpstamp2:
|
||||
os.remove(tmppath)
|
||||
panic("%s has not been modified, exit without writing" % args.file)
|
||||
|
||||
# encrypt the tree
|
||||
tree = load_tree(tmppath, otype)
|
||||
os.remove(tmppath)
|
||||
tree = walk_and_encrypt(tree, key, stash)
|
||||
|
||||
# if we're in -e or -d mode, display to stdout
|
||||
if args.encrypt or args.decrypt:
|
||||
tmppath = write_file(tree, filetype=otype)
|
||||
with open(tmppath, 'r') as f:
|
||||
print(f.read())
|
||||
os.remove(tmppath)
|
||||
|
||||
# otherwise, write the encrypted tree to a file
|
||||
else:
|
||||
path = write_file(tree, path=args.file, filetype=otype)
|
||||
print("file written to %s" % (path), file=sys.stderr)
|
||||
|
||||
|
||||
def detect_filetype(file):
|
||||
"""
|
||||
Detect the type of file based on its extension.
|
||||
Return a string that describes the format: `text`, `yaml`, `json`
|
||||
"""
|
||||
if len(file) > 5:
|
||||
if file[-5:] == '.yaml':
|
||||
return 'yaml'
|
||||
elif file[-5:] == '.json':
|
||||
return 'json'
|
||||
return 'text'
|
||||
|
||||
|
||||
def load_tree(path, filetype):
|
||||
"""
|
||||
Read data from `path` using format defined by `filetype`.
|
||||
Return a dictionary with the data
|
||||
"""
|
||||
with open(path, "r") as fd:
|
||||
if filetype == 'yaml':
|
||||
return ruamel.yaml.load(fd, ruamel.yaml.RoundTripLoader)
|
||||
elif filetype == 'json':
|
||||
return json.load(fd)
|
||||
else:
|
||||
tree = dict()
|
||||
tree['data'] = ""
|
||||
tree['sops'] = dict()
|
||||
tree['sops']['kms'] = dict()
|
||||
for line in fd:
|
||||
if line.startswith(SOPS_FOOTER):
|
||||
continue
|
||||
elif line.startswith('SOPS_KMS_ARN'):
|
||||
tree['sops']['kms']['arn'] = line.rstrip('\n').split('=', 1)[1]
|
||||
elif line.startswith('SOPS_KMS_ENCTS'):
|
||||
tree['sops']['kms']['enc_ts'] = line.rstrip('\n').split('=', 1)[1]
|
||||
elif line.startswith('SOPS_KMS_ENC'):
|
||||
tree['sops']['kms']['enc'] = line.rstrip('\n').split('=', 1)[1]
|
||||
else:
|
||||
tree['data'] += line
|
||||
return tree
|
||||
|
||||
|
||||
def walk_and_decrypt(branch, key, stash=None):
|
||||
"""
|
||||
Walk the branch recursively and decrypt leaves
|
||||
"""
|
||||
for k, v in branch.items():
|
||||
if k == 'sops':
|
||||
continue # everything under the `sops` key stays in clear
|
||||
nstash = dict()
|
||||
if stash:
|
||||
stash[k] = {'has_stash': True}
|
||||
nstash = stash[k]
|
||||
if isinstance(v, dict):
|
||||
branch[k] = walk_and_decrypt(v, key, nstash)
|
||||
else:
|
||||
# this is a value, decrypt it
|
||||
if isinstance(v, ruamel.yaml.scalarstring.PreservedScalarString):
|
||||
ev = decrypt(str(v), key, nstash)
|
||||
branch[k] = ruamel.yaml.scalarstring.PreservedScalarString(ev)
|
||||
elif isinstance(v, list):
|
||||
lstash = dict()
|
||||
kl = []
|
||||
for i, lv in enumerate(list(v)):
|
||||
if nstash:
|
||||
nstash[i] = {'has_stash': True}
|
||||
lstash = nstash[i]
|
||||
kl.append(decrypt(lv, key, lstash))
|
||||
branch[k] = kl
|
||||
else:
|
||||
branch[k] = decrypt(v, key, nstash)
|
||||
return branch
|
||||
|
||||
|
||||
def decrypt(value, key, stash=None):
|
||||
"""
|
||||
Return a decrypted value
|
||||
"""
|
||||
# extract fields using a regex
|
||||
res = re.match(r'^ENC\[AES256_GCM,data=(.+),iv=(.+),aad=(.+),tag=(.+)\]$',
|
||||
value)
|
||||
enc_value = b64decode(res.group(1))
|
||||
iv = b64decode(res.group(2))
|
||||
aad = b64decode(res.group(3))
|
||||
tag = b64decode(res.group(4))
|
||||
decryptor = Cipher(algorithms.AES(key),
|
||||
modes.GCM(iv, tag),
|
||||
default_backend()).decryptor()
|
||||
decryptor.authenticate_additional_data(aad)
|
||||
cleartext = decryptor.update(enc_value) + decryptor.finalize()
|
||||
if stash:
|
||||
# save the values for later if we need to reencrypt
|
||||
stash['iv'] = iv
|
||||
stash['aad'] = aad
|
||||
stash['cleartext'] = cleartext
|
||||
return cleartext
|
||||
|
||||
|
||||
def walk_and_encrypt(branch, key, stash=None):
|
||||
"""
|
||||
Walk the branch recursively and call encrypt of leaves.
|
||||
"""
|
||||
for k, v in branch.items():
|
||||
if k == 'sops':
|
||||
continue # everything under the `sops` key stays in clear
|
||||
nstash = dict()
|
||||
if stash and k in stash:
|
||||
nstash = stash[k]
|
||||
if isinstance(v, dict):
|
||||
# recursively walk the tree
|
||||
branch[k] = walk_and_encrypt(v, key, nstash)
|
||||
else:
|
||||
# this is a value, convert v to an encryptable type
|
||||
# and encrypt
|
||||
if isinstance(v, ruamel.yaml.scalarstring.PreservedScalarString):
|
||||
ev = encrypt(str(v), key, nstash)
|
||||
branch[k] = ruamel.yaml.scalarstring.PreservedScalarString(ev)
|
||||
elif type(v) is not list and isinstance(v, list):
|
||||
lstash = dict()
|
||||
kl = []
|
||||
for i, lv in enumerate(list(v)):
|
||||
if nstash and i in nstash:
|
||||
lstash = nstash[i]
|
||||
kl.append(encrypt(lv, key, lstash))
|
||||
branch[k] = kl
|
||||
else:
|
||||
branch[k] = encrypt(v, key, nstash)
|
||||
return branch
|
||||
|
||||
|
||||
def encrypt(value, key, stash=None):
|
||||
"""
|
||||
Return an encrypted string of the value provided.
|
||||
"""
|
||||
# if we have a stash, and the value of cleartext has not changed,
|
||||
# attempt to take the IV and AAD value from the stash.
|
||||
# if the stash has no existing value, or the cleartext has changed,
|
||||
# generate new IV and AAD.
|
||||
if stash and stash['cleartext'] == value:
|
||||
iv = stash['iv']
|
||||
aad = stash['aad']
|
||||
else:
|
||||
iv = os.urandom(32)
|
||||
aad = os.urandom(32)
|
||||
encryptor = Cipher(algorithms.AES(key),
|
||||
modes.GCM(iv),
|
||||
default_backend()).encryptor()
|
||||
encryptor.authenticate_additional_data(aad)
|
||||
enc_value = encryptor.update(value) + encryptor.finalize()
|
||||
return "ENC[AES256_GCM,data={value},iv={iv},aad={aad}," \
|
||||
"tag={tag}]".format(value=b64encode(enc_value),
|
||||
iv=b64encode(iv),
|
||||
aad=b64encode(aad),
|
||||
tag=b64encode(encryptor.tag))
|
||||
|
||||
|
||||
def get_kms_arn(tree):
|
||||
"""
|
||||
Return the ARN of the KMS Key used to encrypt/decrypt the AES key.
|
||||
If the value exists in the tree, use it. Otherwise try to read it
|
||||
from env variable SOPS_KMS_ARN. If neither exist, panic!
|
||||
"""
|
||||
if 'sops' not in tree or \
|
||||
'kms' not in tree['sops'] or \
|
||||
'arn' not in tree['sops']['kms'] or \
|
||||
tree['sops']['kms']['arn'] == "":
|
||||
# key is not in tree, try the env variable and store it in the tree
|
||||
if SOPS_KMS_ARN != "":
|
||||
if 'sops' not in tree:
|
||||
tree['sops'] = dict()
|
||||
if 'kms' not in tree['sops']:
|
||||
tree['sops']['kms'] = dict()
|
||||
tree['sops']['kms']['arn'] = SOPS_KMS_ARN
|
||||
return SOPS_KMS_ARN
|
||||
else:
|
||||
panic("KMS ARN not found, unable to continue")
|
||||
return tree['sops']['kms']['arn']
|
||||
|
||||
|
||||
def get_kms_data_key(tree):
|
||||
"""
|
||||
Obtain a key and an IV. The values are either stored in encrypted form in
|
||||
the tree, and we ask KMS to decrypt it, or it doesn't exist and we
|
||||
ask KMS to generate one.
|
||||
Return a 32 bytes key
|
||||
"""
|
||||
kms = boto3.client('kms')
|
||||
kms_key = get_kms_arn(tree)
|
||||
if 'sops' not in tree or \
|
||||
'kms' not in tree['sops'] or \
|
||||
'enc' not in tree['sops']['kms'] or \
|
||||
tree['sops']['kms'] == "":
|
||||
# no key found, ask KMS for a data key
|
||||
try:
|
||||
kms_response = kms.generate_data_key(KeyId=kms_key,
|
||||
NumberOfBytes=32)
|
||||
except:
|
||||
panic("Could not obtain data key from KMS using ARN %s" % kms_key)
|
||||
# store the encrypted blob of the data key in the tree
|
||||
if 'sops' not in tree:
|
||||
tree['sops'] = dict()
|
||||
if 'kms' not in tree['sops']:
|
||||
tree['sops']['kms'] = dict()
|
||||
tree['sops']['kms']['enc'] = b64encode(kms_response['CiphertextBlob'])
|
||||
tree['sops']['kms']['enc_ts'] = time.time()
|
||||
print("new data key generated from kms: %s..." %
|
||||
tree['sops']['kms']['enc'][0:32], file=sys.stderr)
|
||||
else:
|
||||
# use existing data key, ask kms to decrypt it
|
||||
try:
|
||||
kms_response = kms.decrypt(
|
||||
CiphertextBlob=b64decode(tree['sops']['kms']['enc']))
|
||||
except Exception as e:
|
||||
panic("Data key decryption error %s" % e)
|
||||
return kms_response['Plaintext'][:32]
|
||||
|
||||
|
||||
def write_file(tree, path=None, filetype=None):
|
||||
"""
|
||||
Write the content of `tree` encoded using the format defined by `filetype`
|
||||
at the location `path`.
|
||||
If `path` is not defined, a tempfile is created.
|
||||
if `filetype` is not defined, tree is treated as a blob of data.
|
||||
|
||||
Return the path of the file written.
|
||||
"""
|
||||
if path:
|
||||
fd = open(path, "wb")
|
||||
else:
|
||||
fd = tempfile.NamedTemporaryFile(suffix="."+filetype, delete=False)
|
||||
path = fd.name
|
||||
if filetype == "yaml":
|
||||
fd.write(ruamel.yaml.dump(tree, Dumper=ruamel.yaml.RoundTripDumper,
|
||||
indent=4))
|
||||
elif filetype == "json":
|
||||
json.dump(tree, fd, sort_keys=True, indent=4)
|
||||
else:
|
||||
if 'data' in tree:
|
||||
fd.write(tree['data'] + "\n")
|
||||
if 'sops' in tree and 'kms' in tree['sops']:
|
||||
fd.write("%s\n" % SOPS_FOOTER)
|
||||
if 'arn' in tree['sops']['kms']:
|
||||
fd.write("SOPS_KMS_ARN=%s\n" % tree['sops']['kms']['arn'])
|
||||
if 'enc' in tree['sops']['kms']:
|
||||
fd.write("SOPS_KMS_ENC=%s\n" % tree['sops']['kms']['enc'])
|
||||
if 'enc_ts' in tree['sops']['kms']:
|
||||
fd.write("SOPS_KMS_ENCTS=%s\n" %
|
||||
tree['sops']['kms']['enc_ts'])
|
||||
fd.close()
|
||||
return path
|
||||
|
||||
|
||||
def run_editor(path):
|
||||
"""
|
||||
Call a text editor on the file given by path.
|
||||
"""
|
||||
editor = "vim"
|
||||
if 'EDITOR' in os.environ:
|
||||
editor = os.environ['EDITOR']
|
||||
subprocess.call([editor, path])
|
||||
return
|
||||
|
||||
|
||||
def panic(msg):
|
||||
from sys import exit
|
||||
print(msg, file=sys.stderr)
|
||||
exit(1)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
Загрузка…
Ссылка в новой задаче