diff --git a/CHANGELOG.md b/CHANGELOG.md index 3a390cf2..f2553296 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,21 +1,40 @@ -2.0.0 / 2018-11-22 +2.0.0 / 2018-11-26 ================== -2.0.0 is a significant change. -If you are upgrading from a prior version be sure to make backups before upgrading. +2.0.0 is a *significant*, backwards-incompaitble release. -In addition, 2.0.0 is a backwards-incompatible release. Many plugins will be broken in CTFd 2.0.0, and if you're having -trouble updating your plugins please join [the CTFd Slack](https://slack.ctfd.io/) for help and discussion. +Many unofficial plugins will not be supported in CTFd 2.0.0. If you're having trouble updating your plugins +please join [the CTFd Slack](https://slack.ctfd.io/) for help and discussion. + +If you are upgrading from a prior version be sure to make backups and have a reversion plan before upgrading. + +* If upgrading from 1.2.0 please make use of the `migrations/1_2_0_upgrade_2_0_0.py` script as follows: + 1. Make all necessary backups. Backup the database, uploads folder, and source code directory. + 2. Upgrade the source code directory (i.e. `git pull`) but do not run any updated code yet. + 3. Set the `DATABASE_URL` in `CTFd/config.py` to point to your existing CTFd database. + 3. Run the upgrade script from the CTFd root folder i.e. `python migrations/1_2_0_upgrade_2_0_0.py`. + * This migration script will attempt to migrate data inside the database to 2.0.0 but it cannot account for every situation. + * Examples of situations where you may need to manually migrate data: + * Tables/columns created by plugins + * Tables/columns created by forks + * Using databases which are not officially supported (e.g. sqlite, postgres) + 4. Setup the rest of CTFd (i.e. config.py), migrate/update any plugins, and run normally. +* If upgrading from a version before 1.2.0, please upgrade to 1.2.0 and then continue with the steps above. **General** * Seperation of Teams into Users and Teams. -* Integration with MajorLeagueCyber. (https://majorleaguecyber.org) + * Use User Mode if you want users to register as themselves and play on their own. + * Use Team Mode if you want users to create and join teams to play together. +* Integration with MajorLeagueCyber (MLC). (https://majorleaguecyber.org) + * Organizers can register their event with MLC and will receive OAuth Client ID & Client Secret. + * Organizers can set those OAuth credentials in CTFd to allow users and teams to automatically register in a CTF. * Data is now provided to the front-end via the REST API. (#551) * Javascript uses `fetch()` to consume the REST API. -* Dynamic Challenges built in. -* S3 Uploader built in. (#661) -* Real time notifications. (#600) +* Dynamic Challenges are built in. +* S3 backed uploading/downloading built in. (#661) +* Real time notifications/announcements. (#600) + * Uses long-polling instead of websockets to simplify deployment. * Email address domain whitelisting. (#603) * Database exporting to CSV. (#656) * Imports/Exports rewritten to act as backups. @@ -27,6 +46,7 @@ trouble updating your plugins please join [the CTFd Slack](https://slack.ctfd.io * Based on https://github.com/umpirsky/country-list/blob/master/data/en_US/country.csv. * Sessions are no longer stored using secure cookies. (#658) * Sessions are now stored server side in a cache (`filesystem` or `redis`) allowing for session revocation. + * In order to delete the cache during local development you can delete `CTfd/.data/filesystem_cache`. * Challenges can now have requirements which must be met before the challenge can be seen/solved. * Workshop mode, score hiding, registration hiding, challenge hiding have been changed to visibility settings. * Users and Teams can now be banned preventing access to the CTF. @@ -34,13 +54,8 @@ trouble updating your plugins please join [the CTFd Slack](https://slack.ctfd.io * WORKERS count in `docker-entrypoint.sh` defaults to 1. (#716) * `docker-entrypoint.sh` exits on any error. (#717) * Increased test coverage. +* Create `SAFE_MODE` configuration to disable loading of plugins. * Migrations have been reset. - * If upgrading from 1.2.0: - 1. Make all necessary backups. Backup the database, uploads folder, and source code directory. - 2. Upgrade the source code directory. - 3. Set the `DATABASE_URL` in `CTFd/config.py`. - 3. Run the upgrade script from the CTFd folder i.e. `python migrations/1_2_0_upgrade_2_0_0.py`. - 4. Setup the rest of CTFd and run normally. **Themes** @@ -48,24 +63,27 @@ trouble updating your plugins please join [the CTFd Slack](https://slack.ctfd.io * Javascript uses `fetch()` to consume the REST API. * The admin theme is no longer considered seperated from the core theme and should always be together. * Themes now use `url_for()` to generate URLs instead of hardcoding. -* socket.io is used to connect to CTFd to receive notifications. +* socket.io (via long-polling) is used to connect to CTFd to receive notifications. * `ctf_name()` renamed to `get_ctf_name()` in themes. * `ctf_logo()` renamed to `get_ctf_logo()` in themes. * `ctf_theme()` renamed to `get_ctf_theme()` in themes. * Update Font-Awesome to 5.4.1. * Update moment.js to 2.22.2. (#704) +* Workshop mode, score hiding, registration hiding, challenge hiding have been changed to visibility functions. + * `accounts_visible()`, `challenges_visible()`, `registration_visible()`, `scores_visible()` **Plugins** * Plugins are loaded in `sorted()` order -* Rename challenge type plugins to use .html and have simplified names. (create, update, view) -* Many functions moved around because utils.py has been broken up and refactored. (#475) -* Marshmallow (https://marshmallow.readthedocs.io) is now used by the REST API to validate and serialize/deserialize data. - * Marshmallow schemas and views are used to restrict SQLAlchemy columns to user types. +* Rename challenge type plugins to use `.html` and have simplified names. (create, update, view) +* Many functions have moved around because utils.py has been broken up and refactored. (#475) +* Marshmallow (https://marshmallow.readthedocs.io) is now used by the REST API to validate and serialize/deserialize API data. + * Marshmallow schemas and views are used to restrict SQLAlchemy columns to user roles. * The REST API features swagger support but this requires more utilization internally. * Errors can now be provided between routes and decoraters through message flashing. (CTFd.utils.helpers; get_errors, get_infos, info_for, error_for) * Email registration regex relaxed. (#693) * Many functions have moved and now have dedicated utils packages for their category. +* Create `SAFE_MODE` configuration to disable loading of plugins. 1.2.0 / 2018-05-04 diff --git a/CTFd/plugins/dynamic_challenges/__init__.py b/CTFd/plugins/dynamic_challenges/__init__.py index 2e930ca6..238f32bb 100644 --- a/CTFd/plugins/dynamic_challenges/__init__.py +++ b/CTFd/plugins/dynamic_challenges/__init__.py @@ -2,7 +2,7 @@ from __future__ import division # Use floating point for math calculations from CTFd.plugins.challenges import BaseChallenge, CHALLENGE_CLASSES from CTFd.plugins import register_plugin_assets_directory from CTFd.plugins.flags import get_flag_class -from CTFd.models import db, Solves, Fails, Flags, Challenges, Files, Tags, Teams, Hints +from CTFd.models import db, Solves, Fails, Flags, Challenges, ChallengeFiles, Tags, Teams, Hints from CTFd import utils from CTFd.utils.migrations import upgrade from CTFd.utils.user import get_ip @@ -122,10 +122,10 @@ class DynamicValueChallenge(BaseChallenge): Fails.query.filter_by(challenge_id=challenge.id).delete() Solves.query.filter_by(challenge_id=challenge.id).delete() Flags.query.filter_by(challenge_id=challenge.id).delete() - files = Files.query.filter_by(challenge_id=challenge.id).all() + files = ChallengeFiles.query.filter_by(challenge_id=challenge.id).all() for f in files: delete_file(f.id) - Files.query.filter_by(challenge_id=challenge.id).delete() + ChallengeFiles.query.filter_by(challenge_id=challenge.id).delete() Tags.query.filter_by(challenge_id=challenge.id).delete() Hints.query.filter_by(challenge_id=challenge.id).delete() DynamicChallenge.query.filter_by(id=challenge.id).delete() diff --git a/CTFd/themes/admin/templates/configs/settings.html b/CTFd/themes/admin/templates/configs/settings.html index 904b46eb..b3a33232 100644 --- a/CTFd/themes/admin/templates/configs/settings.html +++ b/CTFd/themes/admin/templates/configs/settings.html @@ -38,6 +38,9 @@ Admins Only + + This setting should generally be the same as Account Visibility to avoid conflicts. +
@@ -58,6 +61,9 @@ Admins Only + + This setting should generally be the same as Score Visibility to avoid conflicts. +
diff --git a/CTFd/utils/updates/__init__.py b/CTFd/utils/updates/__init__.py index 1bdf8651..e58d4910 100644 --- a/CTFd/utils/updates/__init__.py +++ b/CTFd/utils/updates/__init__.py @@ -36,7 +36,7 @@ def update_check(force=False): if update: try: - name = get_config('ctf_name') or '' + name = str(get_config('ctf_name')) or '' params = { 'ctf_id': sha256(name), 'current': app.VERSION, diff --git a/README.md b/README.md index 303990f5..b5f4fb4b 100644 --- a/README.md +++ b/README.md @@ -11,11 +11,12 @@ CTFd is a Capture The Flag framework focusing on ease of use and customizability ## Features * Create your own challenges, categories, hints, and flags from the Admin Interface - * Dynamic Scoring Challenges + * Dynamic Scoring Challenges * Unlockable challenge support * Challenge plugin architecture to create your own custom challenges * Static & Regex based flags - * Users can unlock hints for free or with points + * Custom flag plugins + * Unlockable hints * File uploads to the server or an Amazon S3-compatible backend * Limit challenge attempts & hide challenges * Automatic bruteforce protection @@ -57,7 +58,7 @@ https://demo.ctfd.io/ ## Support To get basic support, you can join the [CTFd Slack Community](https://slack.ctfd.io/): [![CTFd Slack](https://slack.ctfd.io/badge.svg)](https://slack.ctfd.io/) -If you prefer commercial support or have a special project, send us an email: [support@ctfd.io](mailto:support@ctfd.io). +If you prefer commercial support or have a special project, feel free to [contact us](https://ctfd.io/contact/). ## Managed Hosting Looking to use CTFd but don't want to deal with managing infrastructure? Check out [the CTFd website](https://ctfd.io/) for managed CTFd deployments. @@ -67,7 +68,7 @@ CTFd is heavily integrated with [MajorLeagueCyber](https://majorleaguecyber.org/ By registering your CTF event with MajorLeagueCyber users can automatically login, track their individual and team scores, submit writeups, and get notifications of important events. -To integrate with MajorLeagueCyber, simply register an account, create an event, and install the client ID and client secret in the relevant portion in CTFd/config.py: +To integrate with MajorLeagueCyber, simply register an account, create an event, and install the client ID and client secret in the relevant portion in `CTFd/config.py` or in the admin panel: ```python OAUTH_CLIENT_ID = None diff --git a/migrations/1_2_0_upgrade_2_0_0.py b/migrations/1_2_0_upgrade_2_0_0.py index 04de75a5..f42fdded 100644 --- a/migrations/1_2_0_upgrade_2_0_0.py +++ b/migrations/1_2_0_upgrade_2_0_0.py @@ -129,7 +129,7 @@ if __name__ == '__main__': submissions = [] for solve in old_data['solves']: - solve.pop('id') # ID of a solve doesn't really matter + solve.pop('id') # ID of a solve doesn't really matter solve['challenge_id'] = solve.pop('chalid') solve['user_id'] = solve.pop('teamid') solve['provided'] = solve.pop('flag') @@ -138,7 +138,7 @@ if __name__ == '__main__': submissions.append(solve) for wrong_key in old_data['wrong_keys']: - wrong_key.pop('id') # ID of a fail doesn't really matter. + wrong_key.pop('id') # ID of a fail doesn't really matter. wrong_key['challenge_id'] = wrong_key.pop('chalid') wrong_key['user_id'] = wrong_key.pop('teamid') wrong_key['provided'] = wrong_key.pop('flag') @@ -168,16 +168,70 @@ if __name__ == '__main__': print('MIGRATING Config') banned = [ 'ctf_version', - 'setup' ] + workshop_mode = None + hide_scores = None + prevent_registration = None + view_challenges_unregistered = None + view_scoreboard_if_authed = None + + challenge_visibility = 'private' + registration_visibility = 'public' + score_visibility = 'public' + account_visibility = 'public' for config in old_data['config']: config.pop('id') + + if config['key'] == 'workshop_mode': + workshop_mode = config['value'] + elif config['key'] == 'hide_scores': + hide_scores = config['value'] + elif config['key'] == 'prevent_registration': + prevent_registration = config['value'] + elif config['key'] == 'view_challenges_unregistered': + view_challenges_unregistered = config['value'] + elif config['key'] == 'view_scoreboard_if_authed': + view_scoreboard_if_authed = config['value'] + if config['key'] not in banned: new_conn['config'].insert(dict(config)) + + if workshop_mode: + score_visibility = 'admins' + account_visibility = 'admins' + + if hide_scores: + score_visibility = 'hidden' + + if prevent_registration: + registration_visibility = 'private' + + if view_challenges_unregistered: + challenge_visibility = 'public' + + if view_scoreboard_if_authed: + score_visibility = 'private' + new_conn['config'].insert({ 'key': 'user_mode', 'value': 'users' }) + new_conn['config'].insert({ + 'key': 'challenge_visibility', + 'value': challenge_visibility + }) + new_conn['config'].insert({ + 'key': 'registration_visibility', + 'value': registration_visibility + }) + new_conn['config'].insert({ + 'key': 'score_visibility', + 'value': score_visibility + }) + new_conn['config'].insert({ + 'key': 'account_visibility', + 'value': account_visibility + }) del old_data['config'] manual = [] @@ -192,7 +246,7 @@ if __name__ == '__main__': for row in data: new_conn[table].insert(dict(row)) ran = True - else: # We finished inserting + else: # We finished inserting if ran: manual.append(table) @@ -208,6 +262,7 @@ if __name__ == '__main__': for table in manual: print('\t', 'ALTER TABLE `{table}` ADD PRIMARY KEY(id)'.format(table=table)) - print('The following tables were not created because they were empty and must be manually recreated (e.g. app.db.create_all()') + print( + 'The following tables were not created because they were empty and must be manually recreated (e.g. app.db.create_all()') for table in not_created: print('\t', table) diff --git a/tests/challenges/__init__.py b/tests/challenges/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/tests/challenges/test_dynamic.py b/tests/challenges/test_dynamic.py new file mode 100644 index 00000000..0f1a8852 --- /dev/null +++ b/tests/challenges/test_dynamic.py @@ -0,0 +1,116 @@ +#!/usr/bin/env python +# -*- coding: utf-8 -*- + +from CTFd.plugins.dynamic_challenges import DynamicChallenge, DynamicValueChallenge +from tests.helpers import * + + +def test_can_create_dynamic_challenge(): + """Test that dynamic challenges can be made from the API/admin panel""" + app = create_ctfd(enable_plugins=True) + with app.app_context(): + register_user(app) + client = login_as_user(app, name="admin", password="password") + + challenge_data = { + "name": "name", + "category": "category", + "description": "description", + "value": 100, + "decay": 20, + "minimum": 1, + "state": "hidden", + "type": "dynamic" + } + + r = client.post('/api/v1/challenges', json=challenge_data) + assert r.get_json().get('data')['id'] == 1 + + challenges = DynamicChallenge.query.all() + assert len(challenges) == 1 + + challenge = challenges[0] + assert challenge.value == 100 + assert challenge.initial == 100 + assert challenge.decay == 20 + assert challenge.minimum == 1 + destroy_ctfd(app) + + +def test_can_update_dynamic_challenge(): + """Test that dynamic challenges can be deleted""" + app = create_ctfd(enable_plugins=True) + with app.app_context(): + challenge_data = { + "name": "name", + "category": "category", + "description": "description", + "value": 100, + "decay": 20, + "minimum": 1, + "state": "hidden", + "type": "dynamic" + } + req = FakeRequest(form=challenge_data) + challenge = DynamicValueChallenge.create(req) + + assert challenge.value == 100 + assert challenge.initial == 100 + assert challenge.decay == 20 + assert challenge.minimum == 1 + + challenge_data = { + "name": "new_name", + "category": "category", + "description": "new_description", + "value": "200", + "initial": "200", + "decay": "40", + "minimum": "5", + "max_attempts": "0", + "state": "visible" + } + + req = FakeRequest(form=challenge_data) + challenge = DynamicValueChallenge.update(challenge, req) + + assert challenge.name == 'new_name' + assert challenge.description == "new_description" + assert challenge.value == 200 + assert challenge.initial == 200 + assert challenge.decay == 40 + assert challenge.minimum == 5 + assert challenge.state == "visible" + + destroy_ctfd(app) + + +def test_can_delete_dynamic_challenge(): + app = create_ctfd(enable_plugins=True) + with app.app_context(): + register_user(app) + client = login_as_user(app, name="admin", password="password") + + challenge_data = { + "name": "name", + "category": "category", + "description": "description", + "value": 100, + "decay": 20, + "minimum": 1, + "state": "hidden", + "type": "dynamic" + } + + r = client.post('/api/v1/challenges', json=challenge_data) + assert r.get_json().get('data')['id'] == 1 + + challenges = DynamicChallenge.query.all() + assert len(challenges) == 1 + + challenge = challenges[0] + DynamicValueChallenge.delete(challenge) + + challenges = DynamicChallenge.query.all() + assert len(challenges) == 0 + destroy_ctfd(app) diff --git a/tests/helpers.py b/tests/helpers.py index 49abf364..5a9e61ff 100644 --- a/tests/helpers.py +++ b/tests/helpers.py @@ -4,6 +4,7 @@ from CTFd.models import * from CTFd.cache import cache from sqlalchemy_utils import database_exists, create_database, drop_database from sqlalchemy.engine.url import make_url +from collections import namedtuple import datetime import six import gc @@ -16,6 +17,9 @@ else: binary_type = bytes +FakeRequest = namedtuple('FakeRequest', ['form']) + + def create_ctfd(ctf_name="CTFd", name="admin", email="admin@ctfd.io", password="password", user_mode="users", setup=True, enable_plugins=False): if enable_plugins: TestingConfig.SAFE_MODE = False