Commit Graph

163 Commits

Author SHA1 Message Date
Kevin Chung
9264e96428 Mark 3.1.0 (#1634)
# 3.1.0 / 2020-09-08

**General**

- Loosen team password confirmation in team settings to also accept the team captain's password to make it easier to change the team password
- Adds the ability to add custom user and team fields for registration/profile settings.
- Improve Notifications pubsub events system to use a subscriber per server instead of a subscriber per browser. This should improve the reliability of CTFd at higher load and make it easier to deploy the Notifications system

**Admin Panel**

- Add a comments functionality for admins to discuss challenges, users, teams, pages
- Adds a legal section in Configs where users can add a terms of service and privacy policy
- Add a Custom Fields section in Configs where admins can add/edit custom user/team fields
- Move user graphs into a modal for Admin Panel

**API**

- Add `/api/v1/comments` to manipulate and create comments

**Themes**

- Make scoreboard caching only cache the score table instead of the entire page. This is done by caching the specific template section. Refer to #1586, specifically the changes in `scoreboard.html`.
- Add rel=noopener to external links to prevent tab napping attacks
- Change the registration page to reference links to Terms of Service and Privacy Policy if specified in configuration

**Miscellaneous**

- Make team settings modal larger in the core theme
- Update tests in Github Actions to properly test under MySQL and Postgres
- Make gevent default in serve.py and add a `--disable-gevent` switch in serve.py
- Add `tenacity` library for retrying logic
- Add `pytest-sugar` for slightly prettier pytest output
- Add a `listen()` method to `CTFd.utils.events.EventManager` and `CTFd.utils.events.RedisEventManager`.
  - This method should implement subscription for a CTFd worker to whatever underlying notification system there is. This should be implemented with gevent or a background thread.
  - The `subscribe()` method (which used to implement the functionality of the new `listen()` function) now only handles passing notifications from CTFd to the browser. This should also be implemented with gevent or a background thread.
2020-09-08 00:08:35 -04:00
Kevin Chung
976b191b53 Fix submission search (#1605)
* Fix submission searching in Admin Panel
2020-08-14 12:20:41 -04:00
Kevin Chung
5d7e0e39c7 1580 fix hidden admin scores (#1581)
* Fix issue where admins could not see user graphs/api data if score visibility was set to hidden
* Closes #1580
2020-08-04 13:23:46 -04:00
Kevin Chung
dc3a4d275b Fix issue with previewing certain pages (#1571)
* Fix previewing pages when page attributes are set
2020-07-24 14:52:35 -04:00
Kevin Chung
4d4097a267 Add channel to update check and use new endpoint (#1568)
* Add a channel key to update check and use a new GET based update check endpoint
* Bump `black` version to `19.10b0`
2020-07-24 01:59:14 -04:00
Kevin Chung
1725e632cf 3.0.0a2 dev (#1528)
# 3.0.0a2 / 2020-07-09

**General**

* Accept additional profile fields during registration (affiliation, website, country)
  * This does not add additional inputs. Themes or additional JavaScript can add the form inputs.

**Admin Panel**

* Redesign the challenge creation form to use a radio button with challenge type selection instead of a select input

**API**

* Admins can no longer ban themselves through `PATCH /api/v1/users/[user_id]`

**Themes**

* Spinner centering has been switched from a hard coded margin in CSS to flexbox CSS classes from Bootstrap

**Plugins**

* Revert plugin menu (`register_admin_plugin_menu_bar`, `register_user_page_menu_bar`) changes to 2.x code

**Miscellaneous**

* Fix issue with `Configs.ctf_name` returning incorrect value
* Add prerender step back into challenges.js
* Better handling of missing challenge types. Missing challenge types no longer bring down all other challenges.
2020-07-09 13:40:35 -04:00
Kevin Chung
7dcfba40b3 Simple bugfix merges (#1531)
* Closes #1530 
* Clean up some wording and comments
* Pin isort version to fix flake8 issue (https://github.com/gforcada/flake8-isort/issues/88)
2020-07-06 15:34:55 -04:00
Kevin Chung
adc70fb320 3.0.0a1 (#1523)
Alpha release of CTFd v3. 

# 3.0.0a1 / 2020-07-01

**General**

- CTFd is now Python 3 only
- Render markdown with the CommonMark spec provided by `cmarkgfm`
- Render markdown stripped of any malicious JavaScript or HTML.
  - This is a significant change from previous versions of CTFd where any HTML content from an admin was considered safe.
- Inject `Config`, `User`, `Team`, `Session`, and `Plugin` globals into Jinja
- User sessions no longer store any user-specific attributes.
  - Sessions only store the user's ID, CSRF nonce, and an hmac of the user's password
  - This allows for session invalidation on password changes
- The user facing side of CTFd now has user and team searching
- GeoIP support now available for converting IP addresses to guessed countries

**Admin Panel**

- Use EasyMDE as an improved description/text editor for Markdown enabled fields.
- Media Library button now integrated into EasyMDE enabled fields
- VueJS now used as the underlying implementation for the Media Library
- Fix setting theme color in Admin Panel
- Green outline border has been removed from the Admin Panel

**API**

- Significant overhauls in API documentation provided by Swagger UI and Swagger json
- Make almost all API endpoints provide filtering and searching capabilities
- Change `GET /api/v1/config/<config_key>` to return structured data according to ConfigSchema

**Themes**

- Themes now have access to the `Configs` global which provides wrapped access to `get_config`.
  - For example, `{{ Configs.ctf_name }}` instead of `get_ctf_name()` or `get_config('ctf_name')`
- Themes must now specify a `challenge.html` which control how a challenge should look.
- The main library for charts has been changed from Plotly to Apache ECharts.
- Forms have been moved into wtforms for easier form rendering inside of Jinja.
  - From Jinja you can access forms via the Forms global i.e. `{{ Forms }}`
  - This allows theme developers to more easily re-use a form without having to copy-paste HTML.
- Themes can now provide a theme settings JSON blob which can be injected into the theme with `{{ Configs.theme_settings }}`
- Core theme now includes the challenge ID in location hash identifiers to always refer the right challenge despite duplicate names

**Plugins**

- Challenge plugins have changed in structure to better allow integration with themes and prevent obtrusive Javascript/XSS.
  - Challenge rendering now uses `challenge.html` from the provided theme.
  - Accessing the challenge view content is now provided by `/api/v1/challenges/<challenge_id>` in the `view` section. This allows for HTML to be properly sanitized and rendered by the server allowing CTFd to remove client side Jinja rendering.
  - `challenge.html` now specifies what's required and what's rendered by the theme. This allows the challenge plugin to avoid having to deal with aspects of the challenge besides the description and input.
  - A more complete migration guide will be provided when CTFd v3 leaves beta
- Display current attempt count in challenge view when max attempts is enabled
- `get_standings()`, `get_team_stanadings()`, `get_user_standings()` now has a fields keyword argument that allows for specificying additional fields that SQLAlchemy should return when building the response set.
  - Useful for gathering additional data when building scoreboard pages
- Flags can now control the message that is shown to the user by raising `FlagException`
- Fix `override_template()` functionality

**Deployment**

- Enable SQLAlchemy's `pool_pre_ping` by default to reduce the likelihood of database connection issues
- Mailgun email settings are now deprecated. Admins should move to SMTP email settings instead.
- Postgres is now considered a second class citizen in CTFd. It is tested against but not a main database backend. If you use Postgres, you are entirely on your own with regards to supporting CTFd.
- Docker image now uses Debian instead of Alpine. See https://github.com/CTFd/CTFd/issues/1215 for rationale.
- `docker-compose.yml` now uses a non-root user to connect to MySQL/MariaDB
- `config.py` should no longer be editting for configuration, instead edit `config.ini` or the environment variables in `docker-compose.yml`
2020-07-01 12:06:05 -04:00
Kevin Chung
7cf6d2b43a 2.5.0 dev (#1453)
2.5.0 / 2020-06-02
==================

**General**
* Use a session invalidation strategy inspired by Django. Newly generated user sessions will now include a HMAC of the user's password. When the user's password is changed by someone other than the user the previous HMACs will no longer be valid and the user will be logged out when they next attempt to perform an action.
* A user and team's place, and score are now cached and invalidated on score changes.

**API**
* Add `/api/v1/challenges?view=admin` to allow admin users to see all challenges regardless of their visibility state
* Add `/api/v1/users?view=admin` to allow admin users to see all users regardless of their hidden/banned state
* Add `/api/v1/teams?view=admin` to allow admin users to see all teams regardless of their hidden/banned state
* The scoreboard endpoints `/api/v1/scoreboard` & `/api/v1/scoreboard/top/[count]` should now be more performant because score and place for Users/Teams are now cached

**Deployment**
* `docker-compose` now provides a basic nginx configuration and deploys nginx on port 80

**Miscellaneous**
* The `get_config` and `get_page` config utilities now use SQLAlchemy Core instead of SQLAlchemy ORM for slight speedups
* Update Flask-Migrate to 2.5.3 and regenerate the migration environment. Fixes using `%` signs in database passwords.
2020-06-02 11:22:01 -04:00
Ernesto Serrano
912016f6f8 Fix file deletion. Related to #1393 (#1396)
* Delete files when the database reference is also deleted Related to #1393
2020-05-13 21:35:46 -04:00
Kevin Chung
2769dc6367 Have EventManagers yield before timer code to force SSE response head… (#1400)
* EventManagers should send an initial ping event to force `text/event-steam` header to be set
2020-05-13 19:33:01 -04:00
Kevin Chung
6f0c0b1a52 Add test for dynamic challenge reset 2020-05-07 12:27:04 -04:00
Kevin Chung
f0921f42f5 Run format 2020-05-04 09:49:31 -04:00
Kevin Chung
672798ece6 Add a test for clear_all_user_sessions 2020-05-04 04:06:29 -04:00
Kevin Chung
fb6b30ab1b Fix lint 2020-05-04 01:29:28 -04:00
Kevin Chung
625c17db30 Invalidate any previously cached attributes on user login 2020-05-03 20:47:25 -04:00
Kevin Chung
805b48d00d Test clear_user_session and clear sessions on user modifications 2020-04-29 20:19:40 -04:00
Kevin Chung
abd8aae84f Reorder code and fix tests 2020-04-29 18:41:18 -04:00
Kevin Chung
453289b0ff Run linters 2020-04-28 04:19:31 -04:00
Kevin Chung
a62b18fadb Fix up reset ui 2020-04-28 03:53:25 -04:00
Kevin Chung
aa2dd04378 Fix reset tests so they cover file deletions 2020-04-28 03:15:28 -04:00
Kevin Chung
1d33ed4cb2 Fix reset tests and enforce foreign keys on sqlite 2020-04-28 02:08:05 -04:00
Kevin Chung
1f87efb6c1 Better constant value management (#1335)
* Starts work on #929 
* Adds Enum classes that can be accessed from JS, Jinja, and Python code. This allows for the sharing of constant values between the three major codebases in CTFd.
2020-04-23 10:48:09 -04:00
Kevin Chung
25fe789da8 Add fix for SMTP server overriding and #1306 (#1330)
* Fix a potential issue where config.py SMTP settings might not have been overrideable from the Admin Panel
* Closes #1306 by using `email.message.EmailMessage` in Python 3. Python 2 will use the old `sendmail` behavior.
2020-04-18 03:33:34 -04:00
Kevin Chung
cd6439f2eb Add a bell curve graph and total points calculation (#1325)
* Add a bell curve graph and total points calculation to admin panel statistics
* Closes #608
2020-04-17 03:10:55 -04:00
Kevin Chung
578b5261b2 Deprecates storing user type in session (#1323)
* Deprecates storing session["type"] as a means of referring to the user's current level. Instead you should refer to the database.
* Adds `CTFd.utils.user.get_current_user_type()` to get the current user's type or return None if the user is unauthed. 
* Closes #1279
2020-04-15 03:04:18 -04:00
Kevin Chung
04e6b2011f Prevent accidental double unlocks (#1315)
* Prevent a hint from being unlocked twice
* Closes  #1301
2020-04-11 22:03:51 -04:00
Kevin Chung
e5f128ec9d Fix awards in teams mode (#1289)
* Fix awards not being properly assigned to teams in `TEAMS_MODE`
2020-03-14 15:36:44 -04:00
Zander Work
2f106d4fc3 Fix #1249 (#1250)
* Fix behavior for `REVERSE_PROXY` setting when set to a boolean instead of a string
2020-02-29 23:37:57 -05:00
Kevin Chung
22c132358e 2.3.0 (#1248)
2.3.0 / 2020-02-17
==================

**General**
* During setup, admins can register their email address with the CTFd LLC newsletter for news and updates
* Fix editting hints from the admin panel
* Allow admins to insert HTML code directly into the header and footer (end of body tag) of pages. This replaces and supercedes the custom CSS feature.
    * The `views.custom_css` route has been removed.
* Admins can now customize the content of outgoing emails and inject certain variables into email content.
* The `manage.py` script can now manipulate the CTFd Configs table via the `get_config` and `set_config` commands. (e.g. `python manage.py get_config ctf_theme` and `python manage.py set_config ctf_theme core`)

**Themes**
* Themes should now reference the `theme_header` and `theme_footer` configs instead of the `views.custom_css` endpoint to allow for user customizations. See the `base.html` file of the core theme.

**Plugins**
* Make `ezq` functions available to `CTFd.js` under `CTFd.ui.ezq`

**Miscellaneous**
* Python imports sorted with `isort` and import order enforced
* Black formatter running on a majority of Python code
2020-02-17 02:17:25 -05:00
Kevin Chung
1049a14b90 Fix SMTP email From header and remove 'Admin' from the From header (#1229)
* Fix SMTP email From header and remove 'Admin' from the From header
2020-02-11 21:35:58 -05:00
Kevin Chung
f660ed1fb7 Strip spaces on registration and have reset password use email address instead of names (#1218)
* Usernames are now properly stripped before being used in registration checks
* Reset password function now uses email addresses instead of user names for tokens
* Prevent MLC users from resetting their password
2020-01-20 14:22:06 -05:00
Kevin Chung
b8d0f80d01 2.2.0 (#1188)
2.2.0 / 2019-12-22
==================

## Notice
2.2.0 focuses on updating the front end of CTFd to use more modern programming practices and changes some aspects of core CTFd design. If your current installation is using a custom theme or custom plugin with ***any*** kind of JavaScript, it is likely that you will need to upgrade that theme/plugin to be useable with v2.2.0. 

**General**
* Team size limits can now be enforced from the configuration panel
* Access tokens functionality for API usage
* Admins can now choose how to deliver their notifications
    * Toast (new default)
    * Alert
    * Background
    * Sound On / Sound Off
* There is now a notification counter showing how many unread notifications were received
* Setup has been redesigned to have multiple steps
    * Added Description
    * Added Start time and End time,
    * Added MajorLeagueCyber integration
    * Added Theme and color selection
* Fixes issue where updating dynamic challenges could change the value to an incorrect value
* Properly use a less restrictive regex to validate email addresses
* Bump Python dependencies to latest working versions
* Admins can now give awards to team members from the team's admin panel page

**API**
* Team member removals (`DELETE /api/v1/teams/[team_id]/members`) from the admin panel will now delete the removed members's Submissions, Awards, Unlocks

**Admin Panel**
* Admins can now user a color input box to specify a theme color which is injected as part of the CSS configuration. Theme developers can use this CSS value to change colors and styles accordingly.
* Challenge updates will now alert you if the challenge doesn't have a flag
* Challenge entry now allows you to upload files and enter simple flags from the initial challenge creation page

**Themes**
* Significant JavaScript and CSS rewrite to use ES6, Webpack, yarn, and babel
* Theme asset specially generated URLs
    * Static theme assets are now loaded with either .dev.extension or .min.extension depending on production or development (i.e. debug server)
    * Static theme assets are also given a `d` GET parameter that changes per server start. Used to bust browser caches.
* Use `defer` for script tags to not block page rendering
* Only show the MajorLeagueCyber button if configured in configuration
* The admin panel now links to https://help.ctfd.io/ in the top right
* Create an `ezToast()` function to use [Bootstrap's toasts](https://getbootstrap.com/docs/4.3/components/toasts/)
* The user-facing navbar now features icons
* Awards shown on a user's profile can now have award icons
* The default MarkdownIt render created by CTFd will now open links in new tabs
* Country flags can now be shown on the user pages

**Deployment**
* Switch `Dockerfile` from `python:2.7-alpine` to `python:3.7-alpine`
* Add `SERVER_SENT_EVENTS` config value to control whether Notifications are enabled
* Challenge ID is now recorded in the submission log

**Plugins**
* Add an endpoint parameter to `register_plugin_assets_directory()` and `register_plugin_asset()` to control what endpoint Flask uses for the added route

**Miscellaneous**
* `CTFd.utils.email.sendmail()` now allows the caller to specify subject as an argument
    * The subject allows for injecting custom variable via the new `CTFd.utils.formatters.safe_format()` function
* Admin user information is now error checked during setup
* Added yarn to the toolchain and the yarn dev, yarn build, yarn verify, and yarn clean scripts
* Prevent old CTFd imports from being imported
2019-12-22 23:17:34 -05:00
Kevin Chung
b15f1787e4 Prevent users from nulling out profile values (#1125)
* Prevent users from nulling out profile values
2019-10-02 01:22:54 -04:00
Kevin Chung
b8c1970b8e Fix CSV exports in Python 3 by converting StringIO to BytesIO (#1107)
* Fix CSV exports in Python 3 by converting StringIO to BytesIO
2019-09-10 03:22:01 -04:00
Kevin Chung
c88e0556eb 1092 fix solve count leak during freeze (#1095)
* Challenges properly get solve count during freeze time
* Closes #1092
2019-08-29 23:22:24 -04:00
MartinJM
941ca8f506 The place indicator on the team page now excludes counting hidden teams (#1094)
* The place indicator in Team Mode on the team pages and user pages now excludes counting hidden teams.
* Updated `flask-marshmallow` to 0.10.1, `marshmallow-sqlalchemy` to 0.17.0 
* Pinned `marshmallow` to 2.20.2
* Closes #1093
* Closes #1088
2019-08-29 22:04:05 -04:00
Kevin Chung
71240bb13d Require password for email change (#1077)
* Require password for email changes
2019-08-04 20:28:20 -04:00
Kevin Chung
3453eafcab Flaky redis events tests (#1072)
* Fix flaky redis tests
* Add a random integer helper for tests
2019-07-30 03:25:29 -04:00
Sachin S. Kamath
34bab12a99 Add feature to search users by IP address (#1059)
* Allows admins to search users by IP address
* Closes #1054
2019-07-30 01:09:41 -04:00
Kevin Chung
cf7959ab16 Attempt to fix randomly failing tests (#1071)
* Fix flaky tests that require redis which were occasionally failing
2019-07-29 23:15:26 -04:00
Kevin Chung
2bdf7b64d6 Fix place ordinal calculation (#1067)
* Fix scoreboard place ordinalization in Python 3
* Extract ordinalization code to `CTFd.utils.humanize.numbers.ordinalize`.
2019-07-24 01:44:20 -04:00
Kevin Chung
b453125726 Fix missing Team API exceptions (#1058)
* Add require_team decorator to endpoints that request teams.
* Change status code for captain endpoints to return 403 instead of 400
2019-07-12 00:53:35 -04:00
Kevin Chung
447f71d41b Disable foreign keys during import (#1031)
* Temporarily disable foreign keys in MySQL, MariaDB, and Postgres during `import_ctf()`
    * Likely also disables SQLite but SQLite is permissive about foreign keys to begin with so irrelevant.
2019-06-21 12:35:55 -04:00
Kevin Chung
b5632f9289 Cache scoreboard page (#1025)
* Cache the `/scoreboard` page to avoid having to rebuild the response so often
* Update `tests.api.v1.test_scoreboard:test_scoreboard_is_cached` to also test if `/scoreboard` is cached
2019-06-16 13:29:50 -04:00
Kevin Chung
e627391b12 Improve caching (#1014)
* Cache get place code for users and teams. 
    * Fix score changing test helpers to clear standings cache when generating a score changing row
    * `utils._get_config` will now return `KeyError` instead of None. 
    * Separate `/api/v1/[users,teams]/[me,id]/[solves,fails,awards]` into seperate API endpoints
* Install `Flask-DebugToolbar` in development

Main goals covered in #1012
2019-06-15 02:07:24 -04:00
Kevin Chung
e978867a2f Add UI code to handle team member removal (#1022)
* Change `/api/v1/teams/[team_id]/members` from taking `id` to `user_id`.
    * Not even the admin panel was using this endpoint so doesn't seem that drastic of a change
* Add UI to handle team member removal
2019-06-09 00:27:36 -04:00
Kevin Chung
8d91a3fa8d Fix hidden pages not being able to load (#1019)
* Fixes bug where pages marked as `hidden` weren't loading
    * It's possible that some users used this behavior however this fix implements the correct behavior. The `draft` setting can be used to completely hide pages.
2019-06-08 01:00:28 -04:00
Kevin Chung
458ce2e518 Fixing issue downloading files when view_after_ctf is enabled (#1011) 2019-06-03 21:08:30 -04:00
Kevin Chung
387b6880d2 Ratelimit team join attempts (#993)
* Ratelimit team join attempts
2019-05-13 22:53:58 -04:00