Commit Graph

167 Commits

Author SHA1 Message Date
Reinier van der Leer
21c0cdcb76 Disable proxy for internal pull requests (#4953) 2023-07-12 17:50:18 +02:00
Reinier van der Leer
b00570f6d9 Fix CI warnings 2023-07-12 15:04:24 +02:00
Reinier van der Leer
e746a45158 Merge branch 'master' into release-v0.4.4 2023-07-11 23:51:48 +02:00
Reinier van der Leer
4d51469473 Fix CI cassette checkout 2023-07-10 18:13:59 +02:00
Reinier van der Leer
62e330469c Speed up Docker CI 2023-07-10 17:51:38 +02:00
Reinier van der Leer
5e39dd1d26 Speed up CI (#4930)
* Match requests in cassette by hash

* Strip requests more extensively for VCR

* Sort JSON keys on cassette save

* Strip max_tokens from cassettes

* Improve logging in retry decorator

* Raise when quota exceeded

* Clean up @retry_api

* Fix @retry_api

* Remove dead tests/vcr/openai_filter.py

* Add debug logging to execute_python_file

* Make Docker CI pass
2023-07-10 17:26:13 +02:00
Reinier van der Leer
4f87f70904 Add slow test report to CI 2023-07-08 18:55:59 +02:00
Reinier van der Leer
6893b7e5e7 Fix log textareas in bug issue template 2023-07-07 01:10:11 +02:00
Nicholas Tindle
c67940edec Update CODEOWNERS (#4884) 2023-07-04 15:34:46 -05:00
Reinier van der Leer
bbaa5b89c2 Add CI trigger for release-* branches 2023-06-15 13:16:25 +02:00
merwanehamadi
3df8c1b501 Make benchmarks create cassettes without using them (#4664) 2023-06-13 11:29:20 -07:00
merwanehamadi
2ce6ae6707 Change memory challenge expectations (#4657) 2023-06-11 14:34:02 -07:00
merwanehamadi
9150f32f8b Fix benchmark logs (#4653) 2023-06-11 15:34:57 +01:00
merwanehamadi
097ce08908 Create benchmarks.yml (#4647) 2023-06-10 15:11:24 -07:00
merwanehamadi
c1ee8cb62e Upload logs as artifacts (#4640) 2023-06-10 15:07:02 -07:00
Erik Peterson
15c6b0c1c3 Implement directory-based plugin system (#4548)
* Implement directory-based plugin system

* Fix Selenium test

---------

Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: Merwane Hamadi <merwanehamadi@gmail.com>
2023-06-10 13:16:00 -07:00
merwanehamadi
3081f56ecb Quicker logs in pytest (#4486) 2023-06-09 15:18:56 -07:00
merwanehamadi
f4505add69 Set proxy headers as secrets (#4620) 2023-06-07 19:39:07 -07:00
Reinier van der Leer
20a4922b40 Increase thresholds for PR autolabeler (#4602) 2023-06-06 16:37:43 -07:00
Reinier van der Leer
dafbd11686 Rearrange tests & fix CI (#4596)
* Rearrange tests into unit/integration/challenge categories

* Fix linting + `tests.challenges` imports

* Fix obscured duplicate test in test_url_validation.py

* Move VCR conftest to tests.vcr

* Specify tests to run & their order (unit -> integration -> challenges) in CI

* Fail Docker CI when tests fail

* Fix import & linting errors in tests

* Fix `get_text_summary`

* Fix linting errors

* Clean up pytest args in CI

* Remove bogus tests from GoCodeo
2023-06-06 10:48:49 -07:00
merwanehamadi
ee6b97ef5e Fix Python CI "update cassettes" step (#4591)
* Fix updated cassettes step

* Clarifications

* Use github.ref_name instead of github.ref

* Fix duplicate runs on `master`

---------

Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-06-06 16:27:08 +02:00
Reinier van der Leer
576f24e3ae Merge branch 'master' into release-0.4.0 2023-06-05 16:08:35 +02:00
Reinier van der Leer
84e58051fa Clean up CI git logic 2023-06-04 16:01:40 +02:00
Merwane Hamadi
0a20fa4fdf Fix CI git authentication and cassettes 2023-06-03 21:19:58 -07:00
Reinier van der Leer
c4e2d8fbdd debug 2023-06-04 03:28:26 +02:00
Reinier van der Leer
3e5868f223 Fix CI git diff 2023-06-04 02:40:51 +02:00
Reinier van der Leer
4c11b21dff Fix CI git authorization 2023-06-04 02:01:35 +02:00
merwanehamadi
79ba85a22e Cache Python Packages in the CI pipeline (#4488) 2023-06-03 15:48:32 +01:00
Reinier van der Leer
ce44012866 Fix pushing cassettes in CI 2023-06-02 23:27:15 +02:00
Reinier van der Leer
9156db41a1 Merge remote-tracking branch 'origin/stable' into release-0.4.0 2023-06-02 22:58:50 +02:00
Reinier van der Leer
9247f9480c Fix CI for internal PRs with CI changes (#4552)
* Port fixed CI workflow from master

* Trigger CI

* Improve CI concurrency check
2023-06-02 13:49:05 -07:00
merwanehamadi
9e9b128b72 ignore cassettes and current score for all push events in github action (#4485) 2023-05-30 19:16:49 -05:00
merwanehamadi
86b6231f70 ignore push if current score changed (#4482) 2023-05-30 12:23:49 -07:00
merwanehamadi
ba8046753e Update current score when PR merged (#4464) 2023-05-29 19:30:41 -07:00
merwanehamadi
d34b8a2b61 Replace 'prompt change' with 'behavior change' (#4473)
* Replace 'prompt change' with 'behavior change'

* replaced behavior by behaviour

* fixed typo on behaviour
2023-05-30 03:01:46 +01:00
merwanehamadi
ee9f10a8d8 remove unused imports automatically (#4449)
* remove unused imports automatically

* add linters to pr template

* remove useless try statement
2023-05-28 05:50:50 -07:00
merwanehamadi
e7c0d3330c Create cassette submodule (#4420)
* gfeat: specify directory of cassettes and automatically load them depending on module

fix: formatting for linter

test: commit newly generated cassettes to their respective folder

tests: update latest fixtures with master

fix: update .gitattributes with updated path to cassettes

fix: use cassettes from master instead of generating them myself

fix: update path in .gitattributes

fix: make sure to match default functionality by using test name for cassette directory

fix: actually add git submodule

ci: checkout git submodules in CI

ci: update git submodules separately to ensure it gets called

feat: add a hooks directory so we can update git submodules on post-checkout

feat: make sure we push the tests/cassettes submodule on merge into master

ci: remove unused code now that we are using git submodules to keep cassettes in sync

fix: simplify how we load the submodule and fix updating cassettes on merge to master

chore: remove echo of checkout hook, it's unneeded

ci: remove unneccesary step

* cassettes submodule

* cassettes submodule

* cassettes submodule

* cassettes submodule

* cassettes submodule

---------

Co-authored-by: Stefan Ayala <stefanayala3266@gmail.com>
2023-05-26 12:33:49 -07:00
merwanehamadi
6c45fcd067 remove cassettes auto commit (#4432) 2023-05-26 11:02:40 -07:00
k-boikov
d3fc8c4286 add 30 min timeout for tests (#4416) 2023-05-25 16:15:27 -07:00
Pi
6e14782f82 Re-point wiki link in PULL_REQUEST_TEMPLATE.md 2023-05-25 16:00:29 +01:00
Pi
2a6f01aae2 Re-point wiki link in 2.feature.yml 2023-05-25 15:59:50 +01:00
Pi
1ab3a40f03 Repoint to wiki page 1.bug.yml 2023-05-25 15:59:04 +01:00
merwanehamadi
9c7f5671e6 CI: set base repo before gh pr checkout (#4363) 2023-05-23 14:23:03 +02:00
merwanehamadi
3a2800bcdd Cassettes should be added in the same pull request, without extra pull request (#4355)
* fix ci

* fix ci

* add only what we need

* Updated ci.yml to inform about challenge score

* Updated ci.yml to inform about challenge score

---------

Co-authored-by: k-boikov <64261260+k-boikov@users.noreply.github.com>
2023-05-22 20:03:23 -07:00
Benny van der Lans
dcb1cbe5d6 Update 1.bug.yml to improve consistency of data (#4293)
* Update 1.bug.yml to improve consistency of data

in this version of the bug report several dropdown fileds have been added and text fields have been removed to have better control over the data being sent in bug reports

* Update 1.bug.yml fixing indent error

fixing of an indent error

* Update 1.bug.yml

changed the file uploads to input fields for the log contents since files are not supported on github

* Update 1.bug.yml

made the issue field required

* Update 1.bug.yml

Added changes according to luke's comments

---------

Co-authored-by: Pi <sunfish7@gmail.com>
2023-05-22 00:15:10 +01:00
Reinier van der Leer
812be60d2a Delete sponsors_readme.yml workflow (#4304) 2023-05-19 18:07:48 +03:00
merwanehamadi
1c399e67f4 Add $ to timestamp in ci.yml and improve diff (#4266) 2023-05-17 17:52:52 +01:00
Richard Beales
feae20d8fa Update PULL_REQUEST_TEMPLATE.md to include linting (#3625)
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
2023-05-16 23:24:57 -05:00
merwanehamadi
55af3e19b4 CI Pipeline: create cassettes in fork (#4257)
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
2023-05-16 21:49:12 -05:00
merwanehamadi
517c080544 fix ci cassettes (#4234)
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
2023-05-15 22:43:21 -05:00