Compare commits

...

206 Commits

Author SHA1 Message Date
maxime c808ba21ab
[fix] oubli import 2024-05-28 03:40:29 +02:00
maxime 9f908d3cee
[release] halfapi 0.6.30 2024-05-28 02:55:17 +02:00
maxime 7b6d9e994a
[deps] starlette v0.37 2024-05-28 02:54:50 +02:00
maxime 8f6330bca7
[deps] starlette 0.35 2024-05-28 02:53:58 +02:00
maxime 8506aa5322
[middleware] fix DomainMiddleware constructor for 0.35 2024-05-28 02:41:41 +02:00
maxime b683e80959
[halfapi] remove on_startup argument according to starlette 0.26 2024-05-28 02:41:19 +02:00
maxime b412d249a1
[deps] bump dependencies versions 2024-05-28 02:39:08 +02:00
Maxime Alves LIRMM f4f9a0fc66 [dockerfile] alpine 3.19 - utilisation d'un venv 2023-12-13 11:37:29 +01:00
Maxime Alves LIRMM c855cce013 [release] 0.6.29 2023-12-13 11:09:42 +01:00
Maxime Alves LIRMM e083c4386e [release] 0.6.28 2023-11-24 14:31:10 +01:00
Maxime Alves LIRMM 476ae29792 [rc] 0.6.28rc5 2023-09-01 04:59:17 +09:00
Maxime Alves LIRMM 673097adeb [half_domain] fix ACL route when checked ACL is a decorator 2023-09-01 04:58:12 +09:00
Maxime Alves LIRMM 1cc1bbd5ef [rc] 0.6.28rc4 (remove rc3) 2023-08-21 08:47:47 +02:00
Maxime Alves LIRMM 135d6e86e4 [conf] fix export SECRET variable 2023-08-21 08:47:19 +02:00
Maxime Alves LIRMM 0fcf433ec6 [ci] use alpine base image directly 2023-08-21 00:08:29 +02:00
Maxime Alves LIRMM 45cf32de2b [ci] utilisation de l'image 3.17 (pour fix le bug exceptiongroup) 2023-08-20 23:53:18 +02:00
Maxime Alves LIRMM 1b713c3816 [ci] fix image pour les tests <3.11 2023-08-20 23:42:10 +02:00
Maxime Alves LIRMM 59889e1e31 [ci] pipenv --skip-lock (has been remove) 2023-08-20 23:34:34 +02:00
Maxime Alves LIRMM 28a1a69435 [rc] 0.6.28rc3 - fix bugs and general configuration management cleanup (see changelog) 2023-08-20 23:32:50 +02:00
maxime 65ecf9817c [ci/Dockerfile] use of alpine OS instead of debian, bump python to version 3.11 2023-08-09 14:46:52 +02:00
maxime 3b7d3bda5c [rc] 0.6.28rc2 2023-08-09 14:24:32 +02:00
maxime e19f27f306 [deps] starlette 0.31 2023-08-09 14:23:09 +02:00
maxime e9c84c9f7c [rc] 0.6.28-rc1 2023-08-08 09:09:16 +02:00
maxime b1595beb14 [tests] write a schema component for dummy_domain for example 2023-08-02 13:29:57 +02:00
maxime 60ff99d0fb [domain] you can specify Schema components under the "schema_components" key of the domain dictionary 2023-08-02 12:55:04 +02:00
maxime 9657f0f9ec [ci-cd] add python 3.11 to testing releases 2023-08-02 10:57:33 +02:00
maxime f646b4d663 [ci-cd] coverage report 2023-08-02 10:49:54 +02:00
maxime 0817882558 [dev-deps] ajout de la dépendance a coverage 2023-08-02 10:43:50 +02:00
maxime 896ce58731 [pytest] do not avoid to run doctest modules in halfapi sources! 2023-08-02 10:39:06 +02:00
maxime 87856cfb42 [cli-conf] halfapi domain : the file provided as argument is a toml file of the format of .halfapi/config, + better config handling 2023-08-02 10:38:36 +02:00
maxime 4856f80b99 [rc] 0.6.28-rc0-1 2023-08-02 06:32:26 +02:00
maxime eac602f0a5 [rc] 0.6.28rc0 2023-08-01 20:50:58 +02:00
maxime 14e051bd91 [doc-schema] the "/" route on a domain now returns the OpenAPI-validated Schema (not a list of schemas), the "dummy_domain" test now validates OpenAPI specs 2023-08-01 20:31:17 +02:00
maxime 20563081f5 [doc-schema] In module-based routers, if there is a path parameter, you can specify an OpenAPI documentation for it, or a default will be used 2023-08-01 20:24:24 +02:00
maxime 7949b3206c [dev-deps] openapi-schema-validator, openapi-spec-validator 2023-08-01 19:41:49 +02:00
maxime c4583b7187 [doc] add docstrings for halfapi routes 2023-08-01 17:43:59 +02:00
maxime 2413436104 [acl] The public acls check routes use the "HEAD" method, deprecated "GET" 2023-08-01 17:32:25 +02:00
Maxime Alves LIRMM 54cc6c17c9 [release] 0.6.27 2023-06-07 11:46:53 +02:00
Maxime Alves LIRMM ff3a39c740 [rc] 0.6.27rc0 2023-06-01 15:39:57 +02:00
Maxime Alves LIRMM@home 8d254bafa0 [feature] changes in the ACLs result availability 2023-06-01 15:39:44 +02:00
Maxime Alves LIRMM 0a385661b9 [rc] 0.6.26rc0 2023-02-23 11:44:57 +01:00
Maxime Alves LIRMM e065fe04e4 [tests] test with multiple optional parameteres 2023-02-21 19:30:59 +01:00
Maxime Alves LIRMM b7c5704c95 [tests] fix tests so the data is sent in json 2023-02-21 19:23:23 +01:00
Maxime Alves LIRMM dd83a337e9 [lib.domain] route_decorator : Adds the "base_url", "cookies" and "url" to the "halfapi" argument of route definitions 2023-02-21 19:22:57 +01:00
Maxime Alves LIRMM f6d08e8309 [release] 0.6.25 2023-02-20 12:18:38 +01:00
Maxime Alves LIRMM 262de901a8 [jwtMiddleware] on expired signature error, returns Nobody and do not raise an exception 2023-02-13 11:14:24 +01:00
Maxime Alves LIRMM e5c25ede1f [rc] 0.6.25-rc0 2023-02-08 12:54:35 +01:00
Maxime Alves LIRMM b4c37ea999 [release] 0.6.24 2023-02-03 14:22:04 +01:00
Maxime Alves LIRMM 5a7e51ae94 [jwtMiddleware] clean "is_fake_user_id" code 2023-02-03 14:20:01 +01:00
Maxime Alves LIRMM 69129fd7af [doc][release] 0.7.0 2023-02-03 12:43:30 +01:00
Maxime Alves LIRMM a3fc6dc830 [authMiddleware] UN-Breaking uses either the cookie or the header names "Authorization" 2023-02-03 12:43:16 +01:00
Maxime Alves LIRMM@home 064127dc16 [fix] tests for running by implementing the dry-run option to the domain command 2023-02-03 00:02:56 +01:00
Maxime Alves LIRMM@home c2eb95533c [fix] fix check_domain test c9ecc1f8d2 2023-02-02 22:33:35 +01:00
Maxime Alves LIRMM bbb027cd0d [authMiddleware][BREAKING] read token from "JWTToken" cookie 2023-02-02 19:56:59 +01:00
Maxime Alves LIRMM c9ecc1f8d2 [cli] allow to run halfapi on a specified domain using the "halfapi domain --run DOMAIN" command 2023-02-02 19:56:11 +01:00
Maxime Alves LIRMM d1a8351997 [halfapi] if there is only one domain it is automatically enabled 2023-02-02 19:55:37 +01:00
Maxime Alves LIRMM dc29abea84 [release] 0.6.23 2023-01-14 12:03:40 +01:00
Maxime Alves LIRMM 4c966e7151 [deps] toml bumps version 0.10 2023-01-14 11:43:38 +01:00
Maxime Alves LIRMM 7deb353b4b [deps] PyYAML bumps version 6.0 2023-01-14 11:39:40 +01:00
Maxime Alves LIRMM 3986083725 [deps] PyJWT bumps version 2.6.0 2023-01-14 11:37:08 +01:00
Maxime Alves LIRMM 301b0eeab5 [deps] click bump version 8 2023-01-14 11:29:14 +01:00
Maxime Alves LIRMM cc0566542b [deps] orjson bump version 3.8.5 2023-01-14 11:23:25 +01:00
Maxime Alves LIRMM 4f4dac0ff2 [deps] Migration from starlette v0.18 to v0.23
Breaking : migrate your tests that use the TestDomain.client method
following the instructions here https://github.com/Kludex/bump-testclient

Squashed commit of the following:

commit 0417f27b3f
Author: Maxime Alves LIRMM <maxime.alves@lirmm.fr>
Date:   Sat Jan 14 11:08:44 2023 +0100

    [deps] starlette 0.23

commit 552f00a65b
Author: Maxime Alves LIRMM <maxime.alves@lirmm.fr>
Date:   Sat Jan 14 10:59:42 2023 +0100

    [deps] starlette 0.22

commit aefe448717
Author: Maxime Alves LIRMM <maxime.alves@lirmm.fr>
Date:   Sat Jan 14 10:55:45 2023 +0100

    [tests][fix] compares the json interpreted value instead of the string

commit 01333a200c
Author: Maxime Alves LIRMM <maxime.alves@lirmm.fr>
Date:   Sat Jan 14 10:55:20 2023 +0100

    [testing] changes from requests to httpx for Starlette TestClient (breaks)

commit f3784fab7f
Author: Maxime Alves LIRMM <maxime.alves@lirmm.fr>
Date:   Sat Jan 14 10:54:10 2023 +0100

    [deps][breaking] starlette 0.21

commit 717d3f8bd6
Author: Maxime Alves LIRMM <maxime.alves@lirmm.fr>
Date:   Sat Jan 14 10:26:31 2023 +0100

    [responses] use a wrapper function for exception handling (fix starlette 0.20)

commit d0876e45da
Author: Maxime Alves LIRMM <maxime.alves@lirmm.fr>
Date:   Sat Jan 14 10:25:21 2023 +0100

    [deps][breaking] starlette 0.20

commit 6504191c53
Author: Maxime Alves LIRMM <maxime.alves@lirmm.fr>
Date:   Sat Jan 14 10:12:51 2023 +0100

    [deps] starlette 0.19

commit 7b639a8dc2
Author: Maxime Alves LIRMM <maxime.alves@lirmm.fr>
Date:   Sat Jan 14 10:11:14 2023 +0100

    [deps] starlette 0.18

commit 20bd9077a4
Author: Maxime Alves LIRMM <maxime.alves@lirmm.fr>
Date:   Sat Jan 14 10:07:48 2023 +0100

    pipenv update
2023-01-14 11:10:13 +01:00
Maxime Alves LIRMM 8b8caa2e15 [release] 0.6.22 2022-12-02 18:11:26 +01:00
Maxime Alves LIRMM 5a70c00541 [rc] 0.6.22-rc6 2022-10-13 12:21:26 +02:00
Maxime Alves LIRMM 7723acb812 [breaking] domain description is defined in a dict, not in "__name__", "__id__", "__routers__" ... 2022-10-13 12:18:13 +02:00
Maxime Alves LIRMM fd682ba0e0 [fix] wrong "data" argument value when there is a default one 2022-09-13 19:15:45 +02:00
Maxime Alves LIRMM@home e9c57049dd [lib/domain] log.error unhandled exceptions 2022-09-06 21:51:48 +02:00
Maxime Alves LIRMM e47ffcddb9 [deps] update pyJWT>2.4.0 2022-09-06 19:58:41 +02:00
Maxime Alves LIRMM f0c898ba20 (0.6.22rc3) [halfapi] Log HTTPException with statuscode 500 as critical 2022-09-05 10:44:04 +02:00
Maxime Alves LIRMM@home b7e678e00f [rc] 0.6.22-rc2 2022-09-05 10:20:08 +02:00
Maxime Alves LIRMM@home d0ec030ce9 [fix] important fix for argument handling 2022-09-05 10:18:57 +02:00
Maxime Alves LIRMM@home 039bc2c8fe [tests][BREAK] arguments are not filtered (since version 0.6.20 probably) 2022-09-05 10:12:47 +02:00
Maxime Alves LIRMM@home 5e21d4c24f fix ret_type feature 2022-08-31 00:14:10 +02:00
Maxime Alves LIRMM@home 8bdf5cab82 [testing] MODULE and CONFIG attributes 2022-08-31 00:13:29 +02:00
Maxime Alves LIRMM 910e1e1497 [release] 0.6.22-rc0 2022-08-18 20:24:57 +02:00
Maxime Alves LIRMM 53a691d985 [responses] html and plaintext return types as ret_type argument 2022-08-18 20:22:14 +02:00
Maxime Alves LIRMM@home 5d1b88daca [release] 0.6.21 2022-08-08 20:26:31 +02:00
Maxime Alves LIRMM@home c0bd6ddc43 [release] 0.6.21 2022-08-08 20:21:05 +02:00
Maxime Alves LIRMM 779dd2d519 [deps] lock pipenv update 2022-07-18 20:35:49 +02:00
Maxime Alves LIRMM 84747e3f73 [release] 0.6.20-rc0 2022-07-18 20:34:12 +02:00
Maxime Alves LIRMM 18748808c9 [fix] argument handling 2022-07-18 18:47:38 +02:00
Maxime Alves LIRMM 34ede09fe2 [release] 0.6.19 2022-07-06 08:34:11 +02:00
Maxime Alves LIRMM 6ebdf765bd [deps] pytest v7, [ci/cd] no tests on python3.6 2022-07-06 08:29:29 +02:00
Maxime Alves LIRMM 2d47789f61 [ci/cd] parallel tests on multiple python versions 2022-07-06 08:15:49 +02:00
maxime 3ec6d7514e [release] 0.6.18 2022-06-23 15:48:58 +02:00
maxime 3c97d39cdc [config] restore dumped configuration when using HALFAPI_DOMAIN_NAME 2022-06-23 15:33:19 +02:00
maxime f96c712aa8 [release] 0.6.17 2022-06-23 11:07:41 +02:00
maxime 23a93026aa [halfdomain] fix feature HALFAPI_DOMAIN_MODULE for acls route 2022-06-23 11:03:52 +02:00
maxime 05cf37c775 [deps] ajout virtualenv dans les deps (pour build) 2022-06-23 11:00:33 +02:00
maxime a46b045ca6 [release] 0.6.16 2022-06-23 07:24:05 +02:00
maxime f68b7e59b8 handle HALFAPI_DOMAIN_MODULE environment variable 2022-06-23 07:21:49 +02:00
Maxime Alves LIRMM@home b17ce623f4 [release] 0.6.15 2022-05-17 16:19:41 +02:00
Maxime Alves LIRMM@home 7dd7d00625 [halfdomain] fix last modification 2022-05-17 16:13:33 +02:00
Maxime Alves LIRMM@home 389823db82 [deps] Pipfile lock 2022-05-17 16:02:23 +02:00
Maxime Alves LIRMM@home e70239433f [halfdomain] add ability to specify a domain's acl module path in __acl__ attribute 2022-05-17 16:01:58 +02:00
Maxime Alves LIRMM@home 739ffc9afa [release] 0.6.14 2022-05-17 10:31:34 +02:00
Maxime Alves LIRMM@home 99d4aaeb8d [lib/responses] ajout format XLSX, car ODS bug avec les dates
https://github.com/pyexcel/pyexcel-ods/issues/31
2022-05-17 10:30:15 +02:00
Maxime Alves LIRMM 0d9dc2a018 [release] 0.6.13
- (rollback from 0.6.12) Remove pytest from dependencies in Docker file and
  remove tests
- (dep) Add "packaging" dependency
- Add dependency check capability when instantiating a domain (__deps__
  variable, see in dummy_domain)
2022-03-21 09:51:04 +01:00
Maxime Alves LIRMM b63b0f52c6 [domain] checks versions specified in __deps__ variable of domain module 2022-03-21 09:45:35 +01:00
Maxime Alves LIRMM 63d6d1e8ea [release] 0.6.12 2022-03-17 17:48:41 +01:00
Maxime Alves LIRMM e8c99e6012 [ci/cd] install pytest in docker image, run pytest at launch 2022-03-17 17:38:20 +01:00
Maxime Alves LIRMM 568aea9ea8 [clena] nettoyage des fonctions non utilisées 2022-03-16 13:09:50 +01:00
Maxime Alves LIRMM 988a1e5bab [ci/cd] fix kaniko call 2022-03-14 15:24:57 +01:00
Maxime Alves LIRMM de72e469d2 [release] 0.6.11 2022-03-14 15:16:06 +01:00
Maxime Alves LIRMM f7879c6388 [release] 0.6.10
- Add "x-out" field in HTTP headers when "out" parameters are specified for a
  route
- Add "out" kwarg for not-async functions that specify it

- Hide data fields in args_check logs

- Fix testing lib for domains (add default secret and debug option)

- Domains now need to include the following variables in their __init__.py
    - __name__ (str, optional)
    - __id__ (str, optional)
- halfapi domain

- Mounts domain routers with their ACLs as decorator
- Configuration example files for systemd and a system-wide halfapi install
- Runs projects
- Handles JWT authentication middleware
2022-03-08 19:24:24 +01:00
Maxime Alves LIRMM 84179743a6 [release] 0.6.9 2022-03-02 16:46:32 +01:00
Maxime Alves LIRMM adf7f872b6 changelog 2022-02-28 10:21:01 +01:00
Maxime Alves LIRMM b96f4908c6 [release] 0.6.8 2022-02-28 10:10:06 +01:00
Maxime Alves LIRMM a388faf1d8 [testing] add "secret" and "production" variable to test_domain configuration 2022-02-28 09:58:47 +01:00
Maxime Alves LIRMM 4d6e935813 [ci/cd] fix kaniko executor 2022-02-10 16:31:26 +01:00
Maxime Alves LIRMM 4e080f805f [fix] odsresponse 2022-02-10 16:30:08 +01:00
Maxime Alves LIRMM 91dd4cbaa8 [ci/cd] check tag and not branch master 2022-02-10 16:01:38 +01:00
Maxime Alves LIRMM 209c6ef40a [ci/cd] add rules to execute build stage 2022-02-10 14:31:30 +00:00
Maxime Alves LIRMM a1c1bf04df [ci/cd] stages 2022-02-10 15:15:08 +01:00
Maxime Alves LIRMM 90203b2edf [ci] add ./tests to pytest command 2022-02-10 12:38:54 +01:00
Maxime Alves LIRMM a69d2b7639 [tests] async router renamed to async_router 2022-02-10 12:30:49 +01:00
Maxime Alves LIRMM c3153921f7 [tests] test_domain import dummy_domain inside test class 2022-02-10 11:59:40 +01:00
Maxime Alves LIRMM 97fee8ca96 [ci] execute pytest on tests folder 2022-02-10 11:56:35 +01:00
Maxime Alves LIRMM d33c82e348 [0.6.7] fix ODSResponse 2022-02-10 10:30:19 +01:00
Maxime Alves LIRMM@home 058121d985 [release] 0.6.6 2022-01-21 14:39:46 +01:00
Maxime Alves LIRMM@home 2e5680d29a [auth] dont activate authenticationMiddleware if secret is missing. NO SECRET ONLY IN FULLY PUBLIC DOMAINS!!! 2022-01-21 14:39:34 +01:00
Maxime Alves LIRMM@home 979007f287 [fix][log] active domains list logging 2022-01-21 14:38:28 +01:00
Maxime Alves LIRMM@home df555c7d26 [fix] when running one-domain mode we should set the configuration as if the domain was activated 2022-01-21 14:37:54 +01:00
Maxime Alves LIRMM@home be312d4b7a [fix][run] wont pop keys if we dont operate on a copy 2022-01-21 14:32:55 +01:00
Maxime Alves LIRMM f02a97fbf8 [testing] set starlette client in setUp 2021-12-15 12:37:49 +01:00
Maxime Alves LIRMM@home e9ffb553c8 [release] 0.6.4 2021-12-14 09:05:50 +01:00
Maxime Alves LIRMM@home 776cc8c85e [testing] fix check_routes of test_domain 2021-12-14 09:04:06 +01:00
Maxime Alves LIRMM@home 5d5ffdfb7c [tests] re-enable dummy_project route testing 2021-12-14 09:03:45 +01:00
Maxime Alves LIRMM@home 7c0f5717f4 [half_route] error handling 2021-12-14 09:01:38 +01:00
Maxime Alves LIRMM 7fb5e25411 [release] 0.6.3 2021-12-13 14:47:21 +01:00
Maxime Alves LIRMM a0dbbca04d [fix] forgot __init__.py in testing 2021-12-13 14:45:00 +01:00
Maxime Alves LIRMM 95fb267e81 [release] 0.6.2 2021-12-13 13:50:52 +01:00
Maxime Alves LIRMM e9bf94a607 [cli/domain] now instantiate HalfAPI and manually add the domain before calling schema function 2021-12-13 13:49:58 +01:00
Maxime Alves LIRMM f82cd5552b [half_domain] remove "config" argument, now uses "config" attribute from HalfAPI instance, add version and halfapi_versrion attributes, update DomainMiddleware init arguments 2021-12-13 12:45:10 +01:00
Maxime Alves LIRMM bdbad9e296 [halfapi] now inherits from Starlette class. Uses a dict to store HalfDomain instances 2021-12-13 12:43:26 +01:00
Maxime Alves LIRMM 76e942ab91 [tests] add test file for lib/domain_middleware 2021-12-13 12:42:12 +01:00
Maxime Alves LIRMM 8fff1f5372 [tests] fix tests and add "mix_stderr=False" to CliRunner instance 2021-12-13 12:41:33 +01:00
Maxime Alves LIRMM 048c9f1bab [tests] fix for schema lists 2021-12-08 16:45:00 +01:00
Maxime Alves LIRMM d5f39a7929 [schemas] always give a list of schemas 2021-12-08 16:42:12 +01:00
Maxime Alves LIRMM 648841d90f [dummy_domain] set an id to dummy_domain 2021-12-08 15:34:00 +01:00
Maxime Alves LIRMM c658815eb5 [cli] fix halfapi domain --read 2021-12-08 13:08:08 +01:00
Maxime Alves LIRMM 46e62575ae [half_domain] fix test_dummy_project 2021-12-08 12:23:47 +01:00
Maxime Alves LIRMM 7001cec86e [wip] refactor half_domain 2021-12-07 11:42:02 +01:00
Maxime Alves LIRMM@home b4157c4a7d [wip][testfail] multiple-domains app 2021-12-07 07:53:36 +01:00
Maxime Alves LIRMM@home 96f78e76c5 [tests][testfail] add default routes testing, /halfapi/acls fail 2021-12-04 09:56:14 +01:00
Maxime Alves LIRMM@home d54dcd641d [conf] fix secret tempfile creation 2021-12-04 00:28:30 +01:00
Maxime Alves LIRMM 7060d201ec [deps] optional requirements for ODSResponse 2021-12-03 17:26:14 +01:00
Maxime Alves LIRMM dbca2f28fb [conf] use of toml for halfapi configs. re-enable possibility of multiple domains 2021-12-03 17:25:57 +01:00
Maxime Alves LIRMM@home d06857bf49 [config][wip][nf] removal of "domains" and "domain" section 2021-12-03 09:49:30 +01:00
Maxime Alves LIRMM@home 3dc951c81e [tests] tidy cli tests 2021-12-03 09:20:40 +01:00
Maxime Alves LIRMM@home a8e5cfc0ff [wip][responses] allow to change return format with "format" route argument, add ODSResponse 2021-12-01 21:16:19 +01:00
Maxime Alves LIRMM@home 20cada4fa0 [halfapi] fix domain importlib 2021-12-01 21:14:17 +01:00
Maxime Alves LIRMM@home c1bb637be7 [lib.router] forbid extra-keys in routes dict (no more FQTN at same level of methods) 2021-12-01 21:13:35 +01:00
Maxime Alves LIRMM@home a37c2356d6 [lib.domain] error log when missing docstring 2021-12-01 21:12:19 +01:00
Maxime Alves LIRMM@home 038715e94a [halfapi] config option "--dry-run", used in test_domain 2021-12-01 21:11:26 +01:00
Maxime Alves LIRMM@home 2f9005a1a5 [release] 0.6.1 2021-12-01 17:02:56 +01:00
Maxime Alves LIRMM@home a2d79f49b9 [tests] dummy domain test (you can use this example in your own domain) 2021-12-01 17:02:23 +01:00
Maxime Alves LIRMM@home cf98b08fa5 [tests] mv base domain test to halfapi/testing 2021-12-01 17:01:56 +01:00
Maxime Alves LIRMM@home c1191bbb0e [deps] update deps 2021-12-01 16:25:53 +01:00
Maxime Alves LIRMM@home 837c646bc5 [ci] switch to bullseye-3.9 2021-12-01 15:49:59 +01:00
Maxime Alves LIRMM@home 49c13c56ac [lib.domain] implement domain schema - try: halfapi domain dummy_domain 2021-12-01 13:31:46 +01:00
Maxime Alves LIRMM@home 1ccfa0d10e [lib.schemas] router schema update 2021-12-01 13:07:01 +01:00
Maxime Alves LIRMM@home 238bd99bd3 [cli.routes] add json schema export 2021-12-01 13:06:00 +01:00
Maxime Alves LIRMM@home 429a90d786 [halfapi] fix typo 2021-12-01 12:21:26 +01:00
Maxime Alves LIRMM@home 1ec244b60f [lib.constants] route keys are optional, in case of empty routers 2021-12-01 12:20:32 +01:00
Maxime Alves LIRMM@home 53ecbb58fc [tests] rework some tests, avoid calling project_runner multiple times (should be tested better, but for now is just buggy imports...) 2021-12-01 12:20:01 +01:00
Maxime Alves LIRMM@home 7e7bbb3a62 [tests] fix de "project_runner" auto clean sys.path, "tree" fixture to debug projects 2021-12-01 12:16:31 +01:00
Maxime Alves LIRMM@home 5e88109b3e manipuler le sys.path : BAAAD 2021-12-01 12:14:27 +01:00
Maxime Alves LIRMM@home e293ac3867 [cli/domain] create domain function rework - creates a basic domain 2021-12-01 12:08:24 +01:00
Maxime Alves LIRMM@home a98aa27485 [pipfile] remove dependency to python3.8, TODO find a way to specify python>=3.8 2021-12-01 12:02:21 +01:00
Maxime Alves LIRMM@home 15794327f9 [tests] pytest.ini includes tests as pythonpath (because of dummy_domain) 2021-12-01 11:55:56 +01:00
Maxime Alves LIRMM@home cf20b76959 [deps] add pytest-pythonpath 2021-12-01 11:43:49 +01:00
Maxime Alves LIRMM 7e1cc21b8c [wip][testfail] update config monodomain - schema gives acls 2021-11-30 18:31:40 +01:00
Maxime Alves LIRMM ec26438340 [schemas] supression des fonctions api_routes 2021-11-30 14:40:07 +01:00
Maxime Alves LIRMM 7230316296 [ci] add pytest version 2021-11-30 12:13:17 +01:00
Maxime Alves LIRMM f5b7e3392a [tests] skip /halfapi/exception test 2021-11-30 12:07:19 +01:00
Maxime Alves LIRMM 18dbbdd584 [app] enable use of SCHEMA to run halfapi, fix tests 2021-11-30 11:20:26 +01:00
Maxime Alves LIRMM b3b32b47f8 [cli.run] halfapi run SCHEMA.csv 2021-11-30 10:42:00 +01:00
Maxime Alves LIRMM 189fcf86f7 [cli.routes] halfapi routes --export 2021-11-30 01:54:48 +01:00
Maxime Alves LIRMM 55878df260 [lib.domain] use args_check by default, even on async functions 2021-11-30 01:52:02 +01:00
Maxime Alves LIRMM a6985fa9bf [halfapi] configuration globals go lowercase 2021-11-30 01:50:27 +01:00
Maxime Alves LIRMM ed6dcb0513 [halfapi] application becomes a private attribute 2021-11-30 01:49:37 +01:00
Maxime Alves LIRMM 7017827b2b [lib] small fixes 2021-11-30 01:46:30 +01:00
Maxime Alves LIRMM 7c2bf60812 [pytest] remove halfapi from tests directory 2021-11-30 01:43:14 +01:00
Maxime Alves LIRMM 1fda2ab15d [lib.schemas] add schema_csv_dict and schema_to_csv function 2021-11-30 01:42:29 +01:00
Maxime Alves LIRMM c9639ddbc0 [constants] add lib "schema" Schemas for all items 2021-11-30 00:37:13 +01:00
Maxime Alves LIRMM 24bd3f5653 [deps] add "schema" as dependency 2021-11-29 17:01:52 +01:00
Maxime Alves LIRMM@home e203552876 [dummy_domain] add async routes, and routes that use arguments 2021-11-29 08:37:52 +01:00
Maxime Alves LIRMM@home ac4aadc2df [conf][testfail] add SCHEMA dictionary to globals 2021-11-29 06:21:48 +01:00
Maxime Alves LIRMM@home 4dae2f3676 [dummy_domain] add tests for dummy_domain acls 2021-11-29 06:09:18 +01:00
Maxime Alves LIRMM@home c27ed3a966 [routers] docstring is mandator in route methods. YAML is used for markup 2021-11-29 06:08:24 +01:00
Maxime Alves LIRMM@home 47d81c048f [lib] route_acl_decorator becomes HalfRoute.acl_decorator, creation of HalfRoute that wraps starlette.route 2021-11-29 05:42:26 +01:00
Maxime Alves LIRMM ad6877a7e9 [run] add arguments for config 2021-11-23 13:53:08 +01:00
Maxime Alves LIRMM e7e1bfed1b [conf] halfapi config now dumps the .ini of the current config 2021-11-23 13:35:12 +01:00
Maxime Alves LIRMM f0e662e060 [conf] remove is_project variable 2021-11-23 11:39:33 +01:00
Maxime Alves LIRMM ea8e7ede65 [cli.routes:export] shows a header to describe the CSV, can be deactivated 2021-11-22 20:14:05 +01:00
Maxime Alves LIRMM f4880f1f9c [lib.domain] wraps function when loading routes from modules, to get the real name 2021-11-22 20:09:57 +01:00
Maxime Alves LIRMM 4a8cb008e6 [cli.routes] add routes command, allows exporting route dict as csv 2021-11-22 20:06:50 +01:00
Maxime Alves LIRMM 049860fce5 [lib.domain] gen_router_routes yields router module also 2021-11-22 20:06:12 +01:00
Maxime Alves LIRMM 908eab5fdc [lib.user] move JWTUser, Nobody and CheckUser to lib.user 2021-11-22 20:05:17 +01:00
Maxime Alves LIRMM ce672eeb30 [cli.domain] fix list routes 2021-11-22 19:09:02 +01:00
Maxime Alves LIRMM 1f20a336e2 [conf] production now defaults to true 2021-11-22 18:30:29 +01:00
Maxime Alves LIRMM 0173eb6d72 HalfAPI class clean and rewrite 2021-11-22 18:18:06 +01:00
Maxime Alves LIRMM ad9bd45ba0 [logging] use logger from halfapi.logging 2021-11-16 17:45:40 +01:00
Maxime Alves LIRMM 00c7b5caf4 [doc] update README and add joel to license 2021-11-16 17:45:40 +01:00
81 changed files with 5041 additions and 1714 deletions

View File

@ -6,7 +6,7 @@
# Official language image. Look for the different tagged releases at:
# https://hub.docker.com/r/library/python/tags/
image: python:3.8
image: python:alpine3.18
# Change pip's cache directory to be inside the project directory since we can
# only cache local items.
@ -27,30 +27,64 @@ cache:
- .cache/pip
- venv/
test:
script:
- python3 -V # Print out python version for debugging
- pip3 install pipenv
- pipenv install --dev --skip-lock
- pipenv run pytest -v
- pipenv run halfapi --version
stages:
- test
- build
run:
.before_script_template: &test
before_script:
- python3 -V # Print out python version for debugging
- python3 -m venv /tmp/venv
- /tmp/venv/bin/pip3 install .["tests","pyexcel"]
- /tmp/venv/bin/pip3 install coverage pytest
test:
image: python:alpine${ALPINEVERSION}
stage: test
<<: *test
parallel:
matrix:
- ALPINEVERSION: ["3.16", "3.17", "3.18", "3.19"]
script:
- /tmp/venv/bin/pytest --version
- PYTHONPATH=./tests/ /tmp/venv/bin/coverage run --source halfapi -m pytest
- /tmp/venv/bin/coverage xml
- /tmp/venv/bin/halfapi --version
artifacts:
reports:
coverage_report:
coverage_format: cobertura
path: coverage.xml
build_pypi:
stage: build
script:
- python3 -V # Print out python version for debugging
- pip3 install pipenv
- pipenv install --dev --skip-lock
- pipenv run python -m build --sdist
- pipenv run python -m build --wheel
- python3 -m venv /tmp/venv
- /tmp/venv/bin/pip3 install .
artifacts:
paths:
- dist/*.whl
rules:
- if: '$CI_COMMIT_TAG != ""'
variables:
TAG: $CI_COMMIT_TAG
build:
build_container:
rules:
- if: '$CI_COMMIT_TAG != ""'
variables:
IMGTAG: $CI_COMMIT_TAG
- if: '$CI_COMMIT_REF_NAME == "devel"'
variables:
IMGTAG: "latest"
stage: build
image: $CI_REGISTRY/devtools/kaniko
script:
- echo "Will upload image halfapi:\"$IMGTAG\""
- echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_REGISTRY_PASSWORD\"}}}" > /kaniko/.docker/config.json
- /kaniko/executor --context $CI_PROJECT_DIR --dockerfile $CI_PROJECT_DIR/Dockerfile --destination $CI_REGISTRY/malves/halfapi:latest
- /kaniko/executor --force --context $CI_PROJECT_DIR --dockerfile $CI_PROJECT_DIR/Dockerfile --destination $CI_REGISTRY/malves/halfapi:"$IMGTAG"
artifacts:
paths:
- /kaniko/.docker/config.json

View File

@ -1,5 +1,233 @@
# HalfAPI
## 0.6.30
Dependencies updates
- pyYAML v6.0.1
- starlette v0.37.2
Warning : the on_startup halfAPI argument is now removed, use the lifeSpan
## 0.6.29
### Dependencies
Starlette version bumped to 0.33.
## 0.6.28
### Dependencies
Starlette version bumped to 0.31 (had to disable a test to make it work but
seems not important).
### Development dependencies
Python 3.7 is no longer supported (openapi_spec_validator is not compatible).
If you are a developper, you should update dev dependencies in your virtual
environment.
### OpenAPI schemas
This release improves OpenAPI specification in routes, and gives a default
"parameters" field for routes that have path parameters.
Also, if you use halfAPI for multi-domain setups, you may be annoyed by the
change in the return value of the "/" route that differs from "/domain" route.
An HalfAPI instance should return one and only one OpenAPI Schema, so you can
rely on it to connect to other software.
The version number that is contained under the "info" dictionnary is now the "version"
of the Api domain, as specified in the domain dictionnary specified at the root
of the Domain.
The title field of the "info" dictionnary is now the Domain's name.
The ACLs list is now available under the "info.x-acls" attribute of the schema.
It is still accessible from the "/halfapi/acls" route.
#### Schema Components
You can now specify a dict in the domain's metadata dict that follows the
"components" key of an OpenAPI schema.
Use it to define models that are used in your API. You can see an exemple of
it's use in the "tests/dummy_domain/__init__.py" file.
### ACLs
The use of an "HEAD" request to check an ACL is now the norm. Please change all
the occurrences of your calls on theses routes with the GET method.
### CLI
Domain command update :
The `--conftest` flag is now allowed when running the `domain` command, it dumps the current configuration as a TOML string.
`halfapi domain --conftest my_domain`
The `--dry-run` flag was buggy and is now fixed when using the `domai ` command with the `--run` flag.
### Configuration
The `port` option in a `domain.my_domain` section in the TOML config file is now prefered to the one in the `project` section.
The `project` section is used as a default section for the whole configuration file. - Tests still have to be written -
The standard configuration precedence is fixed, in this order from the hight to the lower :
- Argument value (i.e. : --log-level)
- Environment value (i.e. : HALFAPI_LOGLEVEL)
- Configuration value under "domain" key
- Configuration value under "project" key
- Default configuration value given in the "DEFAULT_CONF" dictionary of halfapi/conf.py
### Logs
Small cleanup of the logs levels. If you don't want the config to be dumped, just set the HALFAPI_LOGLEVEL to something different than "DEBUG".
### Fixes
- Check an ACL based on a decorator on "/halfapi/acls/MY_ACL"
## 0.6.27
### Breaking changes
- ACLs definition can now include a "public" parameter that defines if there should be an automatic creation of a route to check this acls
- /halfapi/acls does not return the "result", it just returns if there is a public route to check the ACL on /halfapi/acls/acl_name
=======
argument of starlette instead.
>>>>>>> a8c59c6 ([release] halfapi 0.6.27)
## 0.6.26
- Adds the "base_url", "cookies" and "url" to the "halfapi" argument of route definitions
## 0.6.25
- Deletes the "Authorization" cookie on authentication error
- Do not raise an exception on signature expiration, use "Nobody" user instead
## 0.6.24
- Uses the "Authorization" cookie to read authentication token additionnaly to the "Authorization" header
- CLI : allows to run a single domain using the "halfapi domain --run domain_name" command
## 0.6.23
Dependency update version
- starlette v0.23
- orjson v3.8.5
- click v8
- pyJWT v2.6
- pyYAML v6
- toml v0.10
## 0.6.22
- IMPORTANT : Fix bug introduced with 0.6.20 (fix arguments handling)
- BREAKING : A domain should now include it's meta datas in a "domain" dictionary located in the __init__.py file of the domain's root. Consider looking in 'tests/dummy_domain/__init__.py'
- Add *html* return type as default argument ret_type
- Add *txt* return type
- Log unhandled exceptions
- Log HTTPException with statuscode 500 as critical
- PyJWT >=2.4.0,<2.5.0
## 0.6.21
- Store only domain's config in halfapi['config']
- Should run halfapi domain with config_file argument
- Testing : You can specify a "MODULE" attribute to point out the path to the Api's base module
- Testing : You can specify a "CONFIG" attribute to set the domain's testing configuration
- Environment : HALFAPI_DOMAIN_MODULE can be set to specify Api's base module
- Config : 'module' attribute can be set to specify Api's base module
## 0.6.20
- Fix arguments handling
## 0.6.19
- Allow file sending in multipart request (#32)
- Add python-multipart dependency
## 0.6.18
- Fix config coming from .halfapi/config when using HALFAPI_DOMAIN_NAME environment variable
## 0.6.17
- Fix 0.6.16
- Errata : HALFAPI_DOMAIN is HALFAPI_DOMAIN_NAME
- Testing : You can now specify "MODULE" class attribute for your "test_domain"
subclasses
## 0.6.16
- The definition of "HALFAPI_DOMAIN_MODULE" environment variable allows to
specify the base module for a domain structure. It is formatted as a python
import path.
The "HALFAPI_DOMAIN" specifies the "name" of the module
## 0.6.15
- Allows to define a "__acl__" variable in the API module's __init__.py, to
specify how to import the acl lib. You can also specify "acl" in the domain's
config
## 0.6.14
- Add XLSXResponse (with format argument set to "xlsx"), to return .xlsx files
## 0.6.13
- (rollback from 0.6.12) Remove pytest from dependencies in Docker file and
remove tests
- (dep) Add "packaging" dependency
- Add dependency check capability when instantiating a domain (__deps__
variable, see in dummy_domain)
## 0.6.12
- Installs pytest with dependencies in Docker image, tests when building image
## 0.6.11
- Fix "request" has no "path_params" attribute bug
## 0.6.10
- Add "x-out" field in HTTP headers when "out" parameters are specified for a
route
- Add "out" kwarg for not-async functions that specify it
## 0.6.9
- Hide data fields in args_check logs
## 0.6.8
- Fix testing lib for domains (add default secret and debug option)
## 0.6.2
- Domains now need to include the following variables in their __init__.py
- __name__ (str, optional)
- __id__ (str, optional)
- halfapi domain
## 0.1.0
- Mounts domain routers with their ACLs as decorator

View File

@ -1,9 +1,11 @@
# syntax=docker/dockerfile:1
FROM docker.io/python:3.8.12-slim-bullseye
FROM python:alpine3.19
COPY . /halfapi
WORKDIR /halfapi
RUN apt-get update > /dev/null && apt-get -y install git > /dev/null
RUN pip install gunicorn uvicorn
RUN pip install .
CMD gunicorn halfapi.app
ENV VENV_DIR=/opt/venv
RUN mkdir -p $VENV_DIR
RUN python -m venv $VENV_DIR
RUN $VENV_DIR/bin/pip install gunicorn uvicorn
RUN $VENV_DIR/bin/pip install .
RUN ln -s $VENV_DIR/bin/halfapi /usr/local/bin/
CMD $VENV_DIR/bin/gunicorn halfapi.app

View File

@ -1,4 +1,5 @@
Copyright (c) 2020-2021 Maxime ALVES <maxime@freepoteries.fr>
Copyright (c) 2020-2021 Maxime ALVES <maxime@freepoteries.fr>, Joël Maïzi
<joel.maizi@collorg.org>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by

28
Pipfile
View File

@ -4,23 +4,33 @@ url = "https://pypi.org/simple"
verify_ssl = true
[dev-packages]
pytest = "*"
pytest = ">=7,<8"
requests = "*"
pytest-asyncio = "*"
pylint = "*"
build = "*"
twine = "*"
pyflakes = "*"
vulture = "*"
virtualenv = "*"
httpx = "*"
openapi-schema-validator = "*"
openapi-spec-validator = "*"
coverage = "*"
[packages]
click = ">=7.1,<8"
starlette = ">=0.16,<0.17"
click = ">=8,<9"
starlette = ">=0.37,<0.38"
uvicorn = ">=0.13,<1"
orjson = ">=3.4.7,<4"
pyjwt = ">=2.0.1,<3"
pyyaml = ">=5.3.1,<6"
orjson = ">=3.8.5,<4"
pyjwt = ">=2.6.0,<2.7.0"
pyyaml = ">=6.0.1,<7"
timing-asgi = ">=0.2.1,<1"
[requires]
python_version = "3.8"
schema = ">=0.7.4,<1"
toml = ">=0.10,<0.11"
pip = "*"
packaging = ">=19.0"
python-multipart = "*"
[scripts]
halfapi = "python -m halfapi"

1387
Pipfile.lock generated

File diff suppressed because it is too large Load Diff

View File

@ -1,6 +1,9 @@
# HalfAPI
Base tools to develop comlex API with rights management.
Base tools to develop complex API with rights management.
This project was developped by Maxime Alves and Joël Maïzi. The name was chosen
to reference [HalfORM](https://github.com/collorg/halfORM), a project written by Joël Maïzi.
## Dependencies
@ -20,17 +23,15 @@ Base tools to develop comlex API with rights management.
Configure HalfAPI in the file : .halfapi/config .
It's an **ini** file that contains at least two sections, project and domains.
It's a **toml** file that contains at least two sections, project and domains.
https://toml.io/en/
### Project
The main configuration options without which HalfAPI cannot be run.
**name** : Project's name
**halfapi_version** : The HalfAPI version on which you work
**secret** : The file containing the secret to decode the user's tokens.
**port** : The port for the test server.
@ -40,12 +41,28 @@ The main configuration options without which HalfAPI cannot be run.
### Domains
The name of the options should be the name of the domains' module, the value is the
submodule which contains the routers.
Specify the domains configurations in the following form :
Example :
```
[domains.DOMAIN_NAME]
name = "DOMAIN_NAME"
enabled = true
prefix = "/prefix"
module = "domain_name.path.to.api.root"
port = 1002
```
dummy_domain = .routers
Specific configuration can be done under the "config" section :
```
[domains.DOMAIN_NAME.config]
boolean_option = false
string_value = "String"
answer = 42
listylist = ["hello", "world"]
```
And can be accessed through the app's "config" dictionnary.
## Usage
@ -56,6 +73,20 @@ that is available in the python path.
Run the project by using the `halfapi run` command.
You can try the dummy_domain with the following command.
```
PYTHONPATH=$PWD/tests python -m halfapi domain dummy_domain
```
### CLI documentation
Use the CLI help.
```
python -m halfapi --help
python -m halfapi domain --help
```
## API Testing

View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
__version__ = '0.5.13'
__version__ = '0.6.30'
def version():
return f'HalfAPI version:{__version__}'

View File

@ -1,138 +1,7 @@
#!/usr/bin/env python3
"""
app.py is the file that is read when launching the application using an asgi
runner.
import os
from .halfapi import HalfAPI
from .logging import logger
It defines the following globals :
- routes (contains the Route objects for the application)
- application (the asgi application itself - a starlette object)
"""
import logging
import time
# asgi framework
from starlette.applications import Starlette
from starlette.authentication import UnauthenticatedUser
from starlette.middleware import Middleware
from starlette.routing import Route
from starlette.responses import Response, PlainTextResponse
from starlette.middleware.authentication import AuthenticationMiddleware
from timing_asgi import TimingMiddleware
from timing_asgi.integrations import StarletteScopeToName
# module libraries
from .lib.domain_middleware import DomainMiddleware
from .lib.timing import HTimingClient
from halfapi.lib.jwt_middleware import JWTAuthenticationBackend
from halfapi.lib.responses import (ORJSONResponse, UnauthorizedResponse,
NotFoundResponse, InternalServerErrorResponse, NotImplementedResponse,
ServiceUnavailableResponse)
from halfapi.lib.routes import gen_starlette_routes, debug_routes
from halfapi.lib.schemas import get_api_routes, get_api_domain_routes, schema_json, get_acls
from halfapi.logging import logger, config_logging
from halfapi import __version__
class HalfAPI:
def __init__(self, config=None):
config_logging(logging.DEBUG)
if config:
SECRET = config.get('SECRET')
PRODUCTION = config.get('PRODUCTION')
DOMAINS = config.get('DOMAINS', {})
CONFIG = config.get('CONFIG', {
'domains': DOMAINS
})
else:
from halfapi.conf import CONFIG, SECRET, PRODUCTION, DOMAINS
routes = [ Route('/', get_api_routes(DOMAINS)) ]
for route in self.routes():
routes.append(route)
if not PRODUCTION:
for route in debug_routes():
routes.append( route )
if DOMAINS:
for route in gen_starlette_routes(DOMAINS):
routes.append(route)
for domain, m_domain in DOMAINS.items():
routes.append(
Route(
f'/{domain}',
get_api_domain_routes(m_domain)
)
)
self.application = Starlette(
debug=not PRODUCTION,
routes=routes,
exception_handlers={
401: UnauthorizedResponse,
404: NotFoundResponse,
500: InternalServerErrorResponse,
501: NotImplementedResponse,
503: ServiceUnavailableResponse
}
)
self.application.add_middleware(
DomainMiddleware,
config=CONFIG
)
if SECRET:
self.SECRET = SECRET
self.application.add_middleware(
AuthenticationMiddleware,
backend=JWTAuthenticationBackend(secret_key=SECRET)
)
if not PRODUCTION:
self.application.add_middleware(
TimingMiddleware,
client=HTimingClient(),
metric_namer=StarletteScopeToName(prefix="halfapi",
starlette_app=self.application)
)
logger.info('CONFIG:\n%s', CONFIG)
@property
def version(self):
return __version__
async def version_async(self, request, *args, **kwargs):
return Response(self.version)
def routes(self):
""" Halfapi default routes
"""
async def get_user(request, *args, **kwargs):
return ORJSONResponse({'user':request.user})
yield Route('/halfapi/whoami', get_user)
yield Route('/halfapi/schema', schema_json)
yield Route('/halfapi/acls', get_acls)
yield Route('/halfapi/version', self.version_async)
application = HalfAPI().application
def application():
from .conf import CONFIG
return HalfAPI(CONFIG).application

View File

@ -8,12 +8,6 @@ not loaded otherwise.
# builtins
import click
IS_PROJECT = True
try:
from ..conf import DOMAINS
except Exception:
IS_PROJECT = False
@click.group(invoke_without_command=True)
@click.option('--version', is_flag=True)
@click.pass_context
@ -27,10 +21,8 @@ def cli(ctx, version):
from halfapi import version
click.echo(version())
if IS_PROJECT:
from . import config
from . import domain
from . import run
else:
from . import init
from . import config
from . import domain
from . import run
from . import init
from . import routes

View File

@ -1,32 +1,26 @@
#!/usr/bin/env python3
"""
cli/config.py Contains the .halfapi/config and HALFAPI_CONF_DIR/project_name templates
cli/config.py Contains the .halfapi/config
Defines the "halfapi config" command
"""
import click
from .cli import cli
from ..conf import (
read_config,
PROJECT_NAME,
DOMAINSDICT,
HOST,
PORT,
PRODUCTION,
)
from ..conf import CONFIG
CONF_STR=f"""
DOMAIN_CONF_STR="""
[domain]
name = {name}
router = {router}
"""
CONF_STR="""
[project]
name = {PROJECT_NAME}
host = {HOST}
port = {PORT}
production = {PRODUCTION}
[domains]"""
for dom in DOMAINSDICT():
CONF_STR = '\n'.join((CONF_STR, dom))
host = {host}
port = {port}
production = {production}
"""
@cli.command()
@ -34,4 +28,4 @@ def config():
"""
Lists config parameters and their values
"""
click.echo(read_config())
click.echo(CONF_STR.format(**CONFIG))

View File

@ -3,20 +3,30 @@
cli/domain.py Defines the "halfapi domain" cli commands
"""
# builtins
import logging
import os
import sys
import importlib
import subprocess
import json
import toml
import click
import orjson
import uvicorn
from .cli import cli
from ..conf import config, write_config, DOMAINSDICT
from ..conf import CONFIG
from ..half_domain import HalfDomain
from ..lib.schemas import schema_dict_dom
from ..lib.routes import api_routes
from ..lib.responses import ORJSONResponse
from ..conf import CONFIG, PROJECT_LEVEL_KEYS
logger = logging.getLogger('halfapi')
from ..logging import logger
#################
# domain create #
@ -29,11 +39,53 @@ def create_domain(domain_name: str, module_path: str):
# logger.warning('Domain **%s** is already in project', domain_name)
# sys.exit(1)
if not config.has_section('domains'):
config.add_section('domains')
def domain_tree_create():
def create_init(path):
with open(os.path.join(os.getcwd(), path, '__init__.py'), 'w') as f:
f.writelines([
'"""',
f'name: {domain_name}',
f'router: {module_path}',
'"""'
])
logger.debug('created %s', os.path.join(os.getcwd(), path, '__init__.py'))
def create_acl(path):
with open(os.path.join(path, 'acl.py'), 'w') as f:
f.writelines([
'from halfapi.lib.acl import public, ACLS',
])
logger.debug('created %s', os.path.join(path, 'acl.py'))
os.mkdir(domain_name)
create_init(domain_name)
router_path = os.path.join(domain_name, 'routers')
create_acl(domain_name)
os.mkdir(router_path)
create_init(router_path)
# TODO: Generate config file
domain_tree_create()
"""
try:
importlib.import_module(module_path)
except ImportError:
logger.error('cannot import %s', domain_name)
domain_tree_create()
"""
"""
try:
importlib.import_module(domain_name)
except ImportError:
click.echo('Error in domain creation')
logger.debug('%s', subprocess.run(['tree', 'a', os.getcwd()]))
raise Exception('cannot create domain {}'.format(domain_name))
"""
config.set('domains', domain_name, module_path)
write_config()
###############
@ -45,68 +97,146 @@ def list_routes(domain, m_dom):
"""
click.echo(f'\nDomain : {domain}\n')
routes = api_routes(m_dom)[0]
if len(routes):
for key, item in routes.items():
methods = '|'.join(list(item.keys()))
click.echo(f'\t{key} : {methods}')
else:
click.echo(f'\t**No ROUTES**')
click.echo('\t**No ROUTES**')
raise Exception('Routeless domain')
def list_api_routes():
"""
Echoes the list of all active domains.
TODO: Rewrite function
"""
click.echo('# API Routes')
for domain, m_dom in DOMAINSDICT().items():
list_routes(domain, m_dom)
# for domain, m_dom in DOMAINSDICT().items():
# list_routes(domain, m_dom)
@click.option('--devel',default=None, is_flag=True)
@click.option('--watch',default=False, is_flag=True)
@click.option('--production',default=None, is_flag=True)
@click.option('--port',default=None, type=int)
@click.option('--log-level',default=None, type=str)
@click.option('--dry-run',default=False, is_flag=True)
@click.option('--run',default=False, is_flag=True)
@click.option('--read',default=False, is_flag=True)
@click.option('--conftest',default=False, is_flag=True)
@click.option('--create',default=False, is_flag=True)
@click.option('--update',default=False, is_flag=True)
@click.option('--delete',default=False, is_flag=True)
@click.option('--domains',default=None)
@click.argument('config_file', type=click.File(mode='rb'), required=False)
@click.argument('domain',default=None, required=False)
@cli.command()
def domain(domains, delete, update, create, read): #, domains, read, create, update, delete):
def domain(domain, config_file, delete, update, create, conftest, read, run, dry_run, log_level, port, production, watch, devel):
"""
The "halfapi domain" command
Parameters:
domain (List[str]|None): The list of the domains to list/update
domain (str|None): The domain name
The parameter has a misleading name as it is a multiple option
but this would be strange to use it several times named as "domains"
update (boolean): If set, update the database for the selected domains
"""
if not domains:
if not domain:
if create:
# TODO: Connect to the create_domain function
raise NotImplementedError
raise Exception('Missing domain name')
if config_file:
ARG_CONFIG = toml.load(config_file.name)
if 'project' in ARG_CONFIG:
for key, value in ARG_CONFIG['project'].items():
if key in PROJECT_LEVEL_KEYS:
CONFIG[key] = value
if 'domain' in ARG_CONFIG and domain in ARG_CONFIG['domain']:
for key, value in ARG_CONFIG['domain'][domain].items():
if key in PROJECT_LEVEL_KEYS:
CONFIG[key] = value
CONFIG['domain'].update(ARG_CONFIG['domain'])
if create:
raise NotImplementedError
elif update:
raise NotImplementedError
elif delete:
raise NotImplementedError
elif read:
from ..halfapi import HalfAPI
halfapi = HalfAPI(CONFIG)
click.echo(orjson.dumps(
halfapi.domains[domain].schema(),
option=orjson.OPT_NON_STR_KEYS,
default=ORJSONResponse.default_cast)
)
domains = DOMAINSDICT().keys()
else:
domains_ = []
for domain_name in domains.split(','):
if domain_name in DOMAINSDICT().keys():
domains_.append(domain_name)
continue
if dry_run:
CONFIG['dryrun'] = True
domains = CONFIG.get('domain')
for key in domains.keys():
if key != domain:
domains[key]['enabled'] = False
else:
domains[key]['enabled'] = True
if not log_level:
log_level = CONFIG.get('domain', {}).get('loglevel', CONFIG.get('loglevel', False))
else:
CONFIG['loglevel'] = log_level
if not port:
port = CONFIG.get('domain', {}).get('port', CONFIG.get('port', False))
else:
CONFIG['port'] = port
if devel is None and production is not None and (production is False or production is True):
CONFIG['production'] = production
if devel is not None:
CONFIG['production'] = False
CONFIG['loglevel'] = 'debug'
if conftest:
click.echo(
f'Domain {domain_name}s is not activated in the configuration')
toml.dumps(CONFIG)
)
domains = domains_
else:
# domain section port is preferred, if it doesn't exist we use the global one
for domain_name in domains:
if update:
raise NotImplementedError
if delete:
raise NotImplementedError
uvicorn_kwargs = {}
if read:
list_api_routes()
if CONFIG.get('port'):
uvicorn_kwargs['port'] = CONFIG['port']
if CONFIG.get('loglevel'):
uvicorn_kwargs['log_level'] = CONFIG['loglevel'].lower()
if watch:
uvicorn_kwargs['reload'] = True
uvicorn.run(
'halfapi.app:application',
factory=True,
**uvicorn_kwargs
)
sys.exit(0)

View File

@ -5,7 +5,6 @@ cli/init.py Defines the "halfapi init" cli commands
Helps the user to create a new project
"""
# builtins
import logging
import os
import sys
import re
@ -17,10 +16,9 @@ from ..conf import CONF_DIR
from .cli import cli
logger = logging.getLogger('halfapi')
from ..logging import logger
TMPL_HALFAPI_ETC = """[project]
name = {project}
host = 127.0.0.1
port = 8000
secret = /path/to/secret_file
@ -28,20 +26,10 @@ production = False
base_dir = {base_dir}
"""
def format_halfapi_etc(project, path):
"""
Returns the formatted template for /etc/half_api/PROJECT_NAME
"""
return TMPL_HALFAPI_ETC.format(
project=project,
base_dir=path
)
TMPL_HALFAPI_CONFIG = """[project]
name = {name}
halfapi_version = {halfapi_version}
[domains]
[domain]
"""
@click.argument('project')
@ -67,9 +55,7 @@ def init(project):
with open(f'{project}/.halfapi/config', 'w') as conf_file:
conf_file.write(TMPL_HALFAPI_CONFIG.format(
name=project,
halfapi_version=__version__))
click.echo(f'Configure halfapi project in {CONF_DIR}/{project}')
click.echo(format_halfapi_etc(project, CONF_DIR))

54
halfapi/cli/routes.py Normal file
View File

@ -0,0 +1,54 @@
#!/usr/bin/env python3
"""
cli/routes.py Defines the "halfapi routes" cli commands
"""
# builtins
import sys
import importlib
from pprint import pprint
import orjson
import click
from .cli import cli
from ..logging import logger
# from ..lib.domain import domain_schema_dict
from ..lib.constants import DOMAIN_SCHEMA, ROUTE_SCHEMA
from ..lib.responses import ORJSONResponse
@click.argument('module', required=True)
@click.option('--export', default=False, is_flag=True)
@click.option('--validate', default=False, is_flag=True)
@click.option('--check', default=False, is_flag=True)
@click.option('--noheader', default=False, is_flag=True)
@click.option('--schema', default=False, is_flag=True)
@cli.command()
def routes(module, export=False, validate=False, check=False, noheader=False, schema=False):
"""
The "halfapi routes" command
"""
# try:
#  mod = importlib.import_module(module)
# except ImportError as exc:
#  raise click.BadParameter('Cannot import this module', param=module) from exc
# if export:
#  click.echo(schema_to_csv(module, header=not noheader))
# if schema:
#  routes_d = domain_schema_dict(mod)
#  ROUTE_SCHEMA.validate(routes_d)
#  click.echo(orjson.dumps(routes_d,
#  option=orjson.OPT_NON_STR_KEYS,
#  default=ORJSONResponse.default_cast))
# if validate:
#  routes_d = domain_schema_dict(mod)
#  try:
#  ROUTE_SCHEMA.validate(routes_d)
#  except Exception as exc:
#  raise exc

View File

@ -9,47 +9,79 @@ import uvicorn
from .cli import cli
from .domain import list_api_routes
from ..conf import (PROJECT_NAME, HOST, PORT,
PRODUCTION, LOGLEVEL, DOMAINSDICT)
from ..conf import CONFIG, SCHEMA
from ..logging import logger
from ..lib.schemas import schema_csv_dict
from ..half_domain import HalfDomain
@click.option('--host', default=None)
@click.option('--port', default=None)
@click.option('--host', default=CONFIG.get('host'))
@click.option('--port', default=CONFIG.get('port'))
@click.option('--reload', default=False)
@click.option('--secret', default=CONFIG.get('secret'))
@click.option('--production', default=CONFIG.get('secret'))
@click.option('--loglevel', default=CONFIG.get('loglevel'))
@click.option('--prefix', default='/')
@click.option('--check', default=True)
@click.option('--dryrun', default=False, is_flag=True)
@click.argument('schema', type=click.File('r'), required=False)
@click.argument('domain', required=False)
@cli.command()
def run(host, port, reload):
def run(host, port, reload, secret, production, loglevel, prefix, check, dryrun,
schema, domain):
"""
The "halfapi run" command
"""
if not host:
host = HOST
if not port:
port = PORT
logger.debug('[run] host=%s port=%s reload=%s secret=%s production=%s loglevel=%s prefix=%s schema=%s',
host, port, reload, secret, production, loglevel, prefix, schema
)
port = int(port)
if PRODUCTION and reload:
if production and reload:
reload = False
raise Exception('Can\'t use live code reload in production')
log_level = LOGLEVEL or 'info'
click.echo(f'Launching application')
click.echo(f'Launching application {PROJECT_NAME}')
if secret:
CONFIG['secret'] = secret
sys.path.insert(0, os.getcwd())
if schema:
# Populate the SCHEMA global with the data from the given file
for key, val in schema_csv_dict(schema, prefix).items():
SCHEMA[key] = val
list_api_routes()
if domain:
# If we specify a domain to run as argument
if 'domain' not in CONFIG:
CONFIG['domain'] = {}
# Disable all domains
keys = list(CONFIG.get('domain').keys())
for key in keys:
CONFIG['domain'].pop(key)
# And activate the desired one, mounted without prefix
CONFIG['domain'][domain] = {
'name': domain,
'prefix': False,
'enabled': True
}
# list_api_routes()
click.echo(f'uvicorn.run("halfapi.app:application"\n' \
f'host: {host}\n' \
f'port: {port}\n' \
f'log_level: {log_level}\n' \
f'log_level: {loglevel}\n' \
f'reload: {reload}\n'
)
if dryrun:
CONFIG['dryrun'] = True
uvicorn.run('halfapi.app:application',
host=host,
port=int(port),
log_level=log_level,
log_level=loglevel,
reload=reload)

View File

@ -10,15 +10,13 @@ It uses the following environment variables :
It defines the following globals :
- PROJECT_NAME (str) - HALFAPI_PROJECT_NAME
- DOMAINSDICT ({domain_name: domain_module}) - HALFAPI_DOMAIN_NAME / HALFAPI_DOMAIN_MODULE
- PRODUCTION (bool) - HALFAPI_PRODUCTION
- LOGLEVEL (string) - HALFAPI_LOGLEVEL
- LOGLEVEL (str) - HALFAPI_LOGLEVEL
- BASE_DIR (str) - HALFAPI_BASE_DIR
- HOST (str) - HALFAPI_HOST
- PORT (int) - HALFAPI_PORT
- CONF_DIR (str) - HALFAPI_CONF_DIR
- IS_PROJECT (bool)
- config (ConfigParser)
- DRYRUN (bool) - HALFAPI_DRYRUN
It reads the following ressource :
@ -27,130 +25,181 @@ It reads the following ressource :
It follows the following format :
[project]
name = PROJECT_NAME
halfapi_version = HALFAPI_VERSION
[domains]
domain_name = requirements-like-url
[domain.domain_name]
name = domain_name
routers = routers
[domain.domain_name.config]
option = Argh
"""
import logging
from .logging import logger
import os
from os import environ
import sys
from configparser import ConfigParser
import importlib
import tempfile
import uuid
from .lib.domain import d_domains
from .logging import logger
import toml
SCHEMA = {}
PROJECT_NAME = environ.get('HALFAPI_PROJECT_NAME') or os.path.basename(os.getcwd())
DOMAINSDICT = lambda: {}
DOMAINS = {}
PRODUCTION = False
LOGLEVEL = 'info'
HOST = '127.0.0.1'
PORT = '3000'
SECRET = ''
CONF_FILE = os.environ.get('HALFAPI_CONF_FILE', '.halfapi/config')
DEFAULT_CONF = {
# Default configuration values
'SECRET': tempfile.mkstemp()[1],
'PROJECT_NAME': os.getcwd().split('/')[-1],
'PRODUCTION': True,
'HOST': '127.0.0.1',
'PORT': 3000,
'LOGLEVEL': 'info',
'BASE_DIR': os.getcwd(),
'CONF_FILE': '.halfapi/config',
'CONF_DIR': '/etc/half_api',
'DRYRUN': None
}
is_project = lambda: os.path.isfile(CONF_FILE)
PROJECT_LEVEL_KEYS = {
# Allowed keys in "project" section of configuration file
'project_name',
'production',
'secret',
'host',
'port',
'loglevel',
'dryrun'
}
DOMAIN_LEVEL_KEYS = PROJECT_LEVEL_KEYS | {
# Allowed keys in "domain" section of configuration file
'name',
'module',
'prefix',
'enabled'
}
CONF_FILE = os.environ.get('HALFAPI_CONF_FILE', DEFAULT_CONF['CONF_FILE'])
CONF_DIR = os.environ.get('HALFAPI_CONF_DIR', DEFAULT_CONF['CONF_DIR'])
config = ConfigParser(allow_no_value=True)
CONF_DIR = environ.get('HALFAPI_CONF_DIR', '/etc/half_api')
HALFAPI_ETC_FILE=os.path.join(
CONF_DIR, 'default.ini'
CONF_DIR, 'config'
)
BASE_DIR = os.environ.get('HALFAPI_BASE_DIR', DEFAULT_CONF['BASE_DIR'])
HALFAPI_DOT_FILE=os.path.join(
os.getcwd(), '.halfapi', 'config')
BASE_DIR, '.halfapi', 'config')
HALFAPI_CONFIG_FILES = [ CONF_FILE, HALFAPI_DOT_FILE ]
HALFAPI_CONFIG_FILES = []
def conf_files():
return [
os.path.join(
CONF_DIR, 'default.ini'
),
os.path.join(
os.getcwd(), '.halfapi', 'config')]
try:
with open(HALFAPI_ETC_FILE, 'r'):
HALFAPI_CONFIG_FILES.append(HALFAPI_ETC_FILE)
except FileNotFoundError:
logger.info('Cannot find a configuration file under %s', HALFAPI_ETC_FILE)
try:
with open(HALFAPI_DOT_FILE, 'r'):
HALFAPI_CONFIG_FILES.append(HALFAPI_DOT_FILE)
except FileNotFoundError:
logger.info('Cannot find a configuration file under %s', HALFAPI_DOT_FILE)
def write_config():
"""
Writes the current config to the highest priority config file
"""
with open(conf_files()[-1], 'w') as halfapi_config:
config.write(halfapi_config)
ENVIRONMENT = {}
# Load environment variables allowed in configuration
def config_dict():
"""
The config object as a dict
"""
return {
section: dict(config.items(section))
for section in config.sections()
}
if 'HALFAPI_DRYRUN' in os.environ:
ENVIRONMENT['dryrun'] = True
def read_config():
if 'HALFAPI_PROD' in os.environ:
ENVIRONMENT['production'] = bool(os.environ.get('HALFAPI_PROD'))
if 'HALFAPI_LOGLEVEL' in os.environ:
ENVIRONMENT['loglevel'] = os.environ.get('HALFAPI_LOGLEVEL').lower()
if 'HALFAPI_SECRET' in os.environ:
ENVIRONMENT['secret'] = os.environ.get('HALFAPI_SECRET')
if 'HALFAPI_HOST' in os.environ:
ENVIRONMENT['host'] = os.environ.get('HALFAPI_HOST')
if 'HALFAPI_PORT' in os.environ:
ENVIRONMENT['port'] = int(os.environ.get('HALFAPI_PORT'))
def read_config(filenames=HALFAPI_CONFIG_FILES):
"""
The highest index in "filenames" are the highest priorty
"""
config.read(HALFAPI_CONFIG_FILES)
d_res = {}
logger.info('Reading config files %s', filenames)
for CONF_FILE in filenames:
if os.path.isfile(CONF_FILE):
conf_dict = toml.load(CONF_FILE)
d_res.update(conf_dict)
logger.info('Read config files (result) %s', d_res)
return { **d_res.get('project', {}), 'domain': d_res.get('domain', {}) }
CONFIG = read_config()
CONFIG.update(**ENVIRONMENT)
PROJECT_NAME = CONFIG.get('project_name',
os.environ.get('HALFAPI_PROJECT_NAME', DEFAULT_CONF['PROJECT_NAME']))
if os.environ.get('HALFAPI_DOMAIN_NAME'):
# Force enabled domain by environment variable
DOMAIN_NAME = os.environ.get('HALFAPI_DOMAIN_NAME')
if 'domain' in CONFIG and DOMAIN_NAME in CONFIG['domain'] \
and 'config' in CONFIG['domain'][DOMAIN_NAME]:
domain_config = CONFIG['domain'][DOMAIN_NAME]['config']
else:
domain_config = {}
CONFIG['domain'] = {}
CONFIG['domain'][DOMAIN_NAME] = {
'enabled': True,
'name': DOMAIN_NAME,
'prefix': False
}
CONFIG['domain'][DOMAIN_NAME]['config'] = domain_config
if os.environ.get('HALFAPI_DOMAIN_MODULE'):
# Specify the pythonpath to import the specified domain (defaults to global)
dom_module = os.environ.get('HALFAPI_DOMAIN_MODULE')
CONFIG['domain'][DOMAIN_NAME]['module'] = dom_module
if len(CONFIG.get('domain', {}).keys()) == 0:
logger.info('No domains')
CONFIG = {}
read_config()
IS_PROJECT = True
PROJECT_NAME = config.get('project', 'name', fallback=PROJECT_NAME)
if len(PROJECT_NAME) == 0:
raise Exception('Need a project name as argument')
DOMAINSDICT = lambda: d_domains(config)
DOMAINS = DOMAINSDICT()
if len(DOMAINS) == 0:
logger.info('Domain-less instance %s', d_domains(config))
HOST = config.get('project', 'host', fallback=environ.get('HALFAPI_HOST', '127.0.0.1'))
PORT = config.getint('project', 'port', fallback=environ.get('HALFAPI_PORT', '3000'))
# Secret
if 'secret' not in CONFIG:
# TODO: Create a temporary secret
CONFIG['secret'] = DEFAULT_CONF['SECRET']
with open(CONFIG['secret'], 'w') as secret_file:
secret_file.write(str(uuid.uuid4()))
try:
with open(config.get('project', 'secret',
fallback=environ.get('HALFAPI_SECRET', ''))) as secret_file:
SECRET = secret_file.read().strip()
CONFIG['secret'] = SECRET.strip()
with open(CONFIG['secret'], 'r') as secret_file:
CONFIG['secret'] = CONFIG['secret'].strip()
except FileNotFoundError as exc:
logger.error('Missing secret file: %s', exc)
logger.warning('Running without secret file: %s', CONFIG['secret'] or 'no file specified')
PRODUCTION = config.getboolean('project', 'production',
fallback=environ.get('HALFAPI_PROD', False))
CONFIG.setdefault('project_name', DEFAULT_CONF['PROJECT_NAME'])
CONFIG.setdefault('production', DEFAULT_CONF['PRODUCTION'])
CONFIG.setdefault('host', DEFAULT_CONF['HOST'])
CONFIG.setdefault('port', DEFAULT_CONF['PORT'])
CONFIG.setdefault('loglevel', DEFAULT_CONF['LOGLEVEL'])
CONFIG.setdefault('dryrun', DEFAULT_CONF['DRYRUN'])
LOGLEVEL = config.get('project', 'loglevel',
fallback=environ.get('HALFAPI_LOGLEVEL', 'info')).lower()
BASE_DIR = config.get('project', 'base_dir',
fallback=environ.get('HALFAPI_BASE_DIR', '.'))
CONFIG = {
'project_name': PROJECT_NAME,
'production': PRODUCTION,
'secret': SECRET,
'domains': DOMAINS,
'domain_config': {}
}
for domain in DOMAINS:
if domain not in config.sections():
continue
CONFIG['domain_config'][domain] = dict(config.items(domain))
# !!!TO REMOVE!!!
SECRET = CONFIG.get('secret')
PRODUCTION = CONFIG.get('production')
# !!!

513
halfapi/half_domain.py Normal file
View File

@ -0,0 +1,513 @@
import importlib
import inspect
import os
import re
from packaging.specifiers import SpecifierSet
from packaging.version import Version
from typing import Coroutine, Dict, Iterator, List, Tuple
from types import ModuleType, FunctionType
from schema import SchemaError
from starlette.applications import Starlette
from starlette.middleware import Middleware
from starlette.routing import Router, Route
from starlette.schemas import SchemaGenerator
from .lib.acl import AclRoute
from .lib.responses import ORJSONResponse
import yaml
from . import __version__
from .lib.constants import API_SCHEMA_DICT, ROUTER_SCHEMA, VERBS
from .half_route import HalfRoute
from .lib import acl as lib_acl
from .lib.responses import PlainTextResponse
from .lib.routes import JSONRoute
from .lib.schemas import param_docstring_default
from .lib.domain import MissingAclError, PathError, UnknownPathParameterType, \
UndefinedRoute, UndefinedFunction, get_fct_name, route_decorator
from .lib.domain_middleware import DomainMiddleware
from .logging import logger
class HalfDomain(Starlette):
def __init__(self, domain, module=None, router=None, acl=None, app=None):
"""
Parameters:
domain (str): Module name (should be importable)
router (str): Router name (should be importable from domain module
defaults to __router__ variable from domain module)
app (HalfAPI): The app instance
"""
self.app = app
self.m_domain = importlib.import_module(domain) if module is None else module
self.d_domain = getattr(self.m_domain, 'domain', domain)
self.name = self.d_domain['name']
self.id = self.d_domain['id']
self.version = self.d_domain['version']
self.halfapi_version = self.d_domain.get('halfapi_version', __version__)
self.deps = self.d_domain.get('deps', tuple())
self.schema_components = self.d_domain.get('schema_components', dict())
if not router:
self.router = self.d_domain.get('routers', '.routers')
else:
self.router = router
self.m_router = None
try:
self.m_router = importlib.import_module(self.router, self.m_domain.__package__)
except AttributeError:
raise Exception('no router module')
self.m_acl = HalfDomain.m_acl(self.m_domain, acl)
self.config = { **app.config }
logger.info('HalfDomain creation %s %s', domain, self.config)
for elt in self.deps:
package, version = elt
specifier = SpecifierSet(version)
package_module = importlib.import_module(package)
if Version(package_module.__version__) not in specifier:
raise Exception(
'Wrong version for package {} version {} (excepting {})'.format(
package, package_module.__version__, specifier
))
super().__init__(
routes=self.gen_domain_routes(),
middleware=[
Middleware(
DomainMiddleware,
domain={
'name': self.name,
'id': self.id,
'version': self.version,
'halfapi_version': self.halfapi_version,
'config': self.config.get('domain', {}).get(self.name, {}).get('config', {})
}
)
]
)
@staticmethod
def name(module):
""" Returns the name declared in the 'domain' dict at the root of the package
"""
return module.domain['name']
@staticmethod
def m_acl(module, acl=None):
""" Returns the imported acl module for the domain module
"""
if not acl:
acl = getattr(module, '__acl__', '.acl')
return importlib.import_module(acl, module.__package__)
@staticmethod
def acls(module, acl=None):
""" Returns the ACLS constant for the given domain
"""
m_acl = HalfDomain.m_acl(module, acl)
try:
return [
lib_acl.ACL(*elt)
for elt in getattr(m_acl, 'ACLS')
]
except AttributeError as exc:
logger.error(exc)
raise Exception(
f'Missing acl.ACLS constant in module {m_acl.__package__}') from exc
@staticmethod
def acls_route(domain, module_path=None, acl=None):
""" Dictionary of acls
Format :
{
[acl_name]: {
callable: fct_reference,
docs: fct_docstring,
}
}
"""
d_res = {}
module = importlib.import_module(domain) \
if module_path is None \
else importlib.import_module(module_path)
m_acl = HalfDomain.m_acl(module, acl)
for elt in HalfDomain.acls(module, acl=acl):
fct = getattr(m_acl, elt.name)
d_res[elt.name] = {
'callable': fct,
'docs': elt.documentation
}
return d_res
@staticmethod
def acls_router(domain, module_path=None, acl=None):
""" Returns a Router object with the following routes :
/ : The "acls" field of the API metadatas
/{acl_name} : If the ACL is defined as public, a route that returns either status code 200 or 401 on HEAD/GET request
"""
routes = []
d_res = {}
module = importlib.import_module(domain) \
if module_path is None \
else importlib.import_module(module_path)
m_acl = HalfDomain.m_acl(module, acl)
for elt in HalfDomain.acls(module, acl=acl):
fct = getattr(m_acl, elt.name)
d_res[elt.name] = {
'callable': fct,
'docs': elt.documentation,
'public': elt.public
}
if elt.public:
try:
if inspect.iscoroutinefunction(fct):
logger.warning('async decorator are not yet supported')
else:
inner = fct()
if inspect.iscoroutinefunction(fct) or callable(inner):
fct = inner
except TypeError:
# Fct is not a decorator or is not well called (has no default arguments)
# We can ignore this
pass
routes.append(
AclRoute(f'/{elt.name}', fct, elt)
)
d_res_under_domain_name = {}
d_res_under_domain_name[HalfDomain.name(module)] = d_res
routes.append(
Route(
'/',
JSONRoute(d_res_under_domain_name),
methods=['GET']
)
)
return Router(routes)
@staticmethod
def gen_routes(m_router: ModuleType,
verb: str,
path: List[str],
params: List[Dict],
path_param_docstrings: Dict[str, str] = {}) -> Tuple[FunctionType, Dict]:
"""
Returns a tuple of the function associatied to the verb and path arguments,
and the dictionary of it's acls
Parameters:
- m_router (ModuleType): The module containing the function definition
- verb (str): The HTTP verb for the route (GET, POST, ...)
- path (List): The route path, as a list (each item being a level of
deepness), from the lowest level (domain) to the highest
- params (Dict): The acl list of the following format :
[{'acl': Function, 'args': {'required': [], 'optional': []}}]
Returns:
(Function, Dict): The destination function and the acl dictionary
"""
if len(params) == 0:
raise MissingAclError('[{}] {}'.format(verb, '/'.join(path)))
if len(path) == 0:
logger.error('Empty path for [{%s}]', verb)
raise PathError()
fct_name = get_fct_name(verb, path[-1])
if hasattr(m_router, fct_name):
fct = getattr(m_router, fct_name)
fct_docstring_obj = yaml.safe_load(fct.__doc__)
if 'parameters' not in fct_docstring_obj and path_param_docstrings:
fct_docstring_obj['parameters'] = list(map(
yaml.safe_load,
path_param_docstrings.values()))
fct.__doc__ = yaml.dump(fct_docstring_obj)
else:
raise UndefinedFunction('{}.{}'.format(m_router.__name__, fct_name or ''))
if not inspect.iscoroutinefunction(fct):
return route_decorator(fct), params
# TODO: Remove when using only sync functions
return lib_acl.args_check(fct), params
@staticmethod
def gen_router_routes(m_router, path: List[str], PATH_PARAMS={}) -> \
Iterator[Tuple[str, str, ModuleType, Coroutine, List]]:
"""
Recursive generator that parses a router (or a subrouter)
and yields from gen_routes
Parameters:
- m_router (ModuleType): The currently treated router module
- path (List[str]): The current path stack
Yields:
(str, str, ModuleType, Coroutine, List): A tuple containing the path, verb,
router module, function reference and parameters of the route.
Function and parameters are yielded from then gen_routes function,
that decorates the endpoint function.
"""
for subpath, params in HalfDomain.read_router(m_router).items():
path.append(subpath)
for verb in VERBS:
if verb not in params:
continue
yield ('/'.join(filter(lambda x: len(x) > 0, path)),
verb,
m_router,
*HalfDomain.gen_routes(m_router, verb, path, params[verb], PATH_PARAMS)
)
for subroute in params.get('SUBROUTES', []):
subroute_module = importlib.import_module(f'.{subroute}', m_router.__name__)
param_match = re.fullmatch('^([A-Z_]+)_([a-z]+)$', subroute)
parameter_name = None
if param_match is not None:
try:
parameter_name = param_match.groups()[0].lower()
if parameter_name in PATH_PARAMS:
raise Exception(f'Duplicate parameter name in same path! {subroute} : {parameter_name}')
parameter_type = param_match.groups()[1]
path.append('{{{}:{}}}'.format(
parameter_name,
parameter_type,
)
)
try:
PATH_PARAMS[parameter_name] = subroute_module.param_docstring
except AttributeError as exc:
PATH_PARAMS[parameter_name] = param_docstring_default(parameter_name, parameter_type)
except AssertionError as exc:
raise UnknownPathParameterType(subroute) from exc
else:
path.append(subroute)
try:
yield from HalfDomain.gen_router_routes(
subroute_module,
path,
PATH_PARAMS
)
except ImportError as exc:
logger.error('Failed to import subroute **{%s}**', subroute)
raise exc
path.pop()
if parameter_name:
PATH_PARAMS.pop(parameter_name)
path.pop()
@staticmethod
def read_router(m_router: ModuleType) -> Dict:
"""
Reads a module and returns a router dict
If the module has a "ROUTES" constant, it just returns this constant,
Else, if the module has an "ACLS" constant, it builds the accurate dict
TODO: May be another thing, may be not a part of halfAPI
"""
m_path = None
try:
if not hasattr(m_router, 'ROUTES'):
routes = {'':{}}
acls = getattr(m_router, 'ACLS') if hasattr(m_router, 'ACLS') else None
if acls is not None:
for method in acls.keys():
if method not in VERBS:
raise Exception(
'This method is not handled: {}'.format(method))
routes[''][method] = []
routes[''][method] = acls[method].copy()
routes['']['SUBROUTES'] = []
if hasattr(m_router, '__path__'):
""" Module is a package
"""
m_path = getattr(m_router, '__path__')
if isinstance(m_path, list) and len(m_path) == 1:
routes['']['SUBROUTES'] = [
elt.name
for elt in os.scandir(m_path[0])
if elt.is_dir()
]
else:
routes = getattr(m_router, 'ROUTES')
try:
ROUTER_SCHEMA.validate(routes)
except SchemaError as exc:
logger.error(routes)
raise exc
return routes
except ImportError as exc:
# TODO: Proper exception handling
raise exc
except FileNotFoundError as exc:
# TODO: Proper exception handling
logger.error(m_path)
raise exc
def gen_domain_routes(self):
"""
Yields the Route objects for a domain
Parameters:
m_domains: ModuleType
Returns:
Generator(HalfRoute)
"""
yield HalfRoute('/',
self.schema_openapi(),
[{'acl': lib_acl.public}],
'GET'
)
for path, method, m_router, fct, params in HalfDomain.gen_router_routes(self.m_router, []):
yield HalfRoute(f'/{path}', fct, params, method)
def schema_dict(self) -> Dict:
""" gen_router_routes return values as a dict
Parameters:
m_router (ModuleType): The domain routers' module
Returns:
Dict: Schema of dict is halfapi.lib.constants.DOMAIN_SCHEMA
@TODO: Should be a "router_schema_dict" function
"""
d_res = {}
for path, verb, m_router, fct, parameters in HalfDomain.gen_router_routes(self.m_router, []):
if path not in d_res:
d_res[path] = {}
if verb not in d_res[path]:
d_res[path][verb] = {}
d_res[path][verb]['callable'] = f'{m_router.__name__}:{fct.__name__}'
try:
d_res[path][verb]['docs'] = yaml.safe_load(fct.__doc__)
except AttributeError:
logger.error(
'Cannot read docstring from fct (fct=%s path=%s verb=%s', fct.__name__, path, verb)
d_res[path][verb]['acls'] = list(map(lambda elt: { **elt, 'acl': elt['acl'].__name__ },
parameters))
return d_res
def schema(self) -> Dict:
schema = { **API_SCHEMA_DICT }
schema['domain'] = {
'name': self.name,
'id': self.id,
'version': getattr(self.m_domain, '__version__', ''),
'patch_release': getattr(self.m_domain, '__patch_release__', ''),
'routers': self.m_router.__name__,
'acls': tuple(getattr(self.m_acl, 'ACLS', ()))
}
schema['paths'] = self.schema_dict()
return schema
def schema_openapi(self) -> Route:
schema = SchemaGenerator(
{
'openapi': '3.0.0',
'info': {
'title': self.name,
'version': self.version,
'x-acls': tuple(getattr(self.m_acl, 'ACLS', ())),
**({
f'x-{key}': value
for key, value in self.d_domain.items()
}),
},
'components': self.schema_components
}
)
async def inner(request, *args, **kwargs):
"""
description: |
Returns the current API routes description (OpenAPI v3)
as a JSON object
responses:
200:
description: API Schema in OpenAPI v3 format
"""
return ORJSONResponse(
schema.get_schema(routes=request.app.routes))
return inner

109
halfapi/half_route.py Normal file
View File

@ -0,0 +1,109 @@
""" HalfRoute
Child class of starlette.routing.Route
"""
from functools import partial, wraps
from typing import Callable, Coroutine, List, Dict
from types import FunctionType
from starlette.requests import Request
from starlette.responses import PlainTextResponse
from starlette.routing import Route
from starlette.exceptions import HTTPException
from .logging import logger
from .lib.domain import MissingAclError, PathError, UnknownPathParameterType, \
UndefinedRoute, UndefinedFunction
class HalfRoute(Route):
""" HalfRoute
"""
def __init__(self, path: List[str], fct: Callable, params: List[Dict], method: str):
logger.info('HalfRoute creation: %s %s %s %s', path, fct, params, method)
if len(params) == 0:
raise MissingAclError('[{}] {}'.format(method, '/'.join(path)))
if len(path) == 0:
logger.error('Empty path for [{%s}]', method)
raise PathError()
super().__init__(
path,
HalfRoute.acl_decorator(
fct,
params
),
methods=[method])
@staticmethod
def acl_decorator(fct: Callable = None, params: List[Dict] = None) -> Coroutine:
"""
Decorator for async functions that calls pre-conditions functions
and appends kwargs to the target function
Parameters:
fct (Callable):
The function to decorate
params List[Dict]:
A list of dicts that have an "acl" key that points to a function
Returns:
async function
"""
if not params:
params = []
if not fct:
return partial(HalfRoute.acl_decorator, params=params)
@wraps(fct)
async def caller(req: Request, *args, **kwargs):
for param in params:
if param.get('acl'):
passed = param['acl'](req, *args, **kwargs)
if isinstance(passed, FunctionType):
passed = param['acl']()(req, *args, **kwargs)
if not passed:
logger.debug(
'ACL FAIL for current route (%s - %s)', fct, param.get('acl'))
continue
logger.debug(
'ACL OK for current route (%s - %s)', fct, param.get('acl'))
req.scope['acl_pass'] = param['acl'].__name__
if 'args' in param:
req.scope['args'] = param['args']
logger.debug(
'Args for current route (%s)', param.get('args'))
if 'out' in param:
req.scope['out'] = param['out']
if 'out' in param:
req.scope['out'] = param['out'].copy()
if 'check' in req.query_params:
return PlainTextResponse(param['acl'].__name__)
logger.debug('acl_decorator %s', param)
logger.debug('calling %s:%s %s %s', fct.__module__, fct.__name__, args, kwargs)
return await fct(
req, *args,
**{
**kwargs,
})
if 'check' in req.query_params:
return PlainTextResponse('')
raise HTTPException(401)
return caller

316
halfapi/halfapi.py Normal file
View File

@ -0,0 +1,316 @@
#!/usr/bin/env python3
"""
app.py is the file that is read when launching the application using an asgi
runner.
It defines the following globals :
- routes (contains the Route objects for the application)
- application (the asgi application itself - a starlette object)
"""
import sys
import logging
import time
import importlib
from datetime import datetime
# asgi framework
from starlette.applications import Starlette
from starlette.authentication import UnauthenticatedUser
from starlette.exceptions import HTTPException
from starlette.middleware import Middleware
from starlette.routing import Router, Route, Mount
from starlette.requests import Request
from starlette.responses import Response, PlainTextResponse
from starlette.middleware.authentication import AuthenticationMiddleware
from timing_asgi import TimingMiddleware
from timing_asgi.integrations import StarletteScopeToName
# module libraries
from .lib.constants import API_SCHEMA_DICT
from .lib.domain_middleware import DomainMiddleware
from .lib.timing import HTimingClient
from .lib.jwt_middleware import JWTAuthenticationBackend, on_auth_error
from .lib.responses import (ORJSONResponse, UnauthorizedResponse,
NotFoundResponse, InternalServerErrorResponse, NotImplementedResponse,
ServiceUnavailableResponse, gen_exception_route)
from .lib.domain import NoDomainsException
from .lib.routes import gen_schema_routes, JSONRoute
from .lib.schemas import schema_json
from .logging import logger, config_logging
from .half_domain import HalfDomain
from halfapi import __version__
class HalfAPI(Starlette):
def __init__(self,
config,
d_routes=None):
# Set log level (defaults to debug)
config_logging(
getattr(logging, config.get('loglevel', 'DEBUG').upper(), 'DEBUG')
)
self.config = config
SECRET = self.config.get('secret')
PRODUCTION = self.config.get('production', True)
DRYRUN = self.config.get('dryrun', False)
TIMINGMIDDLEWARE = self.config.get('timingmiddleware', False)
if DRYRUN:
logger.info('HalfAPI starting in dry-run mode')
else:
logger.info('HalfAPI starting')
self.PRODUCTION = PRODUCTION
self.SECRET = SECRET
# Domains
""" HalfAPI routes (if not PRODUCTION, includes debug routes)
"""
routes = []
routes.append(
Mount('/halfapi', routes=list(self.halfapi_routes()))
)
logger.debug('Config: %s', self.config)
domains = {
key: elt
for key, elt in self.config.get('domain', {}).items()
if elt.get('enabled', False)
}
logger.debug('Active domains: %s', domains)
if d_routes:
# Mount the routes from the d_routes argument - domain-less mode
logger.info('Domain-less mode : the given schema defines the activated routes')
for route in gen_schema_routes(d_routes):
routes.append(route)
else:
pass
startup_fcts = []
if DRYRUN:
startup_fcts.append(
HalfAPI.wait_quit()
)
super().__init__(
debug=not PRODUCTION,
routes=routes,
exception_handlers={
401: gen_exception_route(UnauthorizedResponse),
404: gen_exception_route(NotFoundResponse),
500: gen_exception_route(HalfAPI.exception),
501: gen_exception_route(NotImplementedResponse),
503: gen_exception_route(ServiceUnavailableResponse)
}
)
schemas = []
self.__domains = {}
for key, domain in domains.items():
if not isinstance(domain, dict):
continue
dom_name = domain.get('name', key)
if not domain.get('enabled', False):
continue
if not domain.get('prefix', False):
if len(domains.keys()) > 1:
raise Exception('Cannot use multiple domains and set prefix to false')
path = '/'
else:
path = f'/{dom_name}'
logger.debug('Mounting domain %s on %s', domain.get('name'), path)
domain_key = domain.get('name', key)
add_domain_args = {
**domain,
'path': path
}
self.add_domain(**add_domain_args)
schemas.append(self.__domains[domain_key].schema())
self.add_route('/', JSONRoute(schemas))
if SECRET:
self.add_middleware(
AuthenticationMiddleware,
backend=JWTAuthenticationBackend(),
on_error=on_auth_error
)
if not PRODUCTION and TIMINGMIDDLEWARE:
self.add_middleware(
TimingMiddleware,
client=HTimingClient(),
metric_namer=StarletteScopeToName(prefix="halfapi",
starlette_app=self)
)
@property
def version(self):
return __version__
async def version_async(self, request, *args, **kwargs):
"""
description: Version route
responses:
200:
description: Currently running HalfAPI's version
"""
return Response(self.version)
@staticmethod
async def exception(request: Request, exc: HTTPException):
logger.critical(exc, exc_info=True)
return InternalServerErrorResponse()
@property
def application(self):
return self
def halfapi_routes(self):
""" Halfapi default routes
"""
async def get_user(request, *args, **kwargs):
"""
description: WhoAmI route
responses:
200:
description: The currently logged-in user
content:
application/json:
schema:
type: object
"""
return ORJSONResponse({'user':request.user})
yield Route('/whoami', get_user)
yield Route('/schema', schema_json)
yield Mount('/acls', self.acls_router())
yield Route('/version', self.version_async)
""" Halfapi debug routes definition
"""
if self.PRODUCTION:
return
""" Debug routes
"""
async def debug_log(request: Request, *args, **kwargs):
logger.debug('debuglog# %s', {datetime.now().isoformat()})
logger.info('debuglog# %s', {datetime.now().isoformat()})
logger.warning('debuglog# %s', {datetime.now().isoformat()})
logger.error('debuglog# %s', {datetime.now().isoformat()})
logger.critical('debuglog# %s', {datetime.now().isoformat()})
return Response('')
yield Route('/log', debug_log)
async def error_code(request: Request, *args, **kwargs):
code = request.path_params['code']
raise HTTPException(code)
yield Route('/error/{code:int}', error_code)
async def exception(request: Request, *args, **kwargs):
raise Exception('Test exception')
yield Route('/exception', exception)
@staticmethod
def api_schema(domain):
pass
@staticmethod
def wait_quit():
""" sleeps 1 second and quits. used in dry-run mode
"""
import time
import sys
time.sleep(1)
sys.exit(0)
def acls_router(self):
mounts = {}
for domain, domain_conf in self.config.get('domain', {}).items():
if isinstance(domain_conf, dict) and domain_conf.get('enabled', False):
mounts['domain'] = HalfDomain.acls_router(
domain,
module_path=domain_conf.get('module'),
acl=domain_conf.get('acl')
)
if len(mounts) > 1:
return Router([
Mount(f'/{domain}', acls_router)
for domain, acls_router in mounts.items()
])
elif len(mounts) == 1:
return Mount('/', mounts.popitem()[1])
else:
return Router()
@property
def domains(self):
return self.__domains
def add_domain(self, **kwargs):
if not kwargs.get('enabled'):
raise Exception(f'Domain not enabled ({kwargs})')
name = kwargs['name']
self.config['domain'][name] = kwargs.get('config', {})
if not kwargs.get('module'):
module = name
else:
module = kwargs.get('module')
try:
self.__domains[name] = HalfDomain(
name,
module=importlib.import_module(module),
router=kwargs.get('router'),
acl=kwargs.get('acl'),
app=self
)
except ImportError as exc:
print(
'Cannot instantiate HalfDomain {} with module {}'.format(
name,
module
))
raise exc
self.mount(kwargs.get('path', name), self.__domains[name])
return self.__domains[name]
def __main__():
return HalfAPI(CONFIG).application
if __name__ == '__main__':
__main__()

View File

@ -2,14 +2,16 @@
"""
Base ACL module that contains generic functions for domains ACL
"""
import logging
from dataclasses import dataclass
from functools import wraps
from json import JSONDecodeError
import yaml
from starlette.authentication import UnauthenticatedUser
from starlette.exceptions import HTTPException
from starlette.routing import Route
from starlette.responses import Response
logger = logging.getLogger('uvicorn.asgi')
from ..logging import logger
def public(*args, **kwargs) -> bool:
"Unlimited access"
@ -30,7 +32,9 @@ def connected(fct=public):
or not hasattr(req.user, 'is_authenticated')):
return False
return fct(req, **{**kwargs, **req.path_params})
if hasattr(req, 'path_params'):
return fct(req, **{**kwargs, **req.path_params})
return fct(req, **{**kwargs})
return caller
@ -53,14 +57,26 @@ def args_check(fct):
data_ = {}
if req.method == 'GET':
data_ = req.query_params
data_ = dict(req.query_params)
if req.method in ['POST', 'PATCH', 'PUT', 'DELETE']:
try:
data_ = await req.json()
except JSONDecodeError as exc:
logger.debug('Posted data was not JSON')
pass
elif req.method in ['POST', 'PATCH', 'PUT', 'DELETE']:
if req.scope.get('headers'):
if b'content-type' not in dict(req.scope.get('headers')):
data_ = {}
else:
content_type = dict(req.scope.get('headers')).get(b'content-type').decode().split(';')[0]
if content_type == 'application/json':
try:
data_ = await req.json()
except JSONDecodeError as exc:
logger.debug('Posted data was not JSON')
pass
elif content_type in [
'multipart/form-data', 'application/x-www-form-urlencoded']:
data_ = dict(await req.form())
else:
data_ = await req.body()
def plural(array: list) -> str:
return 's' if len(array) > 1 else ''
@ -68,8 +84,8 @@ def args_check(fct):
return ', '.join(array)
args_d = kwargs.get('args', None)
if args_d is not None:
args_d = req.scope.get('args')
if args_d is not None and isinstance(data_, dict):
required = args_d.get('required', set())
missing = []
@ -87,15 +103,76 @@ def args_check(fct):
optional = args_d.get('optional', set())
for key in optional:
if key in data_:
data[key] = data_[key]
if key in data_:
data[key] = data_[key]
else:
""" Unsafe mode, without specified arguments
""" Unsafe mode, without specified arguments, or plain text mode
"""
data = data_
kwargs['data'] = data
out_s = req.scope.get('out')
if out_s:
kwargs['out'] = list(out_s)
return await fct(req, *args, **kwargs)
return caller
# ACLS list for doc and priorities
# Write your own constant in your domain or import this one
# Format : (acl_name: str, acl_documentation: str, priority: int, [public=False])
#
# The 'priority' integer is greater than zero and the lower values means more
# priority. For a route, the order of declaration of the ACLs should respect
# their priority.
#
# When the 'public' boolean value is True, a route protected by this ACL is
# defined on the "/halfapi/acls/acl_name", that returns an empty response and
# the status code 200 or 401.
ACLS = (
('private', private.__doc__, 0, True),
('public', public.__doc__, 999, True)
)
@dataclass
class ACL():
name: str
documentation: str
priority: int
public: bool = False
class AclRoute(Route):
def __init__(self, path, acl_fct, acl: ACL):
self.acl_fct = acl_fct
self.name = acl.name
self.description = acl.documentation
self.docstring = yaml.dump({
'description': f'{self.name}: {self.description}',
'responses': {
'200': {
'description': 'ACL OK'
},
'401': {
'description': 'ACL FAIL'
}
}
})
async def endpoint(request, *args, **kwargs):
if request.method == 'GET':
logger.warning('Deprecated since 0.6.28, use HEAD method since now')
if self.acl_fct(request, *args, **kwargs) is True:
return Response(status_code=200)
return Response(status_code=401)
endpoint.__doc__ = self.docstring
return super().__init__(path, methods=['HEAD', 'GET'], endpoint=endpoint)

View File

@ -1 +1,71 @@
import re
from schema import Schema, Optional, Or
from .. import __version__
VERBS = ('GET', 'POST', 'PUT', 'PATCH', 'DELETE')
ITERABLE_STR = Or([ str ], { str }, ( str ))
ACLS_SCHEMA = Schema([{
'acl': str,
Optional('args'): {
Optional('required'): ITERABLE_STR,
Optional('optional'): ITERABLE_STR
},
Optional('out'): ITERABLE_STR
}])
ROUTER_ACLS_SCHEMA = Schema([{
'acl': lambda n: callable(n),
Optional('args'): {
Optional('required'): ITERABLE_STR,
Optional('optional'): ITERABLE_STR
},
Optional('out'): ITERABLE_STR
}])
is_callable_dotted_notation = lambda x: re.match(
r'^(([a-zA-Z_])+\.?)*:[a-zA-Z_]+$', 'ab_c.TEST:get')
ROUTE_SCHEMA = Schema({
Optional(str): { # path - Optional when no routes
str: { # method
'callable': is_callable_dotted_notation,
'docs': lambda n: True, # Should validate an openAPI spec
'acls': ACLS_SCHEMA
}
}
})
DOMAIN_SCHEMA = Schema({
'name': str,
'id': str,
Optional('routers'): str,
Optional('version'): str,
Optional('patch_release'): str,
Optional('acls'): [
[str, str, int, Optional(bool)]
]
})
API_SCHEMA_DICT = {
'openapi': '3.0.0',
'info': {
'title': 'HalfAPI',
'version': __version__
},
}
API_SCHEMA = Schema({
**API_SCHEMA_DICT,
'domain': DOMAIN_SCHEMA,
'paths': ROUTE_SCHEMA
})
ROUTER_SCHEMA = Schema({
Or('', str): {
# Optional('GET'): [],#ACLS_SCHEMA,
Optional(Or(*VERBS)): ROUTER_ACLS_SCHEMA,
Optional('SUBROUTES'): [Optional(str)]
}
})

View File

@ -3,75 +3,145 @@
lib/domain.py The domain-scoped utility functions
"""
import os
import re
import sys
import importlib
import inspect
import logging
from functools import wraps
from types import ModuleType, FunctionType
from typing import Coroutine, Generator
from typing import Dict, List, Tuple, Iterator
from typing import Dict, List, Tuple
import yaml
from starlette.exceptions import HTTPException
from halfapi.lib import acl
from halfapi.lib.responses import ORJSONResponse
from halfapi.lib.router import read_router
from halfapi.lib.responses import ORJSONResponse, ODSResponse, XLSXResponse, PlainTextResponse, HTMLResponse
# from halfapi.lib.router import read_router
from halfapi.lib.constants import VERBS
logger = logging.getLogger("uvicorn.asgi")
from ..logging import logger
class MissingAclError(Exception):
""" Exception to use when no acl are specified for a route
"""
pass
class PathError(Exception):
""" Exception to use when the path for a route is malformed
"""
pass
class UnknownPathParameterType(Exception):
""" Exception to use when the path parameter for a route is not supported
"""
pass
class UndefinedRoute(Exception):
""" Exception to use when the route definition cannot be found
"""
pass
class UndefinedFunction(Exception):
""" Exception to use when a function definition cannot be found
"""
pass
def route_decorator(fct: FunctionType, ret_type: str = 'json') -> Coroutine:
class NoDomainsException(Exception):
""" The exception that is raised when HalfAPI is called without domains
"""
pass
def route_decorator(fct: FunctionType) -> Coroutine:
""" Returns an async function that can be mounted on a router
"""
if ret_type == 'json':
@acl.args_check
async def wrapped(request, *args, **kwargs):
fct_args_spec = inspect.getfullargspec(fct).args
fct_args = request.path_params.copy()
@wraps(fct)
@acl.args_check
async def wrapped(request, *args, **kwargs):
fct_args_spec = inspect.getfullargspec(fct).args
fct_args_defaults = inspect.getfullargspec(fct).defaults or []
fct_args_defaults_dict = dict(list(zip(
reversed(fct_args_spec),
reversed(fct_args_defaults)
)))
if 'halfapi' in fct_args_spec:
fct_args['halfapi'] = {
'user': request.user if
'user' in request else None,
'config': request.scope['config'],
'domain': request.scope['domain'],
fct_args = request.path_params.copy()
}
if 'halfapi' in fct_args_spec:
fct_args['halfapi'] = {
'user': request.user if
'user' in request else None,
'config': request.scope.get('config', {}),
'domain': request.scope.get('domain', 'unknown'),
'cookies': request.cookies,
'base_url': request.base_url,
'url': request.url
}
if 'data' in fct_args_spec:
if 'data' in fct_args_defaults_dict:
fct_args['data'] = fct_args_defaults_dict['data']
else:
fct_args['data'] = {}
if 'data' in fct_args_spec:
fct_args['data'] = kwargs.get('data')
fct_args['data'].update(kwargs.get('data', {}))
try:
if 'out' in fct_args_spec:
fct_args['out'] = kwargs.get('out')
""" If format argument is specified (either by get, post param or function argument)
"""
if 'ret_type' in fct_args_defaults_dict:
ret_type = fct_args_defaults_dict['ret_type']
else:
ret_type = fct_args.get('data', {}).get('format', 'json')
logger.debug('Return type {} (defaults: {})'.format(ret_type,
fct_args_defaults_dict))
try:
logger.debug('FCT_ARGS***** %s', fct_args)
if ret_type == 'json':
return ORJSONResponse(fct(**fct_args))
except NotImplementedError as exc:
raise HTTPException(501) from exc
except Exception as exc:
# TODO: Write tests
if not isinstance(exc, HTTPException):
raise HTTPException(500) from exc
raise exc
if ret_type == 'ods':
res = fct(**fct_args)
assert isinstance(res, list)
for elt in res:
assert isinstance(elt, dict)
return ODSResponse(res)
if ret_type == 'xlsx':
res = fct(**fct_args)
assert isinstance(res, list)
for elt in res:
assert isinstance(elt, dict)
return XLSXResponse(res)
if ret_type in ['html', 'xhtml']:
res = fct(**fct_args)
assert isinstance(res, str)
return HTMLResponse(res)
if ret_type in 'txt':
res = fct(**fct_args)
assert isinstance(res, str)
return PlainTextResponse(res)
else:
raise Exception('Return type not available')
raise NotImplementedError
except NotImplementedError as exc:
raise HTTPException(501) from exc
except Exception as exc:
# TODO: Write tests
logger.error(exc, exc_info=True)
if not isinstance(exc, HTTPException):
raise HTTPException(500) from exc
raise exc
return wrapped
@ -120,170 +190,3 @@ def get_fct_name(http_verb: str, path: str) -> str:
return '_'.join(fct_name)
def gen_routes(m_router: ModuleType,
verb: str,
path: List[str],
params: List[Dict]) -> Tuple[FunctionType, Dict]:
"""
Returns a tuple of the function associatied to the verb and path arguments,
and the dictionary of it's acls
Parameters:
- m_router (ModuleType): The module containing the function definition
- verb (str): The HTTP verb for the route (GET, POST, ...)
- path (List): The route path, as a list (each item being a level of
deepness), from the lowest level (domain) to the highest
- params (Dict): The acl list of the following format :
[{'acl': Function, 'args': {'required': [], 'optional': []}}]
Returns:
(Function, Dict): The destination function and the acl dictionary
"""
if len(params) == 0:
raise MissingAclError('[{}] {}'.format(verb, '/'.join(path)))
if len(path) == 0:
logger.error('Empty path for [{%s}]', verb)
raise PathError()
fct_name = get_fct_name(verb, path[-1])
if hasattr(m_router, fct_name):
fct = getattr(m_router, fct_name)
else:
raise UndefinedFunction('{}.{}'.format(m_router.__name__, fct_name or ''))
if not inspect.iscoroutinefunction(fct):
return route_decorator(fct), params
else:
return fct, params
def gen_router_routes(m_router: ModuleType, path: List[str]) -> \
Iterator[Tuple[str, str, Coroutine, List]]:
"""
Recursive generatore that parses a router (or a subrouter)
and yields from gen_routes
Parameters:
- m_router (ModuleType): The currently treated router module
- path (List[str]): The current path stack
Yields:
(str, str, Coroutine, List): A tuple containing the path, verb,
function and parameters of the route
"""
for subpath, params in read_router(m_router).items():
path.append(subpath)
for verb in VERBS:
if verb not in params:
continue
yield ('/'.join(filter(lambda x: len(x) > 0, path)),
verb,
*gen_routes(m_router, verb, path, params[verb])
)
for subroute in params.get('SUBROUTES', []):
#logger.debug('Processing subroute **%s** - %s', subroute, m_router.__name__)
param_match = re.fullmatch('^([A-Z_]+)_([a-z]+)$', subroute)
if param_match is not None:
try:
path.append('{{{}:{}}}'.format(
param_match.groups()[0].lower(),
param_match.groups()[1]))
except AssertionError as exc:
raise UnknownPathParameterType(subroute) from exc
else:
path.append(subroute)
try:
yield from gen_router_routes(
importlib.import_module(f'.{subroute}', m_router.__name__),
path)
except ImportError as exc:
logger.error('Failed to import subroute **{%s}**', subroute)
raise exc
path.pop()
path.pop()
def d_domains(config) -> Dict[str, ModuleType]:
"""
Parameters:
config (ConfigParser): The .halfapi/config based configparser object
Returns:
dict[str, ModuleType]
"""
domains = {}
if os.environ.get('HALFAPI_DOMAIN_NAME') and os.environ.get('HALFAPI_DOMAIN_MODULE', '.routers'):
domains[os.environ.get('HALFAPI_DOMAIN_NAME')] = os.environ.get('HALFAPI_DOMAIN_MODULE')
elif 'domains' in config:
domains = dict(config['domains'].items())
try:
sys.path.append('.')
return {
domain: importlib.import_module(''.join((domain, module)))
for domain, module in domains.items()
}
except ImportError as exc:
logger.error('Could not load a domain : %s', exc)
raise exc
def router_acls(route_params: Dict, path: List, m_router: ModuleType) -> Generator:
for verb in VERBS:
params = route_params.get(verb)
if params is None:
continue
if len(params) == 0:
logger.error('No ACL for route [{%s}] %s', verb, "/".join(path))
else:
for param in params:
fct_acl = param.get('acl')
if not isinstance(fct_acl, FunctionType):
continue
yield fct_acl.__name__, fct_acl
def domain_acls(m_router, path):
routes = read_router(m_router)
for subpath, route_params in routes.items():
path.append(subpath)
yield from router_acls(route_params, path, m_router)
subroutes = route_params.get('SUBROUTES', [])
for subroute in subroutes:
logger.debug('Processing subroute **%s** - %s', subroute, m_router.__name__)
path.append(subroute)
try:
submod = importlib.import_module(f'.{subroute}', m_router.__name__)
except ImportError as exc:
logger.error('Failed to import subroute **{%s}**', subroute)
raise exc
yield from domain_acls(submod, path)
path.pop()
path.pop()

View File

@ -1,29 +1,28 @@
"""
DomainMiddleware
"""
import logging
from starlette.datastructures import URL
from starlette.middleware.base import (BaseHTTPMiddleware,
RequestResponseEndpoint)
from starlette.requests import Request
from starlette.responses import Response
logger = logging.getLogger('uvicorn.asgi')
from ..logging import logger
class DomainMiddleware(BaseHTTPMiddleware):
"""
DomainMiddleware adds the api routes and acls to the following scope keys :
- domains
- api
- acl
"""
def __init__(self, app, config):
def __init__(self, app, domain=None):
""" app: HalfAPI instance
"""
logger.info('DomainMiddleware app:%s domain:%s', app, domain)
super().__init__(app)
self.config = config
self.domains = {}
self.domain = domain
self.request = None
@ -33,14 +32,27 @@ class DomainMiddleware(BaseHTTPMiddleware):
Call of the route fonction (decorated or not)
"""
l_path = URL(scope=request.scope).path.split('/')
cur_domain = l_path[0]
if len(cur_domain) == 0 and len(l_path) > 1:
cur_domain = l_path[1]
request.scope['domain'] = self.domain['name']
if hasattr(request.app, 'config') \
and isinstance(request.app.config, dict):
# Set the config scope to the domain's config
request.scope['config'] = request.app.config.get(
'domain', {}
).get(
self.domain['name'], {}
).copy()
request.scope['domain'] = cur_domain
request.scope['config'] = self.config['domain_config'][cur_domain] \
if cur_domain in self.config.get('domain_config', {}) else {}
# TODO: Remove in 0.7.0
config = request.scope['config'].copy()
request.scope['config']['domain'] = {}
request.scope['config']['domain'][self.domain['name']] = {}
request.scope['config']['domain'][self.domain['name']]['config'] = config
else:
logger.debug('%s', request.app)
logger.debug('%s', getattr(request.app, 'config', None))
response = await call_next(request)
@ -51,13 +63,18 @@ class DomainMiddleware(BaseHTTPMiddleware):
if 'args' in request.scope:
# Set the http headers "x-args-required" and "x-args-optional"
if 'required' in request.scope['args']:
if len(request.scope['args'].get('required', set())):
response.headers['x-args-required'] = \
','.join(request.scope['args']['required'])
if 'optional' in request.scope['args']:
if len(request.scope['args'].get('optional', set())):
response.headers['x-args-optional'] = \
','.join(request.scope['args']['optional'])
response.headers['x-domain'] = cur_domain
if len(request.scope.get('out', set())):
response.headers['x-out'] = \
','.join(request.scope['out'])
response.headers['x-domain'] = self.domain['name']
return response

View File

@ -12,99 +12,43 @@ Raises:
from os import environ
import typing
import logging
from uuid import UUID
from http.cookies import SimpleCookie
import jwt
from starlette.authentication import (
AuthenticationBackend, AuthenticationError, BaseUser, AuthCredentials,
UnauthenticatedUser)
from starlette.requests import HTTPConnection
from starlette.requests import HTTPConnection, Request
from starlette.exceptions import HTTPException
logger = logging.getLogger('uvicorn.error')
from .user import CheckUser, JWTUser, Nobody
from ..logging import logger
from ..conf import CONFIG
from ..lib.responses import ORJSONResponse
SECRET=None
try:
from ..conf import SECRET
except ImportError as exc:
logger.error('Could not import SECRET variable from conf module,'\
' using HALFAPI_SECRET environment variable')
with open(CONFIG.get('secret', ''), 'r') as secret_file:
SECRET = secret_file.read().strip()
except FileNotFoundError:
logger.error('Could not import SECRET variable from conf module,'\
' using HALFAPI_SECRET environment variable')
class Nobody(UnauthenticatedUser):
""" Nobody class
The default class when no token is passed
"""
@property
def json(self):
return {
'id' : '',
'token': '',
'payload': ''
}
class JWTUser(BaseUser):
""" JWTUser class
Is used to store authentication informations
"""
def __init__(self, user_id: UUID, token: str, payload: dict) -> None:
self.__id = user_id
self.token = token
self.payload = payload
def __str__(self):
return str(self.json)
@property
def json(self):
return {
'id' : str(self.__id),
'token': self.token,
'payload': self.payload
}
@property
def is_authenticated(self) -> bool:
return True
@property
def display_name(self) -> str:
return ' '.join(
(self.payload.get('name'), self.payload.get('firstname')))
@property
def id(self) -> str:
return self.__id
class CheckUser(BaseUser):
""" CheckUser class
Is used to call checks with give user_id, to know if it passes the ACLs for
the given route.
It should never be able to run a route function.
"""
def __init__(self, user_id: UUID) -> None:
self.__id = user_id
@property
def is_authenticated(self) -> bool:
return True
@property
def display_name(self) -> str:
return 'check_user'
@property
def id(self) -> str:
return self.__id
def cookies_from_scope(scope):
cookie = dict(scope.get("headers") or {}).get(b"cookie")
if not cookie:
return {}
simple_cookie = SimpleCookie()
simple_cookie.load(cookie.decode("utf8"))
return {key: morsel.value for key, morsel in simple_cookie.items()}
def on_auth_error(request: Request, exc: Exception):
response = ORJSONResponse({"error": str(exc)}, status_code=401)
response.delete_cookie('Authorization')
return response
class JWTAuthenticationBackend(AuthenticationBackend):
def __init__(self, secret_key: str = SECRET,
@ -124,17 +68,22 @@ class JWTAuthenticationBackend(AuthenticationBackend):
self, conn: HTTPConnection
) -> typing.Optional[typing.Tuple['AuthCredentials', 'BaseUser']]:
# Standard way to authenticate via API
# https://datatracker.ietf.org/doc/html/rfc7235#section-4.2
token = conn.headers.get('Authorization')
if not token:
token = cookies_from_scope(conn.scope).get('Authorization')
is_check_call = 'check' in conn.query_params
is_fake_user_id = is_check_call and 'user_id' in conn.query_params
PRODUCTION = conn.scope['app'].debug == False
if not token and not is_check_call:
return AuthCredentials(), Nobody()
try:
if token and not is_fake_user_id:
if token:
payload = jwt.decode(token,
key=self.secret_key,
algorithms=[self.algorithm],
@ -143,24 +92,18 @@ class JWTAuthenticationBackend(AuthenticationBackend):
})
if is_check_call:
if is_fake_user_id:
try:
fake_user_id = UUID(conn.query_params['user_id'])
return AuthCredentials(), CheckUser(fake_user_id)
except ValueError as exc:
raise HTTPException(400, 'user_id parameter not an uuid')
if token:
return AuthCredentials(), CheckUser(payload['user_id'])
else:
return AuthCredentials(), Nobody()
return AuthCredentials(), Nobody()
if PRODUCTION and 'debug' in payload.keys() and payload['debug']:
raise AuthenticationError(
'Trying to connect using *DEBUG* token in *PRODUCTION* mode')
except jwt.ExpiredSignatureError as exc:
return AuthCredentials(), Nobody()
except jwt.InvalidTokenError as exc:
raise AuthenticationError(str(exc)) from exc
except Exception as exc:

View File

@ -56,10 +56,8 @@ def parse_query(q_string: str = ""):
>>> parse_query('limit:10')
<function parse_query.<locals>.select at 0x...>
>>> parse_query('limit=10')
Traceback (most recent call last):
...
starlette.exceptions.HTTPException: 400
# >>> parse_query('limit=10')
# starlette.exceptions.HTTPException: 400
"""

View File

@ -13,16 +13,22 @@ Classes :
- PlainTextResponse
- ServiceUnavailableResponse
- UnauthorizedResponse
- ODSResponse
"""
from datetime import date
import decimal
import typing
from io import BytesIO
import orjson
# asgi framework
from starlette.responses import PlainTextResponse, Response, JSONResponse
from starlette.responses import PlainTextResponse, Response, JSONResponse, HTMLResponse
from starlette.requests import Request
from starlette.exceptions import HTTPException
from .jwt_middleware import JWTUser, Nobody
from .user import JWTUser, Nobody
from ..logging import logger
__all__ = [
@ -95,6 +101,8 @@ class ORJSONResponse(JSONResponse):
JWTUser, Nobody
}
if callable(typ):
return typ.__name__
if type(typ) in str_types:
return str(typ)
if type(typ) in list_types:
@ -110,3 +118,50 @@ class HJSONResponse(ORJSONResponse):
"""
def render(self, content: typing.Generator):
return super().render(list(content))
class ODSResponse(Response):
file_type = 'ods'
def __init__(self, d_rows: typing.List[typing.Dict]):
try:
import pyexcel as pe
except ImportError:
""" ODSResponse is not handled
"""
super().__init__(content=
'pyexcel is not installed, ods format not available'
)
return
with BytesIO() as ods_file:
rows = []
if len(d_rows):
rows_names = list(d_rows[0].keys())
for elt in d_rows:
rows.append(list(elt.values()))
rows.insert(0, rows_names)
self.sheet = pe.Sheet(rows)
self.sheet.save_to_memory(
file_type=self.file_type,
stream=ods_file)
filename = f'{date.today()}.{self.file_type}'
super().__init__(
content=ods_file.getvalue(),
headers={
'Content-Type': 'application/vnd.oasis.opendocument.spreadsheet; charset=UTF-8',
'Content-Disposition': f'attachment; filename="{filename}"'},
status_code = 200)
class XLSXResponse(ODSResponse):
file_type = 'xlsx'
def gen_exception_route(response_cls):
async def exception_route(req: Request, exc: HTTPException):
return response_cls()
return exception_route

View File

@ -1,44 +1,12 @@
import os
import sys
import subprocess
from types import ModuleType
from typing import Dict
from pprint import pprint
from halfapi.lib.constants import VERBS
from schema import SchemaError
from .constants import VERBS, ROUTER_SCHEMA
from ..logging import logger
def read_router(m_router: ModuleType) -> Dict:
"""
Reads a module and returns a router dict
"""
if not hasattr(m_router, 'ROUTES'):
routes = {'':{}}
acls = getattr(m_router, 'ACLS') if hasattr(m_router, 'ACLS') else None
if acls is not None:
for verb in VERBS:
if not hasattr(m_router, verb.lower()):
continue
""" There is a "verb" route in the router
"""
if verb.upper() not in acls:
continue
routes[''][verb.upper()] = []
routes[''][verb.upper()] = acls[verb.upper()].copy()
routes['']['SUBROUTES'] = []
if hasattr(m_router, '__path__'):
""" Module is a package
"""
m_path = getattr(m_router, '__path__')
if isinstance(m_path, list) and len(m_path) == 1:
routes['']['SUBROUTES'] = [
elt.name
for elt in os.scandir(m_path[0])
if elt.is_dir()
]
else:
routes = getattr(m_router, 'ROUTES')
return routes

View File

@ -2,123 +2,88 @@
"""
Routes module
Classes :
- JSONRoute
Fonctions :
- route_acl_decorator
- gen_starlette_routes
- gen_domain_routes
- gen_schema_routes
- api_routes
- api_acls
- debug_routes
Exception :
- DomainNotFoundError
"""
from datetime import datetime
from functools import partial, wraps
import logging
from typing import Callable, List, Dict, Generator, Tuple
import inspect
from typing import Coroutine, Dict, Generator, Tuple, Any
from types import ModuleType, FunctionType
from starlette.exceptions import HTTPException
from starlette.routing import Route
from starlette.requests import Request
from starlette.responses import Response, PlainTextResponse
import yaml
from halfapi.lib.domain import gen_router_routes, domain_acls
from ..conf import DOMAINSDICT
# from .domain import gen_router_routes, domain_acls, route_decorator, domain_schema
from .responses import ORJSONResponse
from .acl import args_check
from ..half_route import HalfRoute
from . import acl
logger = logging.getLogger('uvicorn.asgi')
from ..logging import logger
class DomainNotFoundError(Exception):
""" Exception when a domain is not importable
"""
def route_acl_decorator(fct: Callable = None, params: List[Dict] = None):
def JSONRoute(data: Any) -> Coroutine:
"""
Decorator for async functions that calls pre-conditions functions
and appends kwargs to the target function
Returns a route function that returns the data as JSON
Parameters:
fct (Callable):
The function to decorate
params List[Dict]:
A list of dicts that have an "acl" key that points to a function
data (Any):
The data to return
Returns:
async function
"""
async def wrapped(request, *args, **kwargs):
return ORJSONResponse(data)
if not params:
params = []
if not fct:
return partial(route_acl_decorator, params=params)
return wrapped
@wraps(fct)
async def caller(req: Request, *args, **kwargs):
for param in params:
if param.get('acl'):
passed = param['acl'](req, *args, **kwargs)
if isinstance(passed, FunctionType):
passed = param['acl']()(req, *args, **kwargs)
if not passed:
logger.debug(
'ACL FAIL for current route (%s - %s)', fct, param.get('acl'))
continue
logger.debug(
'ACL OK for current route (%s - %s)', fct, param.get('acl'))
req.scope['acl_pass'] = param['acl'].__name__
if 'args' in param:
req.scope['args'] = param['args']
if 'check' in req.query_params:
return PlainTextResponse(param['acl'].__name__)
return await fct(
req, *args,
**{
**kwargs,
**param
})
if 'check' in req.query_params:
return PlainTextResponse('')
raise HTTPException(401)
return caller
def gen_starlette_routes(d_domains: Dict[str, ModuleType]) -> Generator:
def gen_domain_routes(m_domain: ModuleType):
"""
Yields the Route objects for HalfAPI app
Yields the Route objects for a domain
Parameters:
d_domains (dict[str, ModuleType])
m_domains: ModuleType
Returns:
Generator(Route)
Generator(HalfRoute)
"""
yield HalfRoute('/',
JSONRoute(domain_schema(m_domain)),
[{'acl': acl.public}],
'GET'
)
for domain_name, m_domain in d_domains.items():
for path, verb, fct, params in gen_router_routes(m_domain, []):
yield (
Route(f'/{domain_name}/{path}',
route_acl_decorator(
fct,
params
),
methods=[verb])
)
for path, method, m_router, fct, params in gen_router_routes(m_domain, []):
yield HalfRoute(f'/{path}', fct, params, method)
def gen_schema_routes(schema: Dict):
"""
Yields the Route objects according to a given schema
"""
for path, methods in schema.items():
for verb, definition in methods.items():
fct = definition.pop('fct')
acls = definition.pop('acls')
# TODO: Check what to do with gen_routes, it is almost the same function
if not inspect.iscoroutinefunction(fct):
yield HalfRoute(path, route_decorator(fct), acls, verb)
else:
yield HalfRoute(path, args_check(fct), acls, verb)
def api_routes(m_dom: ModuleType) -> Tuple[Dict, Dict]:
"""
@ -150,56 +115,22 @@ def api_routes(m_dom: ModuleType) -> Tuple[Dict, Dict]:
return l_params
d_res = {}
for path, verb, _, params in gen_router_routes(m_dom, []):
if path not in d_res:
d_res[path] = {}
d_res[path][verb] = str_acl(params)
for path, verb, m_router, fct, params in gen_router_routes(m_dom, []):
try:
if path not in d_res:
d_res[path] = {}
d_res[path][verb] = {
'docs': yaml.load(fct.__doc__, Loader=yaml.FullLoader),
'acls': str_acl(params)
}
except Exception as exc:
logger.error("""Error in route generation
path:%s
verb:%s
router:%s
fct:%s
params:%s """, path, verb, m_router, fct, params)
raise exc
return d_res, d_acls
def api_acls(request):
""" Returns the list of possible ACLs
"""
res = {}
domains = DOMAINSDICT()
doc = 'doc' in request.query_params
for domain, m_domain in domains.items():
res[domain] = {}
for acl_name, fct in domain_acls(m_domain, [domain]):
if not isinstance(fct, FunctionType):
continue
fct_result = fct.__doc__.strip() if doc and fct.__doc__ else fct(request)
if acl_name in res[domain]:
continue
if isinstance(fct_result, FunctionType):
fct_result = fct()(request)
res[domain][acl_name] = fct_result
return res
def debug_routes():
""" Halfapi debug routes definition
"""
async def debug_log(request: Request, *args, **kwargs):
logger.debug('debuglog# %s', {datetime.now().isoformat()})
logger.info('debuglog# %s', {datetime.now().isoformat()})
logger.warning('debuglog# %s', {datetime.now().isoformat()})
logger.error('debuglog# %s', {datetime.now().isoformat()})
logger.critical('debuglog# %s', {datetime.now().isoformat()})
return Response('')
yield Route('/halfapi/log', debug_log)
async def error_code(request: Request, *args, **kwargs):
code = request.path_params['code']
raise HTTPException(code)
yield Route('/halfapi/error/{code:int}', error_code)
async def exception(request: Request, *args, **kwargs):
raise Exception('Test exception')
yield Route('/halfapi/exception', exception)

View File

@ -1,7 +1,6 @@
""" Schemas module
Functions :
- get_api_routes
- schema_json
- schema_dict_dom
- get_acls
@ -11,113 +10,128 @@ Constant :
"""
import os
import logging
from typing import Dict, Coroutine
import importlib
from typing import Dict, Coroutine, List
from types import ModuleType
import yaml
from starlette.schemas import SchemaGenerator
from .. import __version__
from .routes import gen_starlette_routes, api_routes, api_acls
from ..logging import logger
from .routes import api_routes
from .responses import ORJSONResponse
logger = logging.getLogger('uvicorn.asgi')
SCHEMAS = SchemaGenerator(
{"openapi": "3.0.0", "info": {"title": "HalfAPI", "version": __version__}}
)
def get_api_routes(domains: Dict[str, ModuleType]) -> Coroutine:
"""
description: Returns the current API routes dictionary
as a JSON object
example: {
"dummy_domain": {
"abc/alphabet": {
"GET": [
{
"acl": "public"
}
]
},
"abc/alphabet/{test:uuid}": {
"GET": [
{
"acl": "public"
}
],
"POST": [
{
"acl": "public"
}
],
"PATCH": [
{
"acl": "public"
}
],
"PUT": [
{
"acl": "public"
}
]
}
}
}
"""
routes = {
domain: api_routes(m_domain)[0]
for domain, m_domain in domains.items()
}
async def wrapped(request, *args, **kwargs):
return ORJSONResponse(routes)
return wrapped
def get_api_domain_routes(m_domain: ModuleType) -> Coroutine:
routes, _ = api_routes(m_domain)
async def wrapped(request, *args, **kwargs):
"""
description: Returns the current API routes dictionary for a specific
domain as a JSON object
"""
return ORJSONResponse(routes)
return wrapped
async def schema_json(request, *args, **kwargs):
"""
description: Returns the current API routes description (OpenAPI v3)
as a JSON object
description: |
Returns the current API routes description (OpenAPI v3)
as a JSON object
responses:
200:
description: API Schema in OpenAPI v3 format
"""
return ORJSONResponse(
SCHEMAS.get_schema(routes=request.app.routes))
def schema_dict_dom(d_domains: Dict[str, ModuleType]) -> Dict:
def schema_csv_dict(csv: List[str], prefix='/') -> Dict:
package = None
schema_d = {}
modules_d = {}
acl_modules_d = {}
for line in csv:
if not line:
continue
path, verb, router, acl_fct_name, args_req, args_opt, out = line.strip().split(';')
logger.info('schema_csv_dict %s %s %s', path, args_req, args_opt)
path = f'{prefix}{path}'
if path not in schema_d:
schema_d[path] = {}
if verb not in schema_d[path]:
mod_str = router.split(':')[0]
fct_str = router.split(':')[1]
if mod_str not in modules_d:
modules_d[mod_str] = importlib.import_module(mod_str)
if not hasattr(modules_d[mod_str], fct_str):
raise Exception(
'Missing function in module. module:{} function:{}'.format(
router, fct_str
)
)
fct = getattr(modules_d[mod_str], fct_str)
schema_d[path][verb] = {
'module': modules_d[mod_str],
'fct': fct,
'acls': []
}
if package and router.split('.')[0] != package:
raise Exception('Multi-domain is not allowed in that mode')
package = router.split('.')[0]
if not len(package):
raise Exception(
'Empty package name (router=%s)'.format(router))
acl_package = f'{package}.acl'
if acl_package not in acl_modules_d:
if acl_package not in modules_d:
modules_d[acl_package] = importlib.import_module(acl_package)
if not hasattr(modules_d[acl_package], acl_fct_name):
raise Exception(
'Missing acl function in module. module:{} acl:{}'.format(
acl_package, acl_fct_name
)
)
acl_modules_d[acl_package] = {}
acl_modules_d[acl_package][acl_fct_name] = getattr(modules_d[acl_package], acl_fct_name)
schema_d[path][verb]['acls'].append({
'acl': acl_modules_d[acl_package][acl_fct_name],
'args': {
'required': set(args_req.split(',')) if len(args_req) else set(),
'optional': set(args_opt.split(',')) if len(args_opt) else set()
}
})
return schema_d
def param_docstring_default(name, type):
""" Returns a default docstring in OpenAPI format for a path parameter
"""
Returns the API schema of the *m_domain* domain as a python dictionnary
Parameters:
d_domains (Dict[str, moduleType]): The module to scan for routes
Returns:
Dict: A dictionnary containing the description of the API using the
| OpenAPI standard
"""
return SCHEMAS.get_schema(
routes=list(gen_starlette_routes(d_domains)))
async def get_acls(request, *args, **kwargs):
"""
description: A dictionnary of the domains and their acls, with the
result of the acls functions
"""
return ORJSONResponse(api_acls(request))
type_map = {
'str': 'string',
'uuid': 'string',
'path': 'string',
'int': 'number',
'float': 'number'
}
return yaml.dump({
'name': name,
'in': 'path',
'description': f'default description for path parameter {name}',
'required': True,
'schema': {
'type': type_map[type]
}
})

View File

@ -9,7 +9,7 @@ import logging
from timing_asgi import TimingClient
logger = logging.getLogger('uvicorn.asgi')
from ..logging import logger
class HTimingClient(TimingClient):
""" Used to redefine TimingClient.timing

79
halfapi/lib/user.py Normal file
View File

@ -0,0 +1,79 @@
from uuid import UUID
from starlette.authentication import BaseUser, UnauthenticatedUser
class Nobody(UnauthenticatedUser):
""" Nobody class
The default class when no token is passed
"""
@property
def json(self):
return {
'id' : '',
'token': '',
'payload': ''
}
class JWTUser(BaseUser):
""" JWTUser class
Is used to store authentication informations
"""
def __init__(self, user_id: UUID, token: str, payload: dict) -> None:
self.__id = user_id
self.token = token
self.payload = payload
def __str__(self):
return str(self.json)
@property
def json(self):
return {
'id' : str(self.__id),
'token': self.token,
'payload': self.payload
}
@property
def is_authenticated(self) -> bool:
return True
@property
def display_name(self) -> str:
return ' '.join(
(self.payload.get('name'), self.payload.get('firstname')))
@property
def id(self) -> str:
return self.__id
class CheckUser(BaseUser):
""" CheckUser class
Is used to call checks with give user_id, to know if it passes the ACLs for
the given route.
It should never be able to run a route function.
"""
def __init__(self, user_id: UUID) -> None:
self.__id = user_id
@property
def is_authenticated(self) -> bool:
return True
@property
def display_name(self) -> str:
return 'check_user'
@property
def id(self) -> str:
return self.__id

View File

@ -1,8 +1,10 @@
import logging
default_level = logging.DEBUG
default_format = '%(asctime)s [%(process)d] [%(levelname)s] %(message)s'
default_datefmt = '[%Y-%m-%d %H:%M:%S %z]'
def config_logging(level=logging.INFO):
def config_logging(level=default_level, format=default_format, datefmt=default_datefmt):
# When run by 'uvicorn ...', a root handler is already
# configured and the basicConfig below does nothing.
# To get the desired formatting:
@ -12,8 +14,8 @@ def config_logging(level=logging.INFO):
# https://github.com/encode/uvicorn/issues/511
logging.basicConfig(
# match gunicorn format
format='%(asctime)s [%(process)d] [%(levelname)s] %(message)s',
datefmt='[%Y-%m-%d %H:%M:%S %z]',
format=format,
datefmt=datefmt,
level=level)
# When run by 'gunicorn -k uvicorn.workers.UvicornWorker ...',
@ -27,5 +29,4 @@ def config_logging(level=logging.INFO):
logging.getLogger('uvicorn.access').propagate = True
logging.getLogger('uvicorn.error').propagate = True
config_logging()
logger = logging.getLogger('uvicorn.asgi')
logger = logging.getLogger()

View File

@ -1,75 +0,0 @@
create schema api;
create type verb as enum ('POST', 'GET', 'PUT', 'PATCH', 'DELETE');
create table api.domain (
name text,
primary key (name)
);
create table api.router (
name text,
domain text,
primary key (name, domain)
);
alter table api.router add constraint router_domain_fkey foreign key (domain) references api.domain(name) on update cascade on delete cascade;
create table api.route (
http_verb verb,
path text, -- relative to /<domain>/<router>
fct_name text,
router text,
domain text,
primary key (http_verb, path, router, domain)
);
alter table api.route add constraint route_router_fkey foreign key (router, domain) references api.router(name, domain) on update cascade on delete cascade;
create table api.acl_function (
name text,
description text,
domain text,
primary key (name, domain)
);
alter table api.acl_function add constraint acl_function_domain_fkey foreign key (domain) references api.domain(name) on update cascade on delete cascade;
create table api.acl (
http_verb verb,
path text not null,
router text,
domain text,
acl_fct_name text,
keys text[],
primary key (http_verb, path, router, domain, acl_fct_name)
);
alter table api.acl add constraint acl_route_fkey foreign key (http_verb, path,
router, domain) references api.route(http_verb, path, router, domain) on update cascade on delete cascade;
alter table api.acl add constraint acl_function_fkey foreign key (acl_fct_name, domain) references api.acl_function(name, domain) on update cascade on delete cascade;
create schema "api.view";
create view "api.view".route as
select
route.*,
'/'::text || route.domain || '/'::text || route.router || route.path AS abs_path
from
api.route
join api.domain on
route.domain = domain.name
;
create view "api.view".acl as
select
acl.*,
'/'::text || route.domain || '/'::text || route.router || route.path AS abs_path
from
api.acl
join api.acl_function on
acl.acl_fct_name = acl_function.name
join api.route on
acl.domain = route.domain
and acl.router = route.router
and acl.path = route.path;

View File

View File

@ -0,0 +1,152 @@
import importlib
import functools
import os
import sys
import json
from json.decoder import JSONDecodeError
import toml
from unittest import TestCase
from starlette.testclient import TestClient
from click.testing import CliRunner
from ..cli.cli import cli
from ..halfapi import HalfAPI
from ..half_domain import HalfDomain
from ..conf import DEFAULT_CONF
from pprint import pprint
import tempfile
class TestDomain(TestCase):
@property
def domain_name(self):
return getattr(self, 'DOMAIN')
@property
def module_name(self):
return getattr(self, 'MODULE', self.domain_name)
@property
def acl_path(self):
return getattr(self, 'ACL', '.acl')
@property
def router_path(self):
return getattr(self, 'ROUTERS', '.routers')
@property
def router_module(self):
return '.'.join((self.module_name, self.ROUTERS))
def setUp(self):
# CLI
class_ = CliRunner
def invoke_wrapper(f):
"""Augment CliRunner.invoke to emit its output to stdout.
This enables pytest to show the output in its logs on test
failures.
"""
@functools.wraps(f)
def wrapper(*args, **kwargs):
echo = kwargs.pop('echo', False)
result = f(*args, **kwargs)
if echo is True:
sys.stdout.write(result.output)
return result
return wrapper
class_.invoke = invoke_wrapper(class_.invoke)
self.runner = class_(mix_stderr=False)
# HTTP
# Fake default values of default configuration
self.halfapi_conf = {
'secret': 'testsecret',
'production': False,
'domain': {}
}
self.halfapi_conf['domain'][self.domain_name] = {
'name': self.domain_name,
'router': self.router_path,
'acl': self.acl_path,
'module': self.module_name,
'prefix': False,
'enabled': True,
'config': getattr(self, 'CONFIG', {})
}
_, self.config_file = tempfile.mkstemp()
with open(self.config_file, 'w') as fh:
fh.write(toml.dumps(self.halfapi_conf))
self.halfapi = HalfAPI(self.halfapi_conf)
self.client = TestClient(self.halfapi.application)
self.module = importlib.import_module(
self.module_name
)
def tearDown(self):
pass
def check_domain(self):
result = None
try:
result = self.runner.invoke(cli, '--version')
self.assertEqual(result.exit_code, 0)
result = self.runner.invoke(cli, ['domain', '--read', self.DOMAIN, self.config_file])
self.assertEqual(result.exit_code, 0)
result_d = json.loads(result.stdout)
result = self.runner.invoke(cli, ['run', '--help'])
self.assertEqual(result.exit_code, 0)
# result = self.runner.invoke(cli, ['run', '--dryrun', self.DOMAIN])
# self.assertEqual(result.exit_code, 0)
except AssertionError as exc:
print(f'Result {result}')
print(f'Stdout {result.stdout}')
print(f'Stderr {result.stderr}')
raise exc
except JSONDecodeError as exc:
print(f'Result {result}')
print(f'Stdout {result.stdout}')
raise exc
return result_d
def check_routes(self):
r = self.client.request('get', '/')
assert r.status_code == 200
schema = r.json()
assert isinstance(schema, dict)
assert 'openapi' in schema
assert 'info' in schema
assert 'paths' in schema
r = self.client.request('get', '/halfapi/acls')
assert r.status_code == 200
d_r = r.json()
assert isinstance(d_r, dict)
assert self.domain_name in d_r.keys()
ACLS = HalfDomain.acls(self.module, self.acl_path)
assert len(ACLS) == len(d_r[self.domain_name])
for acl_rule in ACLS:
assert len(acl_rule.name) > 0
assert acl_rule.name in d_r[self.domain_name]
assert len(acl_rule.documentation) > 0
assert isinstance(acl_rule.priority, int)
assert acl_rule.priority >= 0
if acl_rule.public is True:
r = self.client.request('get', f'/halfapi/acls/{acl_rule.name}')
assert r.status_code in [200, 401]

View File

@ -0,0 +1,3 @@
pyexcel>=0.6.3,<1
pyexcel-ods>=0.5.6,<1
pyexcel-xlsx=0.6.0,<1

View File

@ -1,4 +1,5 @@
[pytest]
testpaths = tests halfapi
addopts = --doctest-modules
doctest_optionflags = ELLIPSIS
doctest_optionflags = ELLIPSIS IGNORE_EXCEPTION_DETAIL
pythonpath = ./tests

View File

@ -1,5 +1,61 @@
click
starlette
uvicorn
PyJWT
pygit2==0.28.2
alog==0.9.13
anyio==3.4.0
asgiref==3.4.1
astroid==2.9.0
attrs==21.2.0
bleach==4.1.0
build==0.7.0
certifi==2021.10.8
cffi==1.15.0
charset-normalizer==2.0.9
click==7.1.2
colorama==0.4.4
contextlib2==21.6.0
cryptography==36.0.1
docutils==0.18.1
h11==0.12.0
idna==3.3
importlib-metadata==4.8.2
iniconfig==1.1.1
isort==5.10.1
jeepney==0.7.1
keyring==23.4.0
lazy-object-proxy==1.7.1
mccabe==0.6.1
orjson==3.6.5
packaging==21.3
pep517==0.12.0
pkginfo==1.8.2
platformdirs==2.4.0
pluggy==1.0.0
py==1.11.0
pycparser==2.21
pyflakes==2.4.0
Pygments==2.10.0
PyJWT==2.3.0
pylint==2.12.2
pyparsing==3.0.6
pytest==6.2.5
pytest-asyncio==0.16.0
pytest-pythonpath==0.7.3
PyYAML==5.4.1
readme-renderer==32.0
requests==2.26.0
requests-toolbelt==0.9.1
rfc3986==1.5.0
schema==0.7.5
SecretStorage==3.3.1
six==1.16.0
sniffio==1.2.0
starlette==0.17.1
timing-asgi==0.2.1
toml==0.10.2
tomli==2.0.0
tqdm==4.62.3
twine==3.7.1
urllib3==1.26.7
uvicorn==0.16.0
vulture==2.3
webencodings==0.5.1
wrapt==1.13.3
zipp==3.6.0

View File

@ -43,28 +43,43 @@ setup(
packages=get_packages(module_name),
python_requires=">=3.8",
install_requires=[
"PyJWT>=2.0.1",
"starlette>=0.16,<0.17",
"click>=7.1,<8",
"PyJWT>=2.6.0,<2.7.0",
"starlette>=0.33,<0.34",
"click>=8,<9",
"uvicorn>=0.13,<1",
"orjson>=3.4.7,<4",
"pyyaml>=5.3.1,<6",
"timing-asgi>=0.2.1,<1"
"orjson>=3.8.5,<4",
"pyyaml>=6,<7",
"timing-asgi>=0.2.1,<1",
"schema>=0.7.4,<1",
"toml>=0.10,<0.11",
"packaging>=19.0",
"python-multipart"
],
classifiers=[
"Development Status :: 3 - Alpha",
"License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
"Intended Audience :: Developers",
"Topic :: Software Development :: Libraries :: Application Frameworks",
"Programming Language :: Python :: 3.7",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10"
],
extras_require={
"tests":[
"pytest",
"requests",
"pytest>=7,<8",
"pytest-asyncio",
"pylint"
"pylint",
"requests",
"httpx",
"openapi-schema-validator",
"openapi-spec-validator",
"coverage"
],
"pyexcel":[
"pyexcel",
"pyexcel-ods3",
"pyexcel-xlsx"
]
},
entry_points={

93
tests/cli/test_cli.py Normal file
View File

@ -0,0 +1,93 @@
#!/usr/bin/env python3
import os
import subprocess
import importlib
import tempfile
from unittest.mock import patch
import pytest
from click.testing import CliRunner
from configparser import ConfigParser
from halfapi import __version__
from halfapi.cli import cli
Cli = cli.cli
PROJNAME = os.environ.get('PROJ','tmp_api')
def test_options(runner):
# Wrong command
with runner.isolated_filesystem():
r = runner.invoke(Cli, ['foobar'])
assert r.exit_code == 2
# Test existing commands
with runner.isolated_filesystem():
r = runner.invoke(Cli, ['--help'])
assert r.exit_code == 0
with runner.isolated_filesystem():
r = runner.invoke(Cli, ['--version'])
assert r.exit_code == 0
with runner.isolated_filesystem():
r = runner.invoke(Cli, ['init', '--help'])
assert r.exit_code == 0
@pytest.mark.skip
def test_init_project_fail(runner):
# Missing argument (project)
testproject = 'testproject'
r = runner.invoke(Cli, ['init'])
assert r.exit_code == 2
with runner.isolated_filesystem():
# Fail : Wrong project name
r = runner.invoke(Cli, ['init', 'test*-project'])
assert r.exit_code == 1
with runner.isolated_filesystem():
# Fail : Already existing folder
os.mkdir(testproject)
r = runner.invoke(Cli, ['init', testproject])
assert r.exit_code == 1
with runner.isolated_filesystem():
# Fail : Already existing nod
os.mknod(testproject)
r = runner.invoke(Cli, ['init', testproject])
assert r.exit_code == 1
@pytest.mark.skip
def test_init_project(runner):
"""
"""
cp = ConfigParser()
with runner.isolated_filesystem():
env = {
'HALFAPI_CONF_DIR': '.halfapi'
}
res = runner.invoke(Cli, ['init', PROJNAME], env=env)
try:
assert os.path.isdir(PROJNAME)
assert os.path.isdir(os.path.join(PROJNAME, '.halfapi'))
# .halfapi/config check
assert os.path.isfile(os.path.join(PROJNAME, '.halfapi', 'config'))
cp.read(os.path.join(PROJNAME, '.halfapi', 'config'))
assert cp.has_section('project')
assert cp.has_option('project', 'name')
assert cp.get('project', 'name') == PROJNAME
assert cp.get('project', 'halfapi_version') == __version__
# removal of domain section (0.6)
# assert cp.has_section('domain')
except AssertionError as exc:
subprocess.run(['tree', '-a', os.getcwd()])
raise exc
assert res.exit_code == 0
assert res.exception is None

View File

@ -0,0 +1,9 @@
from halfapi.cli.cli import cli
from configparser import ConfigParser
def test_config(cli_runner):
with cli_runner.isolated_filesystem():
result = cli_runner.invoke(cli, ['config'])
cp = ConfigParser()
cp.read_string(result.output)
assert cp.has_section('project')

View File

@ -0,0 +1,84 @@
#!/usr/bin/env python3
import os
import subprocess
import importlib
import tempfile
from unittest.mock import patch
import json
import toml
import pytest
from click.testing import CliRunner
from configparser import ConfigParser
from halfapi.conf import DEFAULT_CONF, PROJECT_LEVEL_KEYS, DOMAIN_LEVEL_KEYS
PROJNAME = os.environ.get('PROJ','tmp_api')
class TestCliProj():
def test_cmds(self, project_runner):
assert project_runner('--help').exit_code == 0
#assert project_runner('run', '--help').exit_code == 0
#assert project_runner('domain', '--help').exit_code == 0
def test_domain_commands(self, project_runner):
""" TODO: Test create command
"""
test_conf = {
'project': {
'port': '3010',
'loglevel': 'warning'
},
'domain': {
'dummy_domain': {
'port': 4242,
'name': 'dummy_domain',
'enabled': True
}
}
}
r = project_runner('domain')
print(r.stdout)
assert r.exit_code == 1
_, tmp_conf = tempfile.mkstemp()
with open(tmp_conf, 'w') as fh:
fh.write(
toml.dumps(test_conf)
)
r = project_runner(f'domain dummy_domain --conftest {tmp_conf}')
assert r.exit_code == 0
r_conf = toml.loads(r.stdout)
for key, value in r_conf.items():
if key == 'domain':
continue
assert key in PROJECT_LEVEL_KEYS
if key == 'port':
assert value == test_conf['domain']['dummy_domain']['port']
elif key == 'loglevel':
assert value == test_conf['project']['loglevel']
else:
assert value == DEFAULT_CONF[key.upper()]
assert json.dumps(test_conf['domain']) == json.dumps(r_conf['domain'])
for key in test_conf['domain']['dummy_domain'].keys():
assert key in DOMAIN_LEVEL_KEYS
# Default command "run"
r = project_runner(f'domain dummy_domain --dry-run {tmp_conf}')
print(r.stdout)
assert r.exit_code == 0
def test_config_commands(self, project_runner):
try:
r = project_runner('config')
assert r.exit_code == 0
except AssertionError as exc:
subprocess.call(['tree', '-a'])
raise exc

33
tests/cli/test_cli_run.py Normal file
View File

@ -0,0 +1,33 @@
import pytest
from click.testing import CliRunner
from halfapi.cli.cli import cli
import os
from unittest.mock import patch
@pytest.mark.skip
def test_run_noproject(cli_runner):
with cli_runner.isolated_filesystem():
result = cli_runner.invoke(cli, ['config'])
print(result.stdout)
assert result.exit_code == 0
result = cli_runner.invoke(cli, ['run', '--dryrun'])
try:
assert result.exit_code == 0
except AssertionError as exc:
print(result.stdout)
raise exc
"""
def test_run_empty_project(cli_runner):
with cli_runner.isolated_filesystem():
os.mkdir('dummy_domain')
result = cli_runner.invoke(cli, ['run', './dummy_domain'])
assert result.exit_code == 1
def test_run_dummy_project(project_runner):
with patch('uvicorn.run', autospec=True) as runMock:
result = project_runner.invoke(cli, ['run'])
runMock.assert_called_once()
"""

View File

@ -18,12 +18,14 @@ from starlette.responses import PlainTextResponse
from starlette.middleware.authentication import AuthenticationMiddleware
from starlette.testclient import TestClient
from halfapi import __version__
from halfapi.halfapi import HalfAPI
from halfapi.cli.cli import cli
from halfapi.cli.init import init, format_halfapi_etc
from halfapi.cli.init import init
from halfapi.cli.domain import domain, create_domain
from halfapi.lib.responses import ORJSONResponse
from halfapi.lib.jwt_middleware import JWTAuthenticationBackend
logger = logging.getLogger('halfapitest')
logger = logging.getLogger()
PROJNAME = os.environ.get('PROJ','tmp_api')
@ -33,6 +35,18 @@ from halfapi.lib.jwt_middleware import (
JWTUser, JWTAuthenticationBackend,
JWTWebSocketAuthenticationBackend)
@pytest.fixture
def dummy_domain():
yield {
'name': 'dummy_domain',
'router': '.routers',
'enabled': True,
'prefix': False,
'config': {
'test': True
}
}
@pytest.fixture
def token_builder():
yield jwt.encode({
@ -62,7 +76,7 @@ def token_debug_true_builder():
@pytest.fixture
def runner():
return CliRunner()
return CliRunner(mix_stderr=False)
@pytest.fixture
@ -90,11 +104,10 @@ def cli_runner():
return wrapper
class_.invoke = invoke_wrapper(class_.invoke)
cli_runner_ = class_()
cli_runner_ = class_(mix_stderr=False)
yield cli_runner_
@pytest.fixture
def halfapicli(cli_runner):
def caller(*args):
@ -102,24 +115,6 @@ def halfapicli(cli_runner):
yield caller
@pytest.fixture
def halfapi_conf_dir():
return confdir('HALFAPI_CONF_DIR')
def confdir(dirname):
d = os.environ.get(dirname)
if not d:
os.environ[dirname] = tempfile.mkdtemp(prefix='halfapi_')
return os.environ.get(dirname)
if not os.path.isdir(d):
os.mkdir(d)
return d
@pytest.fixture
def halform_conf_dir():
return confdir('HALFORM_CONF_DIR')
# store history of failures per test class name and per index in parametrize (if
# parametrize used)
@ -171,36 +166,39 @@ def pytest_runtest_setup(item):
pytest.xfail("previous test failed ({})".format(test_name))
@pytest.fixture
def project_runner(runner, halfapicli, halfapi_conf_dir):
def project_runner(runner, halfapicli, tree):
with runner.isolated_filesystem():
res = halfapicli('init', PROJNAME)
try:
os.chdir(PROJNAME)
except FileNotFoundError as exc:
subprocess.call('tree')
raise exc
os.chdir(PROJNAME)
fs_path = os.getcwd()
sys.path.insert(0, fs_path)
secret = tempfile.mkstemp()
SECRET_PATH = secret[1]
with open(SECRET_PATH, 'w') as f:
f.write(str(uuid1()))
with open(os.path.join(halfapi_conf_dir, PROJNAME), 'w') as halfapi_etc:
"""
with open(os.path.join('.halfapi', PROJNAME), 'w') as halfapi_etc:
PROJ_CONFIG = re.sub('secret = .*', f'secret = {SECRET_PATH}',
format_halfapi_etc(PROJNAME, os.getcwd()))
halfapi_etc.write(PROJ_CONFIG)
"""
###
# add dummy domain
###
create_domain('tests', '.dummy_domain.routers')
create_domain('test_domain', 'test_domain.routers')
###
yield halfapicli
while fs_path in sys.path:
sys.path.remove(fs_path)
@pytest.fixture
def dummy_app():
app = Starlette()
@ -250,7 +248,6 @@ def create_route():
@pytest.fixture
def dummy_project():
sys.path.insert(0, './tests')
halfapi_config = tempfile.mktemp()
halfapi_secret = tempfile.mktemp()
domain = 'dummy_domain'
@ -263,45 +260,63 @@ def dummy_project():
f'secret = {halfapi_secret}\n',
'port = 3050\n',
'loglevel = debug\n',
'[domains]\n',
f'{domain} = .routers',
f'[{domain}]',
'[domain.dummy_domain]\n',
f'name = {domain}\n',
'router = dummy_domain.routers\n',
f'[domain.dummy_domain.config]\n',
'test = True'
])
with open(halfapi_secret, 'w') as f:
f.write('turlututu')
return (halfapi_config, 'dummy_domain', 'routers')
return (halfapi_config, 'dummy_domain', 'dummy_domain.routers')
@pytest.fixture
def routers():
sys.path.insert(0, './tests')
def application_debug(project_runner):
halfAPI = HalfAPI({
'secret':'turlututu',
'production':False,
'domain': {
'dummy_domain': {
'name': 'dummy_domain',
'router': '.routers',
'enabled': True,
'prefix': False,
'config':{
'test': True
}
}
},
})
from dummy_domain import routers
return routers
assert isinstance(halfAPI, HalfAPI)
yield halfAPI.application
@pytest.fixture
def application_debug():
from halfapi.app import HalfAPI
def application_domain(dummy_domain):
return HalfAPI({
'SECRET':'turlututu',
'PRODUCTION':False
}).application
@pytest.fixture
def application_domain(routers):
from halfapi.app import HalfAPI
return HalfAPI({
'SECRET':'turlututu',
'PRODUCTION':True,
'DOMAINS':{'dummy_domain':routers},
'CONFIG':{
'domains': {'dummy_domain':routers},
'domain_config': {'dummy_domain': {'test': True}}
'secret':'turlututu',
'production':True,
'domain': {
'dummy_domain': {
**dummy_domain,
'config': {
'test': True
}
}
}
}).application
@pytest.fixture
def tree():
def wrapped(path):
list_dirs = os.walk(path)
for root, dirs, files in list_dirs:
for d in dirs:
print(os.path.join(root, d))
for f in files:
print(os.path.join(root, f))
return wrapped

View File

@ -0,0 +1,27 @@
from halfapi import __version__ as halfapi_version
domain = {
'name': 'dummy_domain',
'version': '0.0.0',
'id': '8b88e60a625369235b36c2d6d70756a0c02c1c7fb169fcee6dc820bcf9723f5a',
'deps': (
('halfapi', '=={}'.format(halfapi_version)),
),
'schema_components': {
'schemas': {
'Pinnochio': {
'type': 'object',
'required': {
'id',
'name',
'nose_size'
},
'properties': {
'id': {'type': 'string'},
'name': {'type': 'string'},
'nose_size': {'type': 'number'}
}
}
}
}
}

View File

@ -1 +1,13 @@
public = lambda *args: True
from halfapi.lib import acl
from halfapi.lib.acl import public, private, ACLS
from random import randint
def random(*args):
""" Random access ACL
"""
return randint(0,1) == 1
ACLS = (
*ACLS,
('random', random.__doc__, 10)
)

View File

@ -1,5 +0,0 @@
ROUTES={
'': {
'SUBROUTES': ['personne']
}
}

View File

@ -1,13 +0,0 @@
ROUTES={
'': {
'GET': [
{'acl':None, 'out':('id')}
],
},
'{user_id:uuid}': {
'GET': [
{'acl':None, 'out':('id')}
],
'SUBROUTES': ['eo']
}
}

View File

@ -1,10 +0,0 @@
from starlette.responses import Response
ROUTES={
'': {
'GET': [{'acl': 'None', 'in': ['ok']}]
}
}
async def get_(req):
return Response()

View File

@ -1,4 +1,5 @@
from halfapi.lib import acl
from halfapi.lib.responses import ORJSONResponse
ACLS = {
'GET': [{'acl':acl.public}],
'POST': [{'acl':acl.public}],
@ -7,17 +8,52 @@ ACLS = {
'DELETE': [{'acl':acl.public}]
}
def get(test):
return str(test)
async def get(test):
"""
description:
returns the path parameter
responses:
200:
description: test response
"""
return ORJSONResponse(str(test))
def post(test):
"""
description:
returns the path parameter
responses:
200:
description: test response
"""
return str(test)
def patch(test):
"""
description:
returns the path parameter
responses:
200:
description: test response
"""
return str(test)
def put(test):
"""
description:
returns the path parameter
responses:
200:
description: test response
"""
return str(test)
def delete(test):
"""
description:
returns the path parameter
responses:
200:
description: test response
"""
return str(test)

View File

@ -1,6 +1,21 @@
from uuid import uuid4
from halfapi.lib import acl
ACLS = {
'GET' : [{'acl':acl.public}]
}
def get():
raise NotImplementedError
"""
description: The pinnochio guy
responses:
200:
description: test response
content:
application/json:
schema:
$ref: "#/components/schemas/Pinnochio"
"""
return {
'id': str(uuid4()),
'name': 'pinnochio',
'nose_size': 42
}

View File

@ -0,0 +1,79 @@
from ... import acl
from halfapi.logging import logger
ACLS = {
'GET' : [
{
'acl':acl.public,
'args': {
'required': {
'foo', 'bar'
},
'optional': {
'x'
}
}
},
{
'acl':acl.random,
'args': {
'required': {
'foo', 'baz'
},
'optional': {
'truebidoo'
}
}
},
],
'POST' : [
{
'acl':acl.private,
'args': {
'required': {
'foo', 'bar'
},
'optional': {
'x'
}
}
},
{
'acl':acl.public,
'args': {
'required': {
'foo', 'baz'
},
'optional': {
'truebidoo',
'z'
}
}
},
]
}
def get(data):
"""
description:
returns the arguments passed in
responses:
200:
description: test response
"""
logger.error('%s', data['foo'])
return data
def post(data):
"""
description:
returns the arguments passed in
responses:
200:
description: test response
"""
logger.error('%s', data)
return data

View File

@ -0,0 +1,69 @@
from halfapi.lib.responses import ORJSONResponse, NotImplementedResponse
from ... import acl
ROUTES = {
'abc/alphabet/{test:uuid}': {
'GET': [{'acl': acl.public}]
},
'abc/pinnochio': {
'GET': [{'acl': acl.public}]
},
'config': {
'GET': [{'acl': acl.public}]
},
'arguments': {
'GET': [{
'acl': acl.public,
'args': {
'required': {'foo', 'bar'},
'optional': set()
}
}]
},
}
async def get_abc_alphabet_TEST(request, *args, **kwargs):
"""
description: Not implemented
responses:
200:
description: test response
parameters:
- name: test
in: path
description: Test parameter in route with "ROUTES" constant
required: true
schema:
type: string
"""
return NotImplementedResponse()
async def get_abc_pinnochio(request, *args, **kwargs):
"""
description: Not implemented
responses:
200:
description: test response
"""
return NotImplementedResponse()
async def get_config(request, *args, **kwargs):
"""
description: Not implemented
responses:
200:
description: test response
"""
return NotImplementedResponse()
async def get_arguments(request, *args, **kwargs):
"""
description: Liste des datatypes.
responses:
200:
description: test response
"""
return ORJSONResponse({
'foo': kwargs.get('data').get('foo'),
'bar': kwargs.get('data').get('bar')
})

View File

@ -1,11 +1,33 @@
from halfapi.lib import acl
import logging
logger = logging.getLogger('uvicorn.asgi')
from ... import acl
from halfapi.logging import logger
ACLS = {
'GET' : [{'acl':acl.public}]
'GET' : [
{'acl':acl.public},
{'acl':acl.random},
]
}
def get(halfapi):
"""
description:
returns the configuration of the domain
responses:
200:
description: test response
"""
logger.error('%s', halfapi)
# TODO: Remove in 0.7.0
try:
assert 'test' in halfapi['config']['domain']['dummy_domain']['config']
except AssertionError as exc:
logger.error('No TEST in halfapi[config][domain][dummy_domain][config]')
raise exc
try:
assert 'test' in halfapi['config']
except AssertionError as exc:
logger.error('No TEST in halfapi[config]')
raise exc
return halfapi['config']

View File

@ -0,0 +1,8 @@
param_docstring = """
name: second
in: path
description: second parameter description test
required: true
schema:
type: string
"""

View File

@ -0,0 +1,20 @@
from uuid import UUID
from halfapi.lib import acl
ACLS = {
'GET': [{'acl': acl.public}]
}
def get(first, second, third):
"""
description: a Test route for path parameters
responses:
200:
description: The test passed!
500:
description: The test did not pass :(
"""
assert isintance(first, str)
assert isintance(second, UUID)
assert isintance(third, int)
return ''

View File

@ -0,0 +1,13 @@
from halfapi.lib import acl
ACLS = {
'GET': [{'acl':acl.public}]
}
def get(ext, ret_type='html'):
"""
responses:
200:
description: dummy abc.alphabet route
"""
return '\n'.join(('trololo', '', 'ololotr'))

View File

@ -0,0 +1,31 @@
from halfapi.lib import acl
ACLS = {
'GET': [{'acl':acl.public}],
'POST': [
{
'acl':acl.public,
'args': {
'required': {'trou'},
'optional': {'troo'}
}
}
]
}
def get(ext, halfapi={}, ret_type='html'):
"""
responses:
200:
description: dummy abc.alphabet route
"""
return '\n'.join(('trololo', '', 'ololotr'))
def post(ext, data={'troo': 'fidget'}, halfapi={}, ret_type='html'):
"""
responses:
200:
description: dummy abc.alphabet route
"""
print(data)
return '\n'.join(('trololo', '', 'ololotr'))

View File

@ -0,0 +1,13 @@
from halfapi.lib import acl
ACLS = {
'GET': [{'acl':acl.public}]
}
def get(ret_type='html'):
"""
responses:
200:
description: dummy abc.alphabet route
"""
return '\n'.join(('trololo', '', 'ololotr'))

View File

@ -0,0 +1,4 @@
from . import get
def test_get():
assert isinstance(get(), str)

View File

@ -1,7 +1,7 @@
import pytest
from starlette.responses import PlainTextResponse
from starlette.testclient import TestClient
from halfapi.lib.routes import route_acl_decorator
from halfapi.half_route import HalfRoute
from halfapi.lib import acl
def test_acl_Check(dummy_app, token_debug_false_builder):
@ -9,7 +9,7 @@ def test_acl_Check(dummy_app, token_debug_false_builder):
A request with ?check should always return a 200 status code
"""
@route_acl_decorator(params=[{'acl':acl.public}])
@HalfRoute.acl_decorator(params=[{'acl':acl.public}])
async def test_route_public(request, **kwargs):
raise Exception('Should not raise')
return PlainTextResponse('ok')
@ -17,10 +17,10 @@ def test_acl_Check(dummy_app, token_debug_false_builder):
dummy_app.add_route('/test_public', test_route_public)
test_client = TestClient(dummy_app)
resp = test_client.get('/test_public?check')
resp = test_client.request('get', '/test_public?check')
assert resp.status_code == 200
@route_acl_decorator(params=[{'acl':acl.private}])
@HalfRoute.acl_decorator(params=[{'acl':acl.private}])
async def test_route_private(request, **kwargs):
raise Exception('Should not raise')
return PlainTextResponse('ok')
@ -28,10 +28,10 @@ def test_acl_Check(dummy_app, token_debug_false_builder):
dummy_app.add_route('/test_private', test_route_private)
test_client = TestClient(dummy_app)
resp = test_client.get('/test_private')
resp = test_client.request('get', '/test_private')
assert resp.status_code == 401
resp = test_client.get('/test_private?check')
resp = test_client.request('get', '/test_private?check')
assert resp.status_code == 200

15
tests/test_app.py Normal file
View File

@ -0,0 +1,15 @@
import os
from starlette.applications import Starlette
from unittest.mock import MagicMock, patch
from halfapi.halfapi import HalfAPI
from halfapi.lib.domain import NoDomainsException
def test_halfapi_dummy_domain(dummy_domain):
with patch('starlette.applications.Starlette') as mock:
mock.return_value = MagicMock()
config = {}
config['domain'] = {}
config['domain'][dummy_domain['name']] = dummy_domain
print(config)
halfapi = HalfAPI(config)

View File

@ -1,92 +0,0 @@
#!/usr/bin/env python3
import os
import subprocess
import importlib
import tempfile
from unittest.mock import patch
import pytest
from click.testing import CliRunner
from configparser import ConfigParser
from halfapi import __version__
from halfapi.cli import cli
Cli = cli.cli
PROJNAME = os.environ.get('PROJ','tmp_api')
@pytest.mark.incremental
class TestCli():
def test_options(self, runner):
# Wrong command
with runner.isolated_filesystem():
r = runner.invoke(Cli, ['foobar'])
assert r.exit_code == 2
# Test existing commands
with runner.isolated_filesystem():
r = runner.invoke(Cli, ['--help'])
assert r.exit_code == 0
with runner.isolated_filesystem():
r = runner.invoke(Cli, ['--version'])
assert r.exit_code == 0
with runner.isolated_filesystem():
r = runner.invoke(Cli, ['init', '--help'])
assert r.exit_code == 0
def test_init_project_fail(self, runner):
# Missing argument (project)
testproject = 'testproject'
r = runner.invoke(Cli, ['init'])
assert r.exit_code == 2
with runner.isolated_filesystem():
# Fail : Wrong project name
r = runner.invoke(Cli, ['init', 'test*-project'])
assert r.exit_code == 1
with runner.isolated_filesystem():
# Fail : Already existing folder
os.mkdir(testproject)
r = runner.invoke(Cli, ['init', testproject])
assert r.exit_code == 1
with runner.isolated_filesystem():
# Fail : Already existing nod
os.mknod(testproject)
r = runner.invoke(Cli, ['init', testproject])
assert r.exit_code == 1
def test_init_project(self, runner, halfapi_conf_dir):
"""
"""
cp = ConfigParser()
with runner.isolated_filesystem():
env = {
'HALFAPI_CONF_DIR': halfapi_conf_dir
}
res = runner.invoke(Cli, ['init', PROJNAME], env=env)
try:
assert os.path.isdir(PROJNAME)
assert os.path.isdir(os.path.join(PROJNAME, '.halfapi'))
# .halfapi/config check
assert os.path.isfile(os.path.join(PROJNAME, '.halfapi', 'config'))
cp.read(os.path.join(PROJNAME, '.halfapi', 'config'))
assert cp.has_section('project')
assert cp.has_option('project', 'name')
assert cp.get('project', 'name') == PROJNAME
assert cp.get('project', 'halfapi_version') == __version__
assert cp.has_section('domains')
except AssertionError as exc:
subprocess.run(['tree', '-a', os.getcwd()])
raise exc
assert res.exit_code == 0
assert res.exception is None

View File

@ -1,35 +0,0 @@
#!/usr/bin/env python3
import os
import subprocess
import importlib
import tempfile
from unittest.mock import patch
import pytest
from click.testing import CliRunner
from configparser import ConfigParser
PROJNAME = os.environ.get('PROJ','tmp_api')
@pytest.mark.incremental
class TestCliProj():
def test_cmds(self, project_runner):
assert project_runner('--help').exit_code == 0
#assert project_runner('run', '--help').exit_code == 0
#assert project_runner('domain', '--help').exit_code == 0
def test_domain_commands(self, project_runner):
r = project_runner('domain')
assert r.exit_code == 0
@pytest.mark.skip
def test_config_commands(self, project_runner):
try:
r = project_runner('config')
assert r.exit_code == 0
except AssertionError as exc:
subprocess.call(['tree', '-a'])
raise exc

52
tests/test_conf.py Normal file
View File

@ -0,0 +1,52 @@
from unittest import TestCase
import sys
import pytest
from halfapi.halfapi import HalfAPI
class TestConf(TestCase):
def setUp(self):
self.args = {
'domain': {
'dummy_domain': {
'name': 'dummy_domain',
'router': '.routers',
'enabled': True,
'prefix': False,
}
}
}
def tearDown(self):
pass
def test_conf_production_default(self):
halfapi = HalfAPI({
**self.args
})
assert halfapi.PRODUCTION is True
def test_conf_production_true(self):
halfapi = HalfAPI({
**self.args,
'production': True,
})
assert halfapi.PRODUCTION is True
def test_conf_production_false(self):
halfapi = HalfAPI({
**self.args,
'production': False,
})
assert halfapi.PRODUCTION is False
def test_conf_variables(self):
from halfapi.conf import (
CONFIG,
SCHEMA,
)
assert isinstance(CONFIG, dict)
assert isinstance(CONFIG.get('project_name'), str)
assert isinstance(SCHEMA, dict)
assert isinstance(CONFIG.get('secret'), str)
assert isinstance(CONFIG.get('host'), str)
assert isinstance(CONFIG.get('port'), int)

14
tests/test_constants.py Normal file
View File

@ -0,0 +1,14 @@
from schema import Schema
def test_constants():
from halfapi.lib.constants import (
VERBS,
ACLS_SCHEMA,
ROUTE_SCHEMA,
DOMAIN_SCHEMA,
API_SCHEMA)
assert isinstance(VERBS, tuple)
assert isinstance(ACLS_SCHEMA, Schema)
assert isinstance(ROUTE_SCHEMA, Schema)
assert isinstance(DOMAIN_SCHEMA, Schema)
assert isinstance(API_SCHEMA, Schema)

View File

@ -2,33 +2,66 @@
import pytest
from starlette.authentication import UnauthenticatedUser
from starlette.testclient import TestClient
import subprocess
import json
import os
import sys
import pprint
import openapi_spec_validator
import logging
logger = logging.getLogger()
from halfapi.lib.constants import API_SCHEMA
def test_whoami(project_runner, application_debug):
# @TODO : test with fake login
def test_halfapi_whoami(application_debug):
# @TODO : If we use isolated filesystem multiple times that creates a bug.
# So we use a single function with fixture "application debug"
c = TestClient(application_debug)
r = c.get('/halfapi/whoami')
r = c.request('get', '/halfapi/whoami')
assert r.status_code == 200
def test_log(application_debug):
def test_halfapi_log(application_debug):
# @TODO : If we use isolated filesystem multiple times that creates a bug.
# So we use a single function with fixture "application debug"
c = TestClient(application_debug)
r = c.get('/halfapi/log')
r = c.request('get', '/halfapi/log')
assert r.status_code == 200
def test_error(application_debug):
def test_halfapi_error_400(application_debug):
# @TODO : If we use isolated filesystem multiple times that creates a bug.
# So we use a single function with fixture "application debug"
c = TestClient(application_debug)
r = c.get('/halfapi/error/400')
r = c.request('get', '/halfapi/error/400')
assert r.status_code == 400
r = c.get('/halfapi/error/404')
def test_halfapi_error_404(application_debug):
# @TODO : If we use isolated filesystem multiple times that creates a bug.
# So we use a single function with fixture "application debug"
c = TestClient(application_debug)
r = c.request('get', '/halfapi/error/404')
assert r.status_code == 404
r = c.get('/halfapi/error/500')
def test_halfapi_error_500(application_debug):
# @TODO : If we use isolated filesystem multiple times that creates a bug.
# So we use a single function with fixture "application debug"
c = TestClient(application_debug)
r = c.request('get', '/halfapi/error/500')
assert r.status_code == 500
def test_exception(application_debug):
def test_schema(application_debug):
c = TestClient(application_debug)
try:
r = c.get('/halfapi/exception')
assert r.status_code == 500
except Exception:
print('exception')
r = c.request('get', '/')
schema = r.json()
assert isinstance(schema, dict)
openapi_spec_validator.validate_spec(schema)

106
tests/test_domain.py Normal file
View File

@ -0,0 +1,106 @@
import pytest
from halfapi.testing.test_domain import TestDomain
from pprint import pprint
import logging
logger = logging.getLogger()
class TestDummyDomain(TestDomain):
from .dummy_domain import domain
__name__ = domain.get('name')
__routers__ = domain.get('routers')
DOMAIN = __name__
CONFIG = {'test': True}
def test_domain(self):
self.check_domain()
def test_routes(self):
self.check_routes()
def test_html_route(self):
res = self.client.request('get', '/ret_type')
assert res.status_code == 200
assert isinstance(res.content.decode(), str)
assert res.headers['content-type'].split(';')[0] == 'text/html'
res = self.client.request('get', '/ret_type/h24')
assert res.status_code == 200
assert isinstance(res.content.decode(), str)
assert res.headers['content-type'].split(';')[0] == 'text/html'
res = self.client.request('get', '/ret_type/h24/config')
assert res.status_code == 200
assert isinstance(res.content.decode(), str)
assert res.headers['content-type'].split(';')[0] == 'text/html'
res = self.client.request('post', '/ret_type/h24/config', json={
'trou': 'glet'
})
assert res.status_code == 200
assert isinstance(res.content.decode(), str)
assert res.headers['content-type'].split(';')[0] == 'text/html'
def test_arguments__get_routes(self):
res = self.client.request('post', '/arguments?foo=1&x=3')
assert res.status_code == 400
arg_dict = {'foo': '1', 'bar': '2', 'x': '3'}
res = self.client.request('get', '/arguments?foo=1&bar=2&x=3')
assert res.json() == arg_dict
res = self.client.request('get', '/arguments?foo=1&bar=2&x=3&y=4')
assert res.json() == arg_dict
def test_arguments_post_routes(self):
arg_dict = {}
res = self.client.request('post', '/arguments', json=arg_dict)
assert res.status_code == 400
arg_dict = {'foo': '1', 'bar': '3'}
res = self.client.request('post', '/arguments', json=arg_dict)
assert res.status_code == 400
arg_dict = {'foo': '1', 'baz': '3'}
res = self.client.request('post', '/arguments', json=arg_dict)
assert res.json() == arg_dict
arg_dict = {'foo': '1', 'baz': '3', 'truebidoo': '4'}
res = self.client.request('post', '/arguments', json=arg_dict)
assert res.json() == arg_dict
res = self.client.request('post', '/arguments', json={ **arg_dict, 'y': '4'})
assert res.json() == arg_dict
res = self.client.request('post', '/arguments', json={ **arg_dict, 'z': True})
assert res.json() == {**arg_dict, 'z': True}
def test_schema_path_params(self):
res = self.client.request('get', '/halfapi/schema')
schema = res.json()
logger.debug(schema)
assert len(schema['paths']) > 0
route = schema['paths']['/path_params/{first}/one/{second}/two/{third}']
assert 'parameters' in route['get']
parameters = route['get']['parameters']
assert len(parameters) == 3
param_map = {
elt['name']: elt
for elt in parameters
}
assert param_map['second']['description'] == 'second parameter description test'

View File

@ -0,0 +1,19 @@
import importlib
from halfapi.testing.test_domain import TestDomain
def test_dummy_domain():
from . import dummy_domain
from .dummy_domain import acl
assert acl.public() is True
assert isinstance(acl.random(), int)
assert acl.private() is False
from .dummy_domain import routers
from .dummy_domain.routers.arguments import get
from .dummy_domain.routers.abc.alphabet.TEST_uuid import get
from .dummy_domain.routers.abc.pinnochio import get
from .dummy_domain.routers.config import get
from .dummy_domain.routers.config import get
from .dummy_domain.routers import async_router
from .dummy_domain.routers.async_router import ROUTES, get_abc_alphabet_TEST, get_abc_pinnochio, get_config, get_arguments

View File

@ -4,51 +4,106 @@ import importlib
import subprocess
import time
import pytest
import json
from pprint import pprint
from starlette.routing import Route
from starlette.testclient import TestClient
from halfapi.lib.domain import gen_router_routes
def test_get_config_route(dummy_project, application_domain, routers):
def test_get_config_route(dummy_project, application_domain):
c = TestClient(application_domain)
r = c.get('/dummy_domain/config')
r = c.request('get', '/')
assert r.status_code == 200
pprint(r.json())
r = c.request('get', '/config')
assert r.status_code == 200
pprint(r.json())
assert 'test' in r.json()
def test_get_route(dummy_project, application_domain, routers):
def test_get_route(dummy_project, application_domain):
c = TestClient(application_domain)
path = verb = params = None
for path, verb, _, params in gen_router_routes(routers, []):
if len(params):
route_path = '/dummy_domain/{}'.format(path)
dummy_domain_routes = [
('config','GET'),
('config','GET'),
('async_router/abc/pinnochio','GET'),
('async_router/config','GET'),
# ('abc/pinnochio','GET'),
# ('abc/alphabet','GET'),
]
for route_def in dummy_domain_routes:
path, verb = route_def[0], route_def[1]
route_path = '/{}'.format(path)
print(route_path)
try:
if verb.lower() == 'get':
r = c.request('get', route_path)
elif verb.lower() == 'post':
r = c.request('post', route_path)
elif verb.lower() == 'patch':
r = c.request('patch', route_path)
elif verb.lower() == 'put':
r = c.request('put', route_path)
elif verb.lower() == 'delete':
r = c.request('delete', route_path)
else:
raise Exception(verb)
try:
if verb.lower() == 'get':
r = c.get(route_path)
elif verb.lower() == 'post':
r = c.post(route_path)
elif verb.lower() == 'patch':
r = c.patch(route_path)
elif verb.lower() == 'put':
r = c.put(route_path)
elif verb.lower() == 'delete':
r = c.delete(route_path)
else:
raise Exception(verb)
try:
assert r.status_code in [200, 501]
except AssertionError as exc:
print('{} [{}] {}'.format(str(r.status_code), verb, route_path))
assert r.status_code in [200, 501]
except AssertionError as exc:
print('{} [{}] {}'.format(str(r.status_code), verb, route_path))
raise exc from exc
except NotImplementedError:
pass
except NotImplementedError:
pass
if not path:
raise Exception('No route generated')
dummy_domain_path_routes = [
('abc/alphabet/{test}','GET'),
]
#for route_def in dummy_domain_path_routes:
for route_def in []:#dummy_domain_routes:
from uuid import uuid4
test_uuid = uuid4()
for route_def in dummy_domain_path_routes:
path, verb = route_def[0], route_def[1]
path = path.format(test=str(test_uuid))
route_path = f'/{path}'
if verb.lower() == 'get':
r = c.request('get', f'{route_path}')
assert r.status_code == 200
def test_delete_route(dummy_project, application_domain, routers):
def test_delete_route(dummy_project, application_domain):
c = TestClient(application_domain)
from uuid import uuid4
arg = str(uuid4())
r = c.delete(f'/dummy_domain/abc/alphabet/{arg}')
r = c.request('delete', f'/abc/alphabet/{arg}')
assert r.status_code == 200
assert r.json() == arg
assert isinstance(r.json(), str)
def test_arguments_route(dummy_project, application_domain):
c = TestClient(application_domain)
path = '/arguments'
r = c.request('get', path)
assert r.status_code == 400
r = c.request('get', path, params={'foo':True})
assert r.status_code == 400
arg = {'foo':True, 'bar':True}
r = c.request('get', path, params=arg)
assert r.status_code == 200
for key, val in arg.items():
assert json.loads(r.json()[key]) == val
path = '/async_router/arguments'
r = c.request('get', path)
assert r.status_code == 400
r = c.request('get', path, params={'foo':True})
assert r.status_code == 400
arg = {'foo':True, 'bar':True}
r = c.request('get', path, params=arg)
assert r.status_code == 200
for key, val in arg.items():
assert json.loads(r.json()[key]) == val

6
tests/test_halfapi.py Normal file
View File

@ -0,0 +1,6 @@
from halfapi.halfapi import HalfAPI
def test_methods():
assert 'application' in dir(HalfAPI)
assert 'version' in dir(HalfAPI)
assert 'version_async' in dir(HalfAPI)

View File

@ -38,7 +38,7 @@ def test_jwt_NoToken(dummy_app):
dummy_app.add_route('/test', test_route)
test_client = TestClient(dummy_app)
resp = test_client.get('/test')
resp = test_client.request('get', '/test')
assert resp.status_code == 200
def test_jwt_Token(dummy_app, token_builder):
@ -50,13 +50,20 @@ def test_jwt_Token(dummy_app, token_builder):
dummy_app.add_route('/test', test_route)
test_client = TestClient(dummy_app)
resp = test_client.get('/test',
resp = test_client.request('get', '/test',
cookies={
'Authorization': token_builder
})
assert resp.status_code == 200
resp = test_client.request('get', '/test',
headers={
'Authorization': token_builder
})
assert resp.status_code == 200
def test_jwt_DebugFalse(dummy_app, token_debug_false_builder):
async def test_route(request):
assert isinstance(request.user, JWTUser)
@ -65,7 +72,13 @@ def test_jwt_DebugFalse(dummy_app, token_debug_false_builder):
dummy_app.add_route('/test', test_route)
test_client = TestClient(dummy_app)
resp = test_client.get('/test',
resp = test_client.request('get', '/test',
cookies={
'Authorization': token_debug_false_builder
})
assert resp.status_code == 200
resp = test_client.request('get', '/test',
headers={
'Authorization': token_debug_false_builder
})
@ -82,7 +95,13 @@ def test_jwt_DebugTrue(dummy_app, token_debug_true_builder):
dummy_app.add_route('/test', test_route)
test_client = TestClient(dummy_app)
resp = test_client.get('/test',
resp = test_client.request('get', '/test',
cookies={
'Authorization': token_debug_true_builder
})
assert resp.status_code == 400
resp = test_client.request('get', '/test',
headers={
'Authorization': token_debug_true_builder
})
@ -100,8 +119,14 @@ def test_jwt_DebugTrue_DebugApp(dummy_debug_app, token_debug_true_builder):
dummy_debug_app.add_route('/test', test_route)
test_client = TestClient(dummy_debug_app)
resp = test_client.get('/test',
headers={
'Authorization': token_debug_true_builder
resp = test_client.request('get', '/test',
cookies={
'Authorization': token_debug_true_builder
})
assert resp.status_code == 200
resp = test_client.request('get', '/test',
headers={
'Authorization': token_debug_true_builder
})
assert resp.status_code == 200

View File

@ -1,36 +1,124 @@
#!/usr/bin/env python3
import importlib
from halfapi.lib.domain import VERBS, gen_routes, gen_router_routes, MissingAclError
# #!/usr/bin/env python3
# import importlib
# from halfapi.lib.domain import VERBS, gen_routes, gen_router_routes, \
#  MissingAclError, domain_schema_dict, domain_schema_list
# 
# from types import FunctionType
# 
# 
# def test_gen_router_routes():
#  from .dummy_domain import routers
#  for path, verb, m_router, fct, params in gen_router_routes(routers, ['dummy_domain']):
#  assert isinstance(path, str)
#  assert verb in VERBS
#  assert len(params) > 0
#  assert hasattr(fct, '__call__')
#  assert len(m_router.__file__) > 0
# 
# 
# def test_gen_routes():
#  from .dummy_domain.routers.abc.alphabet import TEST_uuid
#  try:
#  gen_routes(
#  TEST_uuid,
#  'get',
#  ['abc', 'alphabet', 'TEST_uuid', ''],
#  [])
#  except MissingAclError:
#  assert True
# 
#  fct, params = gen_routes(
#  TEST_uuid,
#  'get',
#  ['abc', 'alphabet', 'TEST_uuid', ''],
#  TEST_uuid.ACLS['GET'])
# 
#  assert isinstance(fct, FunctionType)
#  assert isinstance(params, list)
#  assert len(TEST_uuid.ACLS['GET']) == len(params)
# 
# def test_domain_schema_dict():
#  from .dummy_domain import routers
#  d_res = domain_schema_dict(routers)
# 
#  assert isinstance(d_res, dict)
# 
# def test_domain_schema_list():
#  from .dummy_domain import routers
#  res = domain_schema_list(routers)
# 
#  assert isinstance(res, list)
#  assert len(res) > 0
from types import FunctionType
from starlette.testclient import TestClient
from starlette.responses import Response
from starlette.routing import Router, Route
from halfapi.lib.domain import route_decorator
from halfapi.lib.user import Nobody
def test_gen_router_routes():
from .dummy_domain import routers
for path, verb, fct, params in gen_router_routes(routers, ['dummy_domain']):
assert isinstance(path, str)
assert verb in VERBS
assert len(params) > 0
assert hasattr(fct, '__call__')
def test_route_decorator():
""" It should decorate an async function that fullfills its arguments
"""
def route(halfapi, data, out, ret_type='txt'):
for key in ['user', 'config', 'domain', 'cookies', 'base_url', 'url']:
assert key in halfapi
def test_gen_routes():
from .dummy_domain.routers.abc.alphabet import TEST_uuid
try:
gen_routes(
TEST_uuid,
'get',
['abc', 'alphabet', 'TEST_uuid', ''],
[])
except MissingAclError:
assert True
assert halfapi['user'] is None
assert isinstance(halfapi['config'], dict)
assert len(halfapi['config']) == 0
assert isinstance(halfapi['domain'], str)
assert halfapi['domain'] == 'unknown'
assert isinstance(halfapi['cookies'], dict)
assert len(halfapi['cookies']) == 0
assert len(str(halfapi['base_url'])) > 0
assert str(halfapi['base_url']) == 'http://testserver/'
assert len(str(halfapi['url'])) > 0
assert str(halfapi['url']) == 'http://testserver/'
assert isinstance(data, dict)
assert len(data) == 0
fct, params = gen_routes(
TEST_uuid,
'get',
['abc', 'alphabet', 'TEST_uuid', ''],
TEST_uuid.ACLS['GET'])
assert out is None
assert isinstance(fct, FunctionType)
assert isinstance(params, list)
assert len(TEST_uuid.ACLS['GET']) == len(params)
assert ret_type is 'txt'
return ''
async_route = route_decorator(route)
app = Router([Route('/', endpoint=async_route, methods=['GET'])])
client = TestClient(app)
response = client.get('/')
assert response.is_success
assert response.content.decode() == ''
def route(data, out, ret_type='txt'):
assert isinstance(data, dict)
assert len(data) == 0
assert out is None
assert ret_type is 'txt'
return ''
async_route = route_decorator(route)
app = Router([Route('/', endpoint=async_route, methods=['GET'])])
client = TestClient(app)
response = client.get('/')
assert response.is_success
assert response.content.decode() == ''
def route(data):
assert isinstance(data, dict)
assert len(data) == 2
assert data['toto'] == 'tata'
assert data['bouboul'] == True
return ''
async_route = route_decorator(route)
app = Router([Route('/', endpoint=async_route, methods=['POST'])])
client = TestClient(app)
response = client.post('/', json={'toto': 'tata', 'bouboul': True})
assert response.is_success
assert response.json() == ''

View File

@ -0,0 +1,38 @@
from starlette.testclient import TestClient
from starlette.middleware.base import BaseHTTPMiddleware
from unittest.mock import patch
from halfapi.lib.domain_middleware import DomainMiddleware
def test_init():
with patch('starlette.middleware.base.BaseHTTPMiddleware.__init__') as init:
mw = DomainMiddleware('app', 'domain')
init.assert_called_once_with('app')
assert mw.domain == 'domain'
assert mw.request == None
def test_call(application_debug):
c = TestClient(application_debug)
r = c.request('get', '/abc/alphabet')
assert r.status_code == 200
assert r.headers['x-domain'] == 'dummy_domain'
assert r.headers['x-acl'] == 'public'
r = c.request('get', '/arguments')
assert r.status_code == 400
assert r.headers['x-domain'] == 'dummy_domain'
assert r.headers['x-acl'] == 'public'
assert 'foo' in r.headers['x-args-required'].split(',')
assert 'bar' in r.headers['x-args-required'].split(',')
assert r.headers['x-args-optional'] == 'x'
c = TestClient(application_debug)
r = c.request('post', '/arguments')
assert r.status_code == 400
assert r.headers['x-domain'] == 'dummy_domain'
assert r.headers['x-acl'] == 'public'
assert 'foo' in r.headers['x-args-required'].split(',')
assert 'baz' in r.headers['x-args-required'].split(',')
assert 'truebidoo' in r.headers['x-args-optional'].split(',')
assert 'z' in r.headers['x-args-optional'].split(',')

View File

@ -1,44 +1,57 @@
import os
from halfapi.lib.router import read_router
def test_read_router_routers():
from .dummy_domain import routers
router_d = read_router(routers)
assert '' in router_d
assert 'SUBROUTES' in router_d['']
assert isinstance(router_d['']['SUBROUTES'], list)
for elt in os.scandir(routers.__path__[0]):
if elt.is_dir():
assert elt.name in router_d['']['SUBROUTES']
def test_read_router_abc():
from .dummy_domain.routers import abc
router_d = read_router(abc)
assert '' in router_d
assert 'SUBROUTES' in router_d['']
assert isinstance(router_d['']['SUBROUTES'], list)
def test_read_router_alphabet():
from .dummy_domain.routers.abc import alphabet
router_d = read_router(alphabet)
assert '' in router_d
assert 'SUBROUTES' in router_d['']
assert isinstance(router_d['']['SUBROUTES'], list)
def test_read_router_TEST():
from .dummy_domain.routers.abc.alphabet import TEST_uuid
router_d = read_router(TEST_uuid)
print(router_d)
assert '' in router_d
assert 'SUBROUTES' in router_d['']
assert isinstance(router_d['']['GET'], list)
assert isinstance(router_d['']['POST'], list)
assert isinstance(router_d['']['PATCH'], list)
assert isinstance(router_d['']['PUT'], list)
# import os
# import pytest
# from schema import SchemaError
# from halfapi.lib.router import read_router
# from halfapi.lib.constants import ROUTER_SCHEMA, ROUTER_ACLS_SCHEMA
# 
# def test_read_router_routers():
#  from .dummy_domain import routers
# 
#  router_d = read_router(routers)
#  assert '' in router_d
#  assert 'SUBROUTES' in router_d['']
#  assert isinstance(router_d['']['SUBROUTES'], list)
# 
#  for elt in os.scandir(routers.__path__[0]):
#  if elt.is_dir():
#  assert elt.name in router_d['']['SUBROUTES']
# 
# def test_read_router_abc():
#  from .dummy_domain.routers import abc
#  router_d = read_router(abc)
# 
#  assert '' in router_d
#  assert 'SUBROUTES' in router_d['']
#  assert isinstance(router_d['']['SUBROUTES'], list)
# 
# def test_read_router_alphabet():
#  from .dummy_domain.routers.abc import alphabet
#  router_d = read_router(alphabet)
# 
#  assert '' in router_d
#  assert 'SUBROUTES' in router_d['']
#  assert isinstance(router_d['']['SUBROUTES'], list)
# 
#  ROUTER_SCHEMA.validate(router_d)
# 
#  with pytest.raises(SchemaError):
#  """ Test that we cannot specify wrong method in ROUTES or ACLS
# 
#  TODO: Write more errors
#  """
#  router_d['']['TEG'] = {}
#  ROUTER_SCHEMA.validate(router_d)
# 
# def test_read_router_TEST():
#  from .dummy_domain.routers.abc.alphabet import TEST_uuid
#  router_d = read_router(TEST_uuid)
# 
#  print(router_d)
#  assert '' in router_d
#  assert 'SUBROUTES' in router_d['']
#  assert isinstance(router_d['']['GET'], list)
#  assert isinstance(router_d['']['POST'], list)
#  assert isinstance(router_d['']['PATCH'], list)
#  assert isinstance(router_d['']['PUT'], list)
# 
# 

View File

@ -1,9 +1,34 @@
from starlette.routing import Route
from halfapi.lib.routes import gen_starlette_routes
def test_gen_starlette_routes():
from .dummy_domain import routers
for route in gen_starlette_routes({
'dummy_domain': routers }):
assert isinstance(route, Route)
# from starlette.routing import Route
# from halfapi.lib.routes import gen_starlette_routes, gen_router_routes
# 
# def test_gen_starlette_routes():
#  from .dummy_domain import routers
#  for route in gen_starlette_routes({
#  'dummy_domain': routers }):
# 
#  assert isinstance(route, Route)
# 
# import pytest
# 
# @pytest.mark.skip
# def test_api_routes():
#  from . import dummy_domain
#  d_res, d_acls = api_routes(dummy_domain)
#  assert isinstance(d_res, dict)
#  assert isinstance(d_acls, dict)
# 
#  yielded = False
# 
#  for path, verb, m_router, fct, params in gen_router_routes(dummy_domain, []):
#  if not yielded:
#  yielded = True
# 
#  assert path in d_res
#  assert verb in d_res[path]
#  assert 'docs' in d_res[path][verb]
#  assert 'acls' in d_res[path][verb]
#  assert isinstance(d_res[path][verb]['docs'], dict)
#  assert isinstance(d_res[path][verb]['acls'], list)
#  assert len(d_res[path][verb]['acls']) == len(params)
# 
#  assert yielded is True

View File

@ -1,47 +0,0 @@
import subprocess
from pprint import pprint
from starlette.testclient import TestClient
from starlette.authentication import (
AuthenticationBackend, AuthenticationError, BaseUser, AuthCredentials,
UnauthenticatedUser)
from halfapi.lib.schemas import schema_dict_dom
from halfapi import __version__
def test_schemas_dict_dom():
from .dummy_domain import routers
schema = schema_dict_dom({
'dummy_domain':routers})
assert isinstance(schema, dict)
def test_get_api_routes(project_runner, application_debug):
c = TestClient(application_debug)
r = c.get('/')
d_r = r.json()
assert isinstance(d_r, dict)
def test_get_schema_route(project_runner, application_debug):
c = TestClient(application_debug)
r = c.get('/halfapi/schema')
d_r = r.json()
assert isinstance(d_r, dict)
assert 'openapi' in d_r.keys()
assert 'info' in d_r.keys()
assert d_r['info']['title'] == 'HalfAPI'
assert d_r['info']['version'] == __version__
assert 'paths' in d_r.keys()
def test_get_api_dummy_domain_routes(application_domain, routers):
c = TestClient(application_domain)
r = c.get('/dummy_domain')
assert r.status_code == 200
d_r = r.json()
assert isinstance(d_r, dict)
assert 'abc/alphabet' in d_r
assert 'GET' in d_r['abc/alphabet']
assert len(d_r['abc/alphabet']['GET']) > 0
assert 'acl' in d_r['abc/alphabet']['GET'][0]