171 lines
5.7 KiB
Markdown
171 lines
5.7 KiB
Markdown
## http-monitoring
|
|
|
|
Simple http monitoring to monitor a target endpoint. Wrapped around as
|
|
requests and exposing the response within REST-way using Flask.
|
|
This project is made as minimally as possible without any database configuration,
|
|
so for storing the data itself is only stored and pre-configured in the JSON file.
|
|
|
|
The targeted endpoint will be hit periodically with the scheduler that has been set with
|
|
a duration interval of 300 seconds
|
|
|
|
This project also supports concurrent requests to handle multiple API
|
|
endpoints that will be targeted with built-in Queue and Threading and also using
|
|
the event emitter method when making HTTP requests while monitoring the system so we
|
|
can see bottlenecks based the response time
|
|
|
|
Other than that, there's also a feature to notify you through your email if your
|
|
targeted endpoint is down (with help from [Mailtrap](https://mailtrap.io/))
|
|
|
|
**Limitation**
|
|
|
|
- Currently it can only target one endpoint (due of i didn't develop it into async requests),
|
|
though when you request the body with more than 2 targets, the response given to hit the target will only go to 1 endpoint
|
|
- Event emitter only displays stream log information without fetch the important data/additional
|
|
information during requests
|
|
- after you've created a new endpoint and then directly fetch the latest status, it wont get the
|
|
expected response (you need to stop the flask server first and then start it again in order to fetch the newest status)
|
|
|
|
### Usage
|
|
|
|
Just install the related requirements with
|
|
|
|
```shell
|
|
pip3 install -r requirements.text
|
|
```
|
|
|
|
Afterwards, run the Flask server by pointing out
|
|
|
|
```shell
|
|
python3 src/app.py
|
|
```
|
|
|
|
**Mailtrap Configuration**
|
|
|
|
To be able to use and get a notification email if an endpoint is down, you need to register first with Mailtrap.
|
|
then get the configuration email which can be found in 'My Inbox' menu. And afterwards, configure the email settings
|
|
that you got from Mailtrap in the `configuration.py` file.
|
|
|
|
after that you can also set the recipient and also the sender for the email you want to send in the `configuration.py`
|
|
file also
|
|
|
|
**Configuration**
|
|
|
|
You can set the HTTP configuration for monitoring the targeted endpoint by editing the default value in the
|
|
`configuration.py` file, in the configuration dict there will be several objects :
|
|
|
|
`DEFAULT_HTTP_TIMEOUT` : used to set the timeout for the HTTP request process, the default is 5 seconds and there's no
|
|
limit to determine the timeout
|
|
|
|
`DEFAULT_HTTP_STATUS_CODE` : expected status code for our target endpoint, default 200 OK
|
|
|
|
`DEFAULT_HTTP_CONFIG` : configuration file used to define the targeted API endpoint. Use a full-path if you are using windows,
|
|
otherwise you will get an error message about no such directory error
|
|
|
|
`DEFAULT_HTTP_RETRIES` : configuration to retries requests if get errors such as gateway timeout, service unavailable or so on.
|
|
default 3 times retries for each session with a limit of 3 times
|
|
|
|
`DEFAULT_BACKOFF_FACTOR` : configuration time to set the random value used before doing the next retry process,
|
|
default is 0.3 and there is no limitation to set it
|
|
|
|
`DEFAULT_ERROR_CODES` : configuration to set any error code status that can be used for the retry process.
|
|
default set to 504 or gateway timeout and can only be used for certain status codes such as 501, 502, 503 and 504.
|
|
Otherwise it will raise an exception
|
|
|
|
### APIs Specification
|
|
|
|
**Create new endpoint that to be targeted**
|
|
|
|
`[POST] {base_url}/status`
|
|
|
|
```shell
|
|
# request body, 201 CREATED if it was successful
|
|
{
|
|
"targets": [
|
|
{
|
|
"title": "Hit the Google!",
|
|
"endpoint": "https://www.google.com/",
|
|
"expected_http_code": 200
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
**Fetch the endpoint status**
|
|
|
|
`[GET] {base_url}/status`
|
|
|
|
```shell
|
|
# response body, 200 OK if it was successfull
|
|
{
|
|
"category": "success",
|
|
"data": {
|
|
"actual_response": 200,
|
|
"elapsed": "0.15 seconds",
|
|
"endpoint": "https://www.google.com/",
|
|
"expected_response": 200,
|
|
"is_down": false,
|
|
"is_up": true,
|
|
"timestamp": "2023-01-24 12:32:41",
|
|
"title": null
|
|
},
|
|
"message": "Success get the response"
|
|
}
|
|
```
|
|
|
|
**Set numbers of worker and size of queue (WIP)**
|
|
|
|
`[POST] {base_url}/status/queue`
|
|
|
|
```shell
|
|
# example request body to set the numbers
|
|
{
|
|
"worker_thread": 2,
|
|
"queue_size": 4
|
|
}
|
|
```
|
|
|
|
if you only set it to 0, then the value of worker and size of queue by default will be set at 5 and 3
|
|
|
|
**Retrieve queue status (WIP)**
|
|
|
|
`[GET] {base_url}/status/queue`
|
|
|
|
```shell
|
|
# response body if it was successful
|
|
{
|
|
"category": "success",
|
|
"data": {
|
|
"end_time": "0.00 seconds",
|
|
"queue_status": false,
|
|
"start_time": "1674723897.10 seconds"
|
|
},
|
|
"message": "Success get the response"
|
|
}
|
|
```
|
|
|
|
**(Manually) trigger send the email notifications**
|
|
|
|
Currently, the behavior for triggering notifications to emails you still need to hit the API,
|
|
in the meantime this will be overhauled and wrapped into a single specific endpoint
|
|
|
|
`[GET] {base_url}/emails`
|
|
|
|
```shell
|
|
# response body if it was successfull, 200 OK
|
|
{
|
|
"category": "email_sent_success",
|
|
"data": null,
|
|
"message": "Success to send the notifications to related recipients"
|
|
}
|
|
```
|
|
|
|
|
|
### Enhancements
|
|
|
|
There will be lots of improvements or enhancements that can be developed in this project, one of which is to make
|
|
it with an async background or target endpoints based on the queue method using the redis queue.
|
|
|
|
Future works, can also replace the data storing system by using a database rather than just a JSON file
|
|
|
|
Otherwise, the improvements also can be done for email notification by using background process,
|
|
maybe we can use Celery and Redis for this one but i think it might be overkill to implement it (lol) |