http-monitoring

Simple http monitoring to monitor a target endpoint. Wrapped around as requests and exposing the response within REST-way using Flask. This project is made as minimally as possible without any database configuration, so for storing the data itself is only stored and pre-configured in the JSON file.

The targeted endpoint will be hit periodically with the scheduler that has been set with a duration interval of 300 seconds

This project also supports concurrent requests to handle multiple API endpoints that will be targeted with built-in Queue and Threading and also using the event emitter method when making HTTP requests while monitoring the system so we can see bottlenecks based the response time

Other than that, there's also a feature to notify you through your email if your targeted endpoint is down (with help from Mailtrap)

Limitation

• Currently it can only target one endpoint (due of i didn't develop it into async requests), though when you request the body with more than 2 targets, the response given to hit the target will only go to 1 endpoint
• Event emitter only displays stream log information without fetch the important data/additional information during requests
• after you've created a new endpoint and then directly fetch the latest status, it wont get the expected response (you need to stop the flask server first and then start it again in order to fetch the newest status)

Usage

Just install the related requirements with

pip3 install -r requirements.text

Afterwards, run the Flask server by pointing out

python3 src/app.py

Mailtrap Configuration

To be able to use and get a notification email if an endpoint is down, you need to register first with Mailtrap. then get the configuration email which can be found in 'My Inbox' menu. And afterwards, configure the email settings that you got from Mailtrap in the configuration.py file.

after that you can also set the recipient and also the sender for the email you want to send in the configuration.py file also

Configuration

You can set the HTTP configuration for monitoring the targeted endpoint by editing the default value in the configuration.py file, in the configuration dict there will be several objects :

DEFAULT_HTTP_TIMEOUT : used to set the timeout for the HTTP request process, the default is 5 seconds and there's no limit to determine the timeout

DEFAULT_HTTP_STATUS_CODE : expected status code for our target endpoint, default 200 OK

DEFAULT_HTTP_CONFIG : configuration file used to define the targeted API endpoint. Use a full-path if you are using windows, otherwise you will get an error message about no such directory error

DEFAULT_HTTP_RETRIES : configuration to retries requests if get errors such as gateway timeout, service unavailable or so on. default 3 times retries for each session with a limit of 3 times

DEFAULT_BACKOFF_FACTOR : configuration time to set the random value used before doing the next retry process, default is 0.3 and there is no limitation to set it

DEFAULT_ERROR_CODES : configuration to set any error code status that can be used for the retry process. default set to 504 or gateway timeout and can only be used for certain status codes such as 501, 502, 503 and 504. Otherwise it will raise an exception

APIs Specification

Create new endpoint that to be targeted

[POST] {base_url}/status

# request body, 201 CREATED if it was successful
{
    "targets": [
        {
            "title": "Hit the Google!",
            "endpoint": "https://www.google.com/",
            "expected_http_code": 200
        }
    ]
}

Fetch the endpoint status

[GET] {base_url}/status

# response body, 200 OK if it was successfull
{
    "category": "success",
    "data": {
        "actual_response": 200,
        "elapsed": "0.15 seconds",
        "endpoint": "https://www.google.com/",
        "expected_response": 200,
        "is_down": false,
        "is_up": true,
        "timestamp": "2023-01-24 12:32:41",
        "title": null
    },
    "message": "Success get the response"
}

Set numbers of worker and size of queue (WIP)

[POST] {base_url}/status/queue

# example request body to set the numbers
{
    "worker_thread": 2,
    "queue_size": 4
}

if you only set it to 0, then the value of worker and size of queue by default will be set at 5 and 3

Retrieve queue status (WIP)

[GET] {base_url}/status/queue

# response body if it was successful
{
    "category": "success",
    "data": {
        "end_time": "0.00 seconds",
        "queue_status": false,
        "start_time": "1674723897.10 seconds"
    },
    "message": "Success get the response"
}

(Manually) trigger send the email notifications

Currently, the behavior for triggering notifications to emails you still need to hit the API, in the meantime this will be overhauled and wrapped into a single specific endpoint

[GET] {base_url}/emails

# response body if it was successfull, 200 OK
{
    "category": "email_sent_success",
    "data": null,
    "message": "Success to send the notifications to related recipients"
}

Enhancements

There will be lots of improvements or enhancements that can be developed in this project, one of which is to make it with an async background or target endpoints based on the queue method using the redis queue.

Future works, can also replace the data storing system by using a database rather than just a JSON file

Otherwise, the improvements also can be done for email notification by using background process, maybe we can use Celery and Redis for this one but i think it might be overkill to implement it (lol)