Website Monitoring

2012 was one of the most precious years for me. This is the first time my proposal was accepted for international Python Conference, where most of the speakers are PhD and I'm the only one who's student in the college.

So my proposal is to talk about Distributed Messages/Tasks Processing (AMQP) on Django. I use one of my previous project website monitoring as the sample project (with the simplest function).

Website Monitoring provides page change monitoring and notification services to internet users worldwide. It will create a change log for the page and alert users by email when there's a change in the page text.

Website Monitoring using Advance Message Queue Protocol (RabbitMQ as the wrapper) for task scheduling system and the website backend using asynchronous tasking process.

As we know, Django is synchronous or we usually say blocking. This means each request will not be returned until all processing (e.g., of a view) is complete. It's the expected behavior and usually required in web applications, but there are times when you need tasks to run in the background (immediately, deferred, or periodically) without blocking.

This proposal show that django can be asynchronous. which give the impression of a really web application by finishing a request as soon as possible, even though a task is running in the background, then update the page incrementally using AJAX. Executing tasks asynchronously and using retries to make sure they are completed successfully. Scheduling periodic tasks. Parallel execution and etc .. its make django look awesomeness and amazing !

Features:

-asynchronous task, handling distribute process

- Scheduling system with asynchronous tasking and checking 4h period , 12h period and 24h period

- diff change compare text content of page

- user management

- Page scraper for checking change page

- Create an API for task management and monitoring. and etc


Libraries:

- pymongo

- carrot (RabbitMQ wrapper for AMQP lib)

- difflib

- beautifulsoup

- Django-piston as Restful Web API's

1 of 4


Source code of website monitoring

https://bitbucket.org/jimmyromanticdevil/website-monitoring/src/default/

Rahmat Ramadhan
Dusun Krajan RT05 RW01, Desa Banjar
68454 Banyuwangi, Indonesia
© 2021, Built with Sanity & Gatsby · Open-source