Data Import Monitoring
Make sure your imports and ETL run correctly.
Data imports are often business-critical. A failed sync can have significant impacts.
MoniTao Heartbeat monitors your import scripts and measures their execution time.
Import challenges
Imports can fail for many reasons:
- Source file not available or corrupted
- Database connection error
- Timeout during large imports
- Silent failure with incomplete data
The solution: Heartbeat
MoniTao Heartbeat allows you to monitor your imports and be alerted in case of problems.
Integration example
Here is a Python example with heartbeat integration:
# Python example with heartbeat
import requests
HEARTBEAT_TOKEN = "YOUR_TOKEN"
API_SECRET = "YOUR_SECRET"
BASE_URL = "https://monitao.com/api/heartbeat"
def import_data():
# Signal start
requests.post(f"{BASE_URL}/start/{HEARTBEAT_TOKEN}",
headers={"Authorization": f"Bearer {API_SECRET}"})
try:
# Your import logic here
process_csv_file()
# Signal success
requests.post(f"{BASE_URL}/ping/{HEARTBEAT_TOKEN}",
headers={"Authorization": f"Bearer {API_SECRET}"})
except Exception as e:
# Signal failure
requests.post(f"{BASE_URL}/fail/{HEARTBEAT_TOKEN}",
headers={"Authorization": f"Bearer {API_SECRET}"})
raise e
Use Cases
- Daily e-commerce order import
- CRM-ERP synchronization
- Supplier product feed retrieval
Configuration
- Create a heartbeat job with your import interval
- Use start/ping/fail based on import result
- Configure alerts appropriate to criticality
Frequently Asked Questions
How to detect an import taking too long?
Use /start at the beginning and /ping at the end. MoniTao will calculate the duration.
Can I be alerted if import exceeds a certain time?
Alerts on excessive duration are planned for a future version.
How to handle imports that can partially fail?
Send /fail if critical errors occur, /ping otherwise.
Can I send the number of imported rows?
Currently heartbeat doesn't accept additional data.
Useful Links
Ready to Sleep Soundly?
Start free, no credit card required.