Scrapy is a great web scraping framework, but it lacks a good logging setup. In this short blog post, I'll show you how to use Structlog with Scrapy.
Most Popular Blog Tags
Custom Python Logging For Scrapy
3 min read
Scrapy is a great web scraping framework, but it lacks a good logging setup. In this short blog post, I'll show you how to use Structlog with Scrapy.
Ingress-Nginx Monitoring with Prometheus and Grafana
4 min read
Ingress-nginx provides an easy integration with Prometheus for monitoring. However, it can be challenging to get started with monitoring Ingress-nginx and creating dashboards and alerts. Therefore, I've created a monitoring mixin for Ingress-nginx which will provide Prometheus alerts and Grafana …
Deploying a Django Channels ASGI Server to Kubernetes
6 min read
Django channels is great for adding support of various network protocols as WebSockets, chat protocols, IoT protocols and other. It's a great way to add real-time functionality to your Django application. To use channels however, requires you to deploy an …
Adding a Shell to the Django Admin
3 min read
Django's admin is an amazing batteries-included addition to the Django framework. It's a great tool for managing your data, and it is easily extendable and customizable. The same goes for Django's ./manage.py shell
command. If we were to combine these …
Sharing Development Secrets with the Team using Vault
4 min read
My most recent projects have consisted of using a microservice architecture and multiple third party services (analytics, events etc.). For better or worse, this seems to be getting more popular, even for smaller companies and startups. Local development becomes more …
ArgoCD Monitoring with Prometheus and Grafana
5 min read
ArgoCD has by default support for notifying when an Application does not have the desired status through triggers. For example, when an application becomes OutOfSync
or Unhealthy
, a notification is sent to your configured notification service (e.g. Slack). This was …
Showcase: Using Jsonnet & Mixins to Simplify Endpoint Monitoring with Blackbox-exporter
4 min read
Blackbox-exporter is a Prometheus exporter that probes endpoints and exposes metrics of the probe result. There are multiple guides on how to use the Blackbox-exporter, and we won't go into that, but rather focus on newer things as Jsonnet as …
Custom Django Error Pages
3 min read
Django comes with default views for 400, 403, 404 and 500 pages which is great. However, the default templates for these views are minimal - they just indicate what error it is. This blog post will walk through how to …
Django Development and Production Logging
12 min read
Django comes with many great built-in features, logging is one of them. Pre-configured logging setups that vary between development (debug mode) and production environments. Easy integration to send you emails on errors, out-of-the-box support for various settings as log levels …
Simple Django User Session Clearing using Celery
1 min read
Django provides session support out-of-the-box and stores sessions in the django_session
database table. Django leaves it up to the project maintainers to purge sessions in their Django project. This means that if it's not done on a regular basis the …