If you care about performance, you may want to avoid CSV files. But since our data sources are often like our family, we can’t make a choice, we’ll see in this blog post how to process a CSV file as fast as possible.
If you care about performance, you may want to avoid CSV files. But since our data sources are often like our family, we can’t make a choice, we’ll see in this blog post how to process a CSV file as fast as possible.
structlog. Or just Structured Logging in general.
Don’t do:
logging.info(f"{something} happened!")
But do
logging.info(“thing-happened”, thing=something)
Why? Your event will become a category, which means it’s easily searchable/findable, you can output either human-readable stuff (the typical
{date}, {loglevel}, {event}) or just straight up JSONL (a JSON object/dict per line). If you have JSON logs you can usejqto query/filter/manipulate your logs, if you have something like ELK, you can insert your logs there and create dashboards.It’s amazing - though it may break your brain initially.