How to prototype with multiple ML libraries in Cloud? Best to build on top of a rich pre-configured environment such as Kaggle image, extending it with a local virtual environment.
Category archives: coding
Efficient Pre-Commit Hooks with GitHub Actions
How to run pre-commit checks on CI/CD effectively?
Building and Publishing Docker with GitHub Actions
In this post I am sharing my recipe for building and publishing Docker using GitHub Actions. It concisely wraps up a few steps that beginners often find problematic. In particular: The sample code is shown below. See it in action on production here and in this template.
Effective Caching with GitHub Actions
GitHub Actions is great as a CI/CD platform. However, to be efficient, they need to leverage some optimization techniques. In this note, I discuss how to use cache effectively, with respect to multiple paths and sudo-installed APT packages.
Making SSH work by proxy
It is a popular misbelief that hiding encrypted connections (SSH) behind a proxy is a dark domain reserved to crime activities. You may need a Russian or Iranian proxy to get your coding job done, when firewalls of your favourite coffee place or wifi in travel forbid the use of SSH. As this happens to …
Free and robust Tweets extraction
As anticipated by many, Twitter stopped offering its (limited!) API for free 1. Now, what options do you have to programmatically access the public content for free?In this context, it is worth mentioning the library snscrape, a tool (well-maintained as of now) for extracting the content from social media services such as Facebook, Instagram or …
Monitoring Azure Experiments
Azure Cloud is a popular work environment for many data scientists, yet many features remain poorly documented. This note shows how to monitor Azure experiments in a more handy and detailed way than through web or cl interface. The trick is to create a dashborad of experiments and their respective runs, up to a desired …
Repo Passwords in Poetry
Poetry, a popular Python package manager, prefers to use keyring to manage passwords for private code repositories. Storing passwords in plain text is a secondary option, but may be needed in case of either issues in poetry itself or with keyring configuration (may not be properly installed, be locked etc). To disable the use of …
Marking Python Tests as Optional
Often code tests are to be run on special demand, rather than in a CI/CD manner: for instance, they may be slow or work only in a local mode with protected data. This note shows how to declare code tests optional in pytest, the leading testing framework for Python. The article is inspired by the …
Debug CI/CD with SSH
What to do when CircleCI reports are not informative enough on errors? Debug the failing environment live with SSH!