Adventures in Data Engineering : GitHub Actions and CI/CD (part 2)

Intro

This is a continuation of an article I wrote last week by the same name except for part 1. If you haven’t read it, you can read it here https://ryhowe.medium.com/adventures-in-data-engineering-github-actions-ci-cd-part-1-452275f2fd4a however it isn’t technically necessary. This will focus on adding in jobs for linters (Ruff and pylint) and an example for pytest.

Linters are tools that analyze source code to identify and flag programming errors, bugs, stylistic inconsistencies, and other potential issues. They play a crucial role in software development by helping developers maintain code quality, adhere to coding standards, and catch common mistakes early in the development process. Linters are language-specific and can be integrated into the development workflow through text editors, integrated development environments (IDEs), or build systems. By providing real-time feedback and automating code reviews, linters contribute to the overall efficiency and reliability of software projects, ultimately leading to cleaner, more maintainable code.

In Python, developers commonly use linters like pylint and flake8 to analyze code for errors, enforce coding standards, and ensure adherence to PEP 8 guidelines. Additionally, tools like black offer automatic code formatting for consistent styling. For static type checking, mypy helps identify and prevent type-related issues. These linters contribute to code quality, readability, and collaboration in…