Posts

Introduction In this article, we’ll build a simple application using the Cats ecosystem. The application will expose a REST API for user management, including an OpenAPI documentation endpoint. Some endpoints will require JWT token authentication, also we’ll use the Bcrypt library to generate password hashes. We’ll demonstrate both unit testing and integration testing using testcontainers to run a Postgres database in Docker. The GitHub repository for this project is available here. ...

In this article, I will explain parallelism and concurrency in Java, covering its evolution from basic threads to modern concurrency patterns. We’ll explore different approaches to concurrent programming in Java, starting with traditional thread-based concurrency and progressing through more advanced patterns. Java’s concurrency model has evolved significantly over the years. The latest addition is Structured Concurrency, introduced as a preview feature in Java 19 and currently in its fifth preview in Java 25 (LTS). This powerful new model, specified in JEP 505, addresses many of the challenges of traditional thread management. We’ll examine Structured Concurrency in detail in the final section of this article. ...

In this short article, I will explain lateral joins in Spark. I will demonstrate lateral joins using a simple example in Scala, and also show alternative approaches like inner join calculation with window functions and pure Spark SQL. The def lateralJoin(right: Dataset[_]): DataFrame function on Dataset was introduced in version 4.0.0, but according to the documentation, the lateral subquery feature was first introduced in version 3.2.0. Source code is located here: https://github.com/JurajBurian/spark-playground. ...

Hugo is a great framework for building websites from markdown documents, but when it comes to diagram integration, you’ll soon find that Hugo doesn’t support them out of the box. Well, actually, I must correct myself - Hugo does support Goat diagrams natively! In this article, I’ll explore various ways to render diagrams in Hugo, including PlantUML, Mermaid and Diagon. The repository of my blog is available here. Goat diagrams Supported out of the box by Hugo. Here is an example: ...

In this short article, I would like to cover the entire development cycle of Spark jobs. I will explain following topics: setup of sbt multi-module project how to write modular code how to write testable code how to write unit tests how to write integration tests Source code is located here: https://github.com/JurajBurian/spark-example . The sbt project. In this sample project, we have defined a single Spark job as a module alongside a common module to demonstrate modular development. This approach is useful when developing several Spark jobs, that share models or functionality. ...

I like physics. The reason behind it is that physics has a beautiful and consistent type system. Let’s start with very simple scalar model, that is used in Newton mechanics. What we need here are several types like Time, Acceleration, Velocity, Displacement for kinematics. If we also deal with dynamic, we need Mass, Force, Momentum and Energy. We are also introducing Constant, that is just a number and can be used solely as multiplier (or divider, but not implemented). ...

In this article we combine opaque type alias hierarchies with implicit resolutions to do calculations. Scala 3 brings several language enhancements that give programmers even better control over types . One of them is opaque keyword. For a deeper look what opaque is please visit opaques page at dotty.epfl.ch. Let’s continue with an example and define some types: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 package types { // base type aliases opaque type StringBased = String opaque type IntBased = Int // instantiable type aliases opaque type TString1 <: StringBased = String opaque type TString2 <: StringBased = String opaque type TInt <: IntBased = Int // constructor functions def TString1(p: String): TString1 = p def TString2(p: String): TString2 = p def TInt(p: Int): TInt = p } Let’s explain these things a bit. We have declared several new type aliases in types scope (in our case scope is package types). Inside of the types scope all of these aliases works as ordinary type aliases, so we can easily create “constructor” functions. Also we declared “base aliases” named StringBased and IntBased. So expression: TString1 <: StringBased we understand that TString1 is subtype of StringBased. Let’s imagine we need do a calculation with our new types, let us call it Transformer . ...