UUID as Primary Key with Phoenix LiveView Authentication

Phoenix Framework has a potent generator for generating authentication for a project. The mix phx.gen.auth command can generate a complete user authentication system with registration, login, email confirmation, and password reset. But by default, it uses bigserial as the primary key. Using a sequence has drawbacks since it is a predictable counter and anyone can easily know or traverse the service for all available users. For this reason, we sometimes like to use UUID for the primary key.

Today, I’ll go through how to change the default Phoenix behavior of having bigserial as the primary key of the user table. If you don’t have Phoenix installed, check out the official documentation for how to install it.

Let’s generate a new Phoenix application to test things out:

mix phx.new demoUUIDAuth

The command will generate a new Phoenix application with Postgresql as a database. Now let’s change the DB configs in config/dev.exs so that Phoenix can connect with our local running database. After that we can generate the LiveView Authentication with the following command:

mix phx.gen.auth Accounts User users --hashing-lib argon2

The --hashing-lib argon2 tells Phoenix which hashing function to use for the password hashing. Available options are, bcrypt, argon2, and pbkdf2. By default, it uses bcrypt. For more information check out the official documentation. After running the command, you’ll be prompted to run the migration and to get the dependencies. Let’s get the dependencies but hold on running the migration because we need to change a few things to have UUID as our user table primary key. Run mix deps.get to get the dependencies, you’ll see it’ll get the argon2 and comeonin packages.

Since we have the packages, now we can change the schema to tell Ecto that we want to use a different type of primary key for our users table. Ecto here acts as the data mapper and database adapter. We can start using UUID as the primary key for the users table by changing the lib/demoUUIDAuth/accounts/user.ex file like the following:

+    @primary_key {:id, :binary_id, autogenerate: true}
+    @derive {Phoenix.Param, key: :id}
     schema "users" do
        field :email, :string
        field :password, :string, virtual: true, redact: true
        field :hashed_password, :string, redact: true
        field :confirmed_at, :naive_datetime

        timestamps(type: :utc_datetime)
  end

By adding the @primary_key attribute, we’re telling our application that the primary key for the schema will be named id and it is of type binary and it’s autogenerated. Ecto v2 guarantees autogeneration of the primary key either by invoking the database function or generation in the adapter. Meaning, either the ID will be generated on the database or it will be generated automatically by the database adapter layer and sent to the database while inserting.

Now, we have to change the users table definition in the migration file. Open your migration file at priv/repo/migrations/20240224203145_create_users_auth_tables.exs and change the following:

-   create table(:users) do
+   create table(:users, primary_key: false) do
+     add :id, :uuid, primary_key: true
      add :email, :citext, null: false
      add :hashed_password, :string, null: false
      add :confirmed_at, :naive_datetime
      timestamps(type: :utc_datetime)
    end

This change should have been enough if the user_tokens table didn’t have a relationship with the users table. But here the user_tokens table has a reference of users.id as user_id. We also have to change the type of user_tokens.user_id . Since we’re already on the migration file, let’s change the type of user_id field by doing the following:

-   add :user_id, references(:users, on_delete: :delete_all), null: false
+   add :user_id, references(:users, type: :uuid, on_delete: :delete_all), null: false

Basically, we’re saying the type on the reference. But this is not enough, since our schema of user_tokens still doesn’t know the type of user_id field. We have to do the following changes in the lib/taskchecklist/accounts/user_token.ex file:

   schema "users_tokens" do
     field :token, :binary
     field :context, :string
     field :sent_to, :string
 -   belongs_to :user, DemoUUIDAuth.Accounts.User
 +   belongs_to :user, DemoUUIDAuth.Accounts.User, type: :binary_id

     timestamps(updated_at: false)
   end

Now, we’re done with our changes and ready to run the mix ecto.migrate / mix ecto.setup command to run our database migrations. After the successful run of the migration we can start the Phoenix app with the following command:

mix phx.server

This will start our Phoenix application in localhost:4000 by default. To register a new user, go to http://localhost:4000/users/register and register your user. If you look into the logs, you’ll see the id for the user is UUID. You can also verify this by IEx. Run the following commands:

$ iex -S mix
iex(1)> DemoUUIDAuth.Repo.all(DemoUUIDAuth.Accounts.User)
[debug] QUERY OK source="users" db=5.3ms decode=1.1ms queue=2.6ms idle=1240.8ms
SELECT u0."id", u0."email", u0."hashed_password", u0."confirmed_at", u0."inserted_at", u0."updated_at" FROM "users" AS u0 []
↳ :elixir.eval_external_handler/3, at: src/elixir.erl:396
[
  #DemoUUIDAuth.Accounts.User<
    __meta__: #Ecto.Schema.Metadata<:loaded, "users">,
    id: "7d715492-3360-4a74-820e-2d9f17d772bd",
    email: "[email protected]",
    confirmed_at: nil,
    inserted_at: ~U[2024-02-24 21:26:12Z],
    updated_at: ~U[2024-02-24 21:26:12Z],
    ...
  >
]
iex(2)>

Congratulations! You’ve successfully changed the primary key for the Users table from bigserial to UUID. This might feel like a long process but it’s not that bad considering I’ve included all steps of generating the authentication part. The actual change of bigserial to UUID only takes changes in 3 files. You can checkout the repository of this article on GitHub.

If you have any questions, feel free to comment on this post, or reach out to me via LinkedIn or Email ([email protected]). Thanks for reading!

In-memory Decompression of gzipped Web-response body Data in Elixir

I was trying to write a web crawler with Elixir and Crawly (A web crawling framework written in Elixir). The website I was trying to crawl had their sitemap gzipped. So, each entry in their main sitemap returned me a file like “sitemap-01.xml.gz” which needed to be gunzipped to get the XML file I was after.

Initially, I thought HTTPoision will handle the decompression. But turns out HTTPoision is just a wrapper around hackney and hackney doesn’t support this. So, I did a few google searches, and being tired, I didn’t use the most effective keywords and ended up with File.stream! which obviously didn’t work. Because, File.stream! needs a file path which should have been a red flag, but I proceeded to go down the rabbit hole anyway.

Then I thought that it might work if I write the response to a file and decompress it with File.stream! but thinking about it gave me the chills, there’ll be a lot of files written, decompressed, and read from. So this wasn’t the solution I was even going to write and try out.

After a whole lot more searches and asking around, I finally found the answer (Huge thanks to Mafinar bhai), which is an Erlang lib called “zlib“. Using this library I could easily get the data I wanted to like the following code block:

response.body
|> :zlib.gunzip()

Now you might be asking why I didn’t use an HTTP client which had compression support like, Tesla? Because, I had HTTPoison free with Crawly, and I didn’t want to explore how to change it to Tesla or Mint due to a deadline. Yes, deadlines are the worst!

Running scheduled tasks in Elixir with Quantum

Quantum is a beautiful and configurable library that allows running scheduled tasks on an Elixir system/application. Even though the documentation may seem adequate to an experienced Elixir/Erlang programmer, being a newcomer it was a bit confusing to me. So, I’m trying my best to explain this so that when I forget, I have a reference…

If you’re future me or a lost soul, continue reading.

What is the problem we’re trying to solve exaclty?

On a software system, there are a few tasks that need scheduled running. Examples of such tasks would be, a database backup that runs everyday at 12 AM. Or, fetching data from some API that updates at 10 PM every day and you just call that API and populate your database. Or, you might need to check the connection with other services each minute. Or. think about renewing your website’s SSL certificate every 3 months. All of these tasks are traditionally handled by the cron jobs of Linux. But we can do better.

Let’s say, you’re deploying your application to 10 different servers. And you run identical scheduled jobs in each server because they are a fleet of identical systems. Yes, you can configure your cron jobs in each of those servers or you can ship your application with its own scheduler and job queue. So that you don’t have to do any extra configuration on an individual server. One less configuration means, one less scope to screw things up. (Don’t ask how many times I’ve messed up crontab in a production server)

Ok, but is shipping a scheduler with my app a good idea?

Well, it depends.

Even though I’m not qualified to talk about how BEAM handles concurrency and schedules the processes, I can link a blog post. Basically, the scheduler process is very light weight and it doesn’t block anything. It’s all handled by the magic we know as BEAM. So, you would barely notice any performace hit shipping a task scheduler with your Elixir application.

Tell me more about that Quantum thing.

According to Quantum‘s documentation Quantum is a “Cron-like job scheduler for Elixir.” Basically it let’s us run our Elixir/Erlang fucntions in a scheduled manner. I’m not going to cover everything about how to utilize this library in your application, you can read their documenation for that. I’ll only cover a few things that I wish I could figure out faster than I was able to when I first used this library.

1. Job format in config.exs file

So, in the Usage section of the documentation they tell you a job format like {"* * * * *", {Heartbeat, :send, []}} but it was really confusing for me to understand what those parameters were. I later figured out that inside the tuple, Heartbeat was the module name, :send is a fucntion inside that Heartbeat module and [] was the argument to that :send fucntion. So the job was bascially calling the Heartbeat.send/0 fucntion. In retrospect, it feels really dumb thinking how much I time I spent figuring this out.

2. You need a TimeZone database

So, like most of us your server timezone might be set to UTC. But you live in Bangladesh and you need to send a daily reminder to your collegues at 10 AM Asia/Dhaka time. Instead of doing the math that Asia/Dhaka time is actually UTC+6, you want to directly type 10 in your job configuration. You look into Quantum’s amazing documentation and find that it has time zone support. Before you jupm and copy the code block for TZ support, becareful and notice that you actually need another module Tzdata and config :elixir, :time_zone_database, Tzdata.TimeZoneDatabase this line in your config.exs file.

Good Bye!

These were the two problems I struggled a bit and spent more time than I should have. So, I’m just documenting them. If you have any other questions, problems or opinions that I’ve not listed here, please put them in the comments or email to me at [email protected] I’d love to hear your experience. I’m alvailable with @m4hi2 handle, almost everywhere. Thanks for reading. 😀

Solution: Hosts file parsing error with erlang/elixir on Windows

If you’re encountering the following error:


2020-12-08 01:01:32.768000
args: ["c:/WINDOWS/System32/drivers/etc/hosts",1]
format: "inet_parse:~p:~p: erroneous line, SKIPPED~n"
label: {error_logger,info_msg}
inet_parse:"c:/WINDOWS/System32/drivers/etc/hosts":1: erroneous line, SKIPPED

when running iex --name alice on Windows, you’re not alone. Turns out, the file encoding has to be ASCII for the parsing to work. But some other application like Docker might access the hosts file and change it’s encoding to UTF-8. This usually doesn’t affect day-to-day usage but for some reason, Earlang’s inet_parse doesn’t like this. To solve the above error you have to change the encoding of the host file to ASCII. This can be easily done with a PowerShell Command.

Open a Powershell window with Admin privileges. And execute the following:

Get-Content -Path "C:\Windows\System32\drivers\etc\hosts" | Out-File -FilePath "C:\Windows\System32\drivers\etc\hosts" -Encoding ascii

This should solve the above issue. If this doesn’t help you, please make sure you’re hosts file isn’t corrupted in any other ways.