Quick Ruby client for Marqo (VectorDB)

Marqo is a cool vector DB that lets you store facts and then query them later with natural language. You can install Marqo using Docker, which I managed to do today on my M1 Mac. Just make sure you stop whatever ElasticSearch or OpenSearch instances you may already have running on your machine first, since Marqo wants to use its own.

Once you have it running, you can use the following Ruby class to access it in your Rails project.

require 'httparty'

class Marqo
  include HTTParty

  base_uri 'http://localhost:8882'

  def initialize(auth = { username: 'admin', password: 'admin' })
    @auth = auth
  end

  def store(index_name, name, detail)
    options = {
      headers: { 'Content-Type' => 'application/json' },
      body: [{name: name, detail: detail}].to_json
    }
    self.class.post("/indexes/#{index_name}/documents", options)
  end

  def search(index_name, query)
    options = {
      basic_auth: @auth,
      headers: { 'Content-Type' => 'application/json' },
      body: { q: query }.to_json
    }
    self.class.post("/indexes/#{index_name}/search", options)
  end

  def self.client
    @client ||= new
  end
end

The data from my personal TIL site is now migrated. Here's how I handled URL redirects in Cloudflare

To set up a subdomain redirect with Cloudflare while preserving the request path and parameters, you can use Page Rules.

In the Cloudflare dashboard for your domain, click on the Page Rules link in the Rules section of the main navigation menu on the left of the page.

Click on the Create Page Rule button to start configuring the redirect.

In the If the URL matches field, enter the pattern for the URLs you want to redirect, using a wildcard (*) to capture the path and query string. In this cae, the pattern is:

til.obiefernandez.com/*

Now click on Add a Setting and select Forwarding URL from the dropdown menu.

Select 301 - Permanent Redirect from the Status Code dropdown.

In the Enter destination URL field, enter the URL where you want to redirect your requests to, using the $1 placeholder to preserve the query string.

For example: https://til.magmalabs.io/$2

Click on Save and Deploy to create the Page Rule and enable the redirect.

Using OpenAI's TikToken in Ruby

The encoding used in the tiktoken library (and the Ruby binding discussed in this post) is a specific way of converting text into a sequence of tokens, which are then represented by their unique IDs. The encoding scheme is designed to work with OpenAI models like gpt-3.5-turbo and is based on the model's vocabulary and tokenizer.

There's a simple Ruby binding for TikToken made by Iapark that compiles the underlying Rust library. https://rubygems.org/gems/tiktoken_ruby

First add it to your Gemfile

gem "tiktoken_ruby"

Then use it in your code. The service module I wrote today to use it in my Rails app looks like this:

require 'tiktoken_ruby'

module TikToken
  extend self

  DEFAULT_MODEL = "gpt-3.5-turbo"

  def count(string, model: DEFAULT_MODEL)
    get_tokens(string, model: model).length
  end

  def get_tokens(string, model: DEFAULT_MODEL)
    encoding = Tiktoken.encoding_for_model(model)
    tokens = encoding.encode(string)
    tokens.map do |token|
      [token, encoding.decode([token])]
    end.to_h
  end
end

Here's what it looks like in practice.

irb> TikToken.count("Absence is to love what wind is to fire; it extinguishes the small, it inflames the great.")
=> 19

irb> TikToken.get_tokens("Absence is to love what wind is to fire; it extinguishes the small, it inflames the great.")
=>
{28878=>"Abs",
 768=>"ence",
 374=>" is",
 311=>" to",
 3021=>" love",
 1148=>" what",
 10160=>" wind",
 4027=>" fire",
 26=>";",
 433=>" it",
 56807=>" extingu",
 21168=>"ishes",
 279=>" the",
 2678=>" small",
 11=>",",
 4704=>" infl",
 986=>"ames",
 2294=>" great",
 13=>"."}

The encoding is essential for processing text with the OpenAI models, as it allows them to understand and generate text in a format that is compatible with their internal representations. In the context of the tiktoken library, the encoding is particularly helpful for estimating token counts in a text string without making an API call to OpenAI services.

Use isSameorAfter to set condition to select specific periods

momentjs is a JavaScript library that helps parse, validate, manipulate, and display date/time in JavaScript in a very easy way.

momentjs provide specific instructions to handle dates which can create conditions in each one, to select a period after and same that you needed, like a >= instruction use isSameorAfter:

moment('2010-10-20').isSameOrAfter('2010-10-19'); // true
moment('2010-10-20').isSameOrAfter('2010-10-20'); // true
moment('2010-10-20').isSameOrAfter('2010-10-21'); // false

Upgrade to Ruby 3.2 to avoid ReDoS attacks

Did you know that in Ruby 3.1.3 and prior some regular expressions could take a long time to process?

Don't believe me? Try running this in a 3.1.3 irb console:

/^a*b?a*$/ =~ "a" * 50000 + "x"

Your system will halt for like 10 seconds before returning no matches. This is the basis for ReDoS (Regexp Denial of Service) attacks.

Thankfully, Ruby 3.2.0 has fixed this and the same regexp gets resolved in 0.003 seconds. They also added a Regex.timeout global option which would prevent your app from falling victim to ReDoS attacks!

Handle environment variables on a heroku app from CLI

The heroku config commands of the Heroku CLI makes it easy to manage your app’s config vars.

cd myapp

# This will list all the env variables from your app
heroku config 

# Set env you want to add/update
heroku config:set S3_KEY=12345

# Get specific env
heroku config:get S3_KEY

# Removes specific env
heroku config:unset S3_KEY

CSS Container Queries

I've always seen responsive elements styled with CSS media queries. These often need more media queries to make them look good on different page sections. There's a way to skip the headache.

Use of container queries to apply CSS styles with the element's container size in mind instead of the viewport size:

/* Create a container context */
.container {
  container-type: inline-size;
}

/* Default font size */
.card h2 {
  font-size: 1em;
}

/* Updated font size when the container is larger than 500px */
@container (min-width: 500px) {
  .card h2 {
    font-size: 2em;
    color: gray;
  }
}

<div class="container">
  <div class="card">
    <h2>Card title</h2>
  </div>
</div>

Example

Enumerable#any? and Enumerable#detect

Someone told me #any? didn't stop iterating after finding the first truthy value, that I should use #detect instead, but turns out both methods behave the same, at least in Ruby 3.0.1.

Try running this for example:

(1..5).to_a.any? do |n|
  puts n
  true
end

(1..5).to_a.detect do |n|
  puts n
  true
end

You'll see this:

1
1

to install ruby 2.4.6 on a Mac m1 with rbenv

Once upon a time (even thouhg we have newer versions of ruby) I had to install an older version of ruby, the 2.4.6 on my Mac M1 (OS X 12.6), I was having this problem:

ruby-build: using readline from homebrew

BUILD FAILED (macOS 12.6 using ruby-build 20220910.1)

So after googling I found that we can do this:

CFLAGS="-Wno-error=implicit-function-declaration" RUBY_CONFIGURE_OPTS='--with-readline-dir=/usr/local/opt/readline/' arch -x86_64 rbenv install 2.4.6

and it worked!

that's it bye!

Tell apt to keep the current version of a package

If you ever need apt to keep the current version of a package, use hold:

sudo apt-mark hold <package>

This will prevent apt from upgrading the package. You can always unhold it back when you want to upgrade it.

Always use retry inside rescue blocks

Ruby includes a retry keyword that would bring execution back to the start of the current method while preserving scope. But what if you do this?

def my_method
  my_var = 1
  retry
end

That would fail with an Invalid retry, which is one of the few illegal statements in Ruby.

retry is only allowed inside rescue blocks.

def my_method
  my_var = 1
  raise Exception
rescue Exception
  retry
end

How to use object destructuring with the index over an array?

const names = ['Roy', 'Liz', 'Sofia', 'Clau']

const {
  0: zero,
  1: one,
  2: two,
  3: three,
} = names

console.log(zero) // 'Roy'

How to rebase all reachable commits including the first commit?

git rebase -i --root

Fr Unit with CSS Grid Layout

I read about CSS Grid Layout, and I found the Fr unit, so I have to try it out.

From w3.org:

Flexible Lengths: the fr unit A flexible length or is a dimension with the fr unit, which represents a fraction of the leftover space in the grid container. Tracks sized with fr units are called flexible tracks as they flex in response to leftover space similar to how flex items with a zero base size fill space in a flex container.

The distribution of leftover space occurs after all non-flexible track sizing functions have reached their maximum. The total size of such rows or columns is subtracted from the available space, yielding the leftover space, which is then divided among the flex-sized rows and columns in proportion to their flex factor.

Each column or row’s share of the leftover space can be computed as the column or row’s * / .

Behavior of Capybara's "have_no_content" when expected with `.to_not` and with `.to`

about to_not have_content:

  expect(page).to_not have_content("not on page")

it waits (Capybara's default waiting time), then it pass. Don't use it! 🚫

whereas to have_no_content:

  expect(page).to have_no_content("not on page")

Waits (same as with .to_not) but it passes as soon as "not on page" disappears from the page; use this one to wait for loading messages to disappear! ✅