Staying in a relatively safe realm of TypeScript code, with strict null checks, is all fine and dandy. Sometimes though one has to deal with the cruel outside world: JSON from the server or from the localStorage, user input, URL route parameters, and so on. Then we have to shoehorn the uncontrolled chaos from the outside into our nice algebraic type system which strives for making the impossible states unrepresentable. Here's a simple example: an enumeration of Languages.
enum Language { DE, FR, IT, EN }
That's a handy thing to define, because, if you write a function returning some localized text, and happen to forget about English…
function thanks(language: Language): string {
switch (language) {
case Language.DE:
return "Danke";
case Language.FR:
return "Merci";
case Language.IT:
return "Grazie";
}
}
The TypeScript transpiler will complain:
a.ts:8:40 - error TS2366: Function lacks ending return statement
and return type does not include 'undefined'.
8 function thanks(language: Language): string {
~~~~~~That makes sure our pattern matching is exhaustive. Fewer bugs!
We'd certainly want to map the stringly-typed 'de', 'fr', 'it', and whatever else (null, undefined, NaN, [object Object]) obtained from the outside, into a member of the Language union, right at our module's boundary, before doing anything with the value. We'd need an utility like this:
function toLanguage(x: any): Language {
switch (String(x)) {
case "fr":
return Language.FR;
case "it":
return Language.IT;
case "en":
return Language.EN;
default:
return Language.DE;
}
}
Thus, we fall back to German on any kind of nonsense received.
Quite a bit of repetition here, no? It'll only get worse as the enum grows. We can do better, of course. Here's a little trick to avoid the repetition, stay type-safe, and never revise the toLanguage function later on, should the enum change.
function toLanguage(x: any): Language {
const key = String(x).toUpperCase();
const mayBeLanguage: Language | undefined = (Language as any)[key];
return mayBeLanguage !== undefined ? mayBeLanguage : Language.DE;
}
Smart cast to the rescue! Code on.
Published: 2018-11-27
Tagged: decode union typescript adt javascript typesafe enum
This week at work I had to deal with fixed-width column data like this. That is: a plain text file where each line holds a single record with a predefined order of columns/fields. Each column has a predefined length (in symbols), and, if the value is shorter, it's padded with spaces. Every line contains every column, and, therefore, every line has exactly the same length. For example,
Field A | Field B | Field C |…
01234567890123456789AB0123456789ABCD…
X YYY ZZZ …
…having the lengths of 10, 12, and 14 respectively, would be
[{
A: "0123456789",
B: "0123456789AB",
C: "0123456789ABCD"
}, {
A: "X",
B: "YY",
C: "ZZZ"
}, …]
Parsing the data, I came up with a very natural approach which relies on Kotlin/Java enums. Nothing groundbreaking or novel here. I'm sure this trick is familiar to many. However, I liked how naturally the tool suits the problem, and decided to share.
fun parseLine(rawData: String): BankRecord {
val parseStringBound = { f: RawBankDataField -> parseString(rawData, f) }
return BankRecord(
clearingNumber = parseInt(rawData, RawBankDataField.CLEARING_NUMBER),
name = parseStringBound(RawBankDataField.NAME),
postalAddress = parseStringBound(RawBankDataField.POSTAL_ADDRESS),
postalCode = parseStringBound(RawBankDataField.POSTAL_CODE),
city = parseStringBound(RawBankDataField.CITY)
)
}
data class BankRecord(
val clearingNumber: Int,
val name: String,
val postalAddress: String,
val postalCode: String,
val city: String
)
private fun parseInt(rawData: String, f: RawBankDataField): Int {
return parseString(rawData, f).toInt()
}
private fun parseString(rawData: String, f: RawBankDataField): String {
return rawData.substring(indicesRange(offset(f), f.length)).trim()
}
private fun indicesRange(start: Int, length: Int) = start.until(start + length)
private fun offset(f: RawBankDataField): Int {
var result = 0
for (i in RawBankDataField.values()) {
if (i == f) return result
result += i.length
}
return result
}
private enum class RawBankDataField(val length: Int) {
GROUP(2), // Gruppe
CLEARING_NUMBER(5), // BCNr
SUBSIDIARY_ID(4), // Filial-ID
NEW_CLEARING_NUMBER(5), // BCNr neu
SIC_NUMBER(6), // SIC-Nr
MAIN_OFFICE_CLEARING_NUMBER(5), // Hauptsitz
CLEARING_NUMBER_TYPE(1), // BC-Art
VALID_SINCE(8), // gültig ab
SIC(1), // SIC
EURO_SIC(1), // euroSIC
LANGUAGE(1), // Sprache
SHORT_NAME(15), // Kurzbez.
NAME(60), // Bank/Institut
DOMICILE_ADDRESS(35), // Domizil
POSTAL_ADDRESS(35), // Postadresse
POSTAL_CODE(10), // PLZ
CITY(35) // Ort
}
Note that, like in this example, we may need just a subset of the fields present in the data source. Nevertheless, obviously, we'd have to enum-erate all the columns anyway, up to the rightmost relevant to us. I particularly like that it's so easy to adapt to the changes in the data source format, and to support additional columns.
I only have to say that, if you have a lot of relevant fields packed into the data line, and a lot of lines, you may wish — for a better performance — to
memoize the offset() function.
Published: 2018-09-03
Recently I've configured Let's Encrypt certificates for a staging HTTPS server for one of our clients. That was amazingly easy to do. I remember checking the ACME protocol client tools in the very beginning of Let's Encrypt initiative. Back then the tools were, of course, very immature: hard to install and configure, and they only worked in interactive mode. It's all in the past now.
Basically, I took this great blog post and codified it in SaltStack states. SaltStack is an automatic configuration management tool similar to Puppet and Ansible. It's quite simple and declarative, yet it scales up to larger infrastructure. Our sysadmins love it.
Below is the annotated YAML of my solution.
backports:
pkgrepo.managed:
- name: deb http://ftp.de.debian.org/debian jessie-backports main contrib non-free
- file: /etc/apt/sources.list.d/backports.list
dehydrated:
pkg.installed:
- fromrepo: jessie-backports
- require:
- pkgrepo: backports
That adds the jessie-backports Debian package repository, and installs the Dehydrated ACME client from it. Starting with Debian Stretch (v9) it's not necessary to add the backports repo. Then, the section above can be reduced to just this:
dehydrated:
pkg.installed
The salt://web-server/conf/example.com nginx configuration file:
server {
listen 80;
server_name example.com;
include /etc/nginx/conf.d/*.http.location.inc;
location / {
return 301 https://example.com$request_uri;
}
}
server {
listen 443 default_server ssl;
server_name example.com;
ssl on;
include /etc/nginx/conf.d/ssl.inc;
include /etc/nginx/conf.d/*.location.inc;
# ...
}
Which is then referenced from the salt://web-server/init.sls:
/etc/nginx/sites-enabled/example.com:
file.managed:
- source: salt://web-server/conf/example.com
- require:
- pkg: nginx
- watch_in:
- service: nginx
/etc/nginx/conf.d/acme-challenge.http.location.inc:
file.managed:
- source: salt://web-server/conf/acme-challenge.http.location.inc
- require:
- pkg: nginx
- pkg: dehydrated
- watch_in:
- service: nginx
The salt://web-server/conf/acme-challenge.http.location.inc file is nothing but:
location ^~ /.well-known/acme-challenge {
auth_basic "off";
alias /var/lib/dehydrated/acme-challenges;
}
Now the tricky bit. The /etc/nginx/conf.d/ssl.inc referenced in the salt://web-server/conf/example.com will in the end contain the path to the certificate file, and the path to the secret server key file. We don't have those initially, before the ACME challenge takes place. Thus, we need some “transitional” certificate and key to make nginx start and serve the ACME challenge Web location over HTTP.
ssl_inc_transitional:
cmd.run:
- name: >
echo 'ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem;' > /etc/nginx/conf.d/ssl.inc &&
echo 'ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;' >> /etc/nginx/conf.d/ssl.inc
- unless: test -f /etc/nginx/conf.d/ssl.inc
- require:
- pkg: nginx
- require_in:
- service: nginx
/etc/dehydrated/domains.txt:
file.managed:
- contents:
- example.com
- require:
- pkg: dehydrated
initial_lets_encrypt_cert:
cmd.run:
- name: /usr/bin/dehydrated --cron
- unless: test -d /var/lib/dehydrated/certs/example.com
- require:
- file: /etc/nginx/conf.d/acme-challenge.http.location.inc
- file: /etc/dehydrated/domains.txt
ssl_inc:
cmd.run:
- name: >
/bin/systemctl stop nginx &&
echo 'ssl_certificate /var/lib/dehydrated/certs/example.com/fullchain.pem;' > /etc/nginx/conf.d/ssl.inc &&
echo 'ssl_certificate_key /var/lib/dehydrated/certs/example.com/privkey.pem;' >> /etc/nginx/conf.d/ssl.inc &&
/bin/systemctl start nginx
- onlyif: test -f /etc/nginx/conf.d/ssl.inc && cat /etc/nginx/conf.d/ssl.inc | grep -F ssl-cert-snakeoil
- require:
- cmd: initial_lets_encrypt_cert
The final part is the certificate renewal cron job:
root_email_for_cron:
cron.env_present:
- user: root
- name: MAILTO
- value: webmaster@example.com
lets_encrypt_cert_update:
cron.present:
- name: chronic /usr/bin/dehydrated --cron && systemctl reload nginx
- identifier: lets_encrypt_cert_update
- user: root
- dayweek: 0
- hour: 4
- minute: 2
The chronic tool is a part of the moreutils Debian package. It runs a command quietly unless it fails. Setting the MAILTO environment variable for cron makes sure all the failures are e-mailed to webmaster@example.com. The cron job configured above will run every Sunday at 04:02.
Published: 2018-07-18