[LEAPSECS] Time math libraries, UTC to TAI

[Steve Summit]

Warner Losh wrote:
> On Thu, Dec 29, 2016 at 12:28 AM, Steve Summit <scs+ls at eskimo.com> wrote:
>> If we try to handle them better, two conclusions seem inescapable:
>>
>> 1. Posix-compatible time_t values will need to be smeared,
>>    spreading the leap second out more gradually, to eliminate
>>    sudden, unseemly jumps.
>>
>> 2. There needs to be a well-defined way to deliver true,
>>    non-smeared, leapsecond-aware UTC time to code that needs it.
>>
>> As a proof of concept, I'm working on a set of Linux kernel mods
>> to do both #1 and #2.
>
> How do we know when to do #1 and when to do #2? Especially in light of
> the uses of #1 that do extra things to get to knowledge of truth?

Bear in mind that I am not putting this forth as a universal
solution, that I could possibly expect everyone to want to adopt.

The answer is that on a system with the mods installed, #1
happens all the time for the system calls time(), gettimeofday(),
and clock_gettime(CLOCK_REALTIME).  It does not happen for
adjtimex, on the assumption that that's what ntpd is using.
#2 happens only when you call clock_gettime(CLOCK_UTC).

I've thought about having the smearing algorithm in #1 selectable
on a per-process basis, but I suspect the cost and confusion
would outweigh any benefits.

My assumption is that since leap-second handling is undefined in
Posix, most code doesn't care about precise leap-second handling
and won't mind (and, if it thought about it, would prefer) a more
gradual smear.

Code that does try to "back out" precise leapsecond semantics
from the undefined mess that is Posix and the legacy implementations
would either have to (a) not adopt these mods or (b) be rewritten
to use the clock_gettime(CLOCK_UTC).

> Also, how do you make this work 'by default' without making every
> single programmer that uses these APIs know all the leap second rules?

I don't know what you mean.  I certainly don't want anyone but
the kernel and the application-level time libraries to "know all
the leap second rules", although the kernel's knowledge of the
current DTAI table is made available to anyone who wants it, via
a new special file /proc/timekeeping/leapseconds.

> How do you deal with acquiring knowledge of leap seconds after you've
> given out 'old' timestamps that might be affected by them? This is
> best described as how well you recover from the leapsecond file being
> corrupted and replaced, though there's many other scenarios here.

[Disclaimer: I suspect you've thought about these problems longer
than I have, and may have nuances in mind that I'm oblivious to.
Apologies if this ends up being a naive answer.]

One answer is simply that not all mistakes can be recovered from,
such that inability to do so is not necessarily a fatal defect.
If your clock was wrong yesterday, and you don't know by how
much, there's no general way to fix all the wrong timestamps
you collected then, either.

But the deeper answer is that I think we make the whole
leapsecond problem even harder than it has to be by thinking
about absolute timestamps as if they carry history of all the
leap seconds that have come before them.  *If* you try to "fix"
the Posix time_t leapsecond problem by defining a new monotonic
count of seconds since the epoch that does include leap seconds,
then yes, if you learn about a leapsecond later, you may have
some timestamps to fix.  But of course that's not the only way
to encode UTC.

In particular, my implementation of clock_gettime(CLOCK_UTC)
uses the often-proposed trick of encoding the leap second as
a nonnormalized timespec.  But that means that at 8am UTC this
Sunday we'll call it (1483257600,0) regardless of whether we knew
there was a leap second at the beginning of that day or not.

Another question is what you can or can't do with a historical
smeared timestamp, and the answer is, nothing special.  If
yesterday was a leapsecond day but you didn't know it then,
the Posix-compatible time_t timestamps you collected yesterday
weren't smeared and there's nothing I'm going to retroactively
do about it.  If last Sunday *was* a leapsecond day and you
did know it, but you want to unsmear the Posix-compatible time_t
timestamps you collected then to recover true UTC, you can't
necessarily do that either, because you might not know what smear
interval was used (it's configurable, but it's obviously not
stored along with each timestamp).

Finally, if you're computing leapsecond-aware intervals with
respect to historical timestamps, you're generally going to need
accurate tables of historical leap seconds in order to do so, but
in general (and, again, assuming the leap seconds aren't baked
into the timestamps) the differences you compute are going to
reflect the accuracy of the leapsecond tables in force when you
do the subtractions, so if you update the tables and redo the
subtractions, you'll get better answers.