[LEAPSECS] Leap second to be introduced at midnight UTC December 31 this year

[Steve Summit]

Hal Murray:
>> Has anybody done any serious thinking about fixing the POSIX problem?

I would love to, love to, love to fix it.  But the solution won't
be to redefine Posix time_t, or abandon it.

Stephen Colebourne:
> The basic observation that I have had over many years is that beyond a
> few experts, very few care about leap seconds.

Right.  Which is why Posix time_t, despite its (to us, anyway)
gaping flaw, is still so ubiquitous.

I think about Posix time_t in three ways:

1. It's utterly broken, in that it claims to be UTC, but fails to
   handle the one facet that distinguishes UTC from other timescales.

2. It's wildly useful, because it's perfectly continuous, and
   allows you to seamlessly integrate clock and calendar handling
   at the second, day, and year scales.

3. Its definition can't be changed, because its published
   definition absolutely allows (and even encourages) any random
   scrap of user code out there to use its own handcrafted logic
   to convert back and forth between time_t and calendar time,
   meaning that we can't just "fix" the Posix leap-second problem
   by redefining time_t and then rewriting library functions like
   localtime() to use our new, improved definition.  Posix time_t
   is here to stay.

Stephen Colebourne:
> My hope when I defined the Java time-scale was that eventually the
> OS would provide it, or something very similar.

Indeed.  I've been exploring a reasonably straightforward
scheme to have the OS kernel keep true UTC internally, but
report smeared Posix time_t by default, but with new way(s)
to report and use true UTC (that is, including an unambiguous
representation of 23:59:60Z) for those few applications that
care.  This is essentially the scheme that Markus Kuhn described
at https://www.cl.cam.ac.uk/~mgk25/posix-clocks.html almost 20
years ago.

Hal Murray again:
> I assume the first step would be to have the kernel keep time in a non 
> leaping time scale.  Is UT1 the obvious choice?

I don't think so.  Civil time is everywhere, even (typically) in
the OS kernel.  UTC and Posix time_t are everywhere, too.  So my
approach is to bite the bullet and implement a "leaping" time
scale in the kernel, i.e. true UTC.  One way to do so is to count
*days* since 1970-01-01 and then seconds -- either 86399, 86400,
or 86401 of them -- within each day.  This is both simple to
implement and simple to convert back and forth between other
representations.

> Would we need a version of ntp that used that time scale?

NTP is well-behaved across leap seconds and can be thought of
as keeping true UTC already, in that its explicit 'insert' and
'delete' bits let you unambiguously interpret its otherwise
"continuous" (Posix-like) time scale.  So while improvements to
NTP are possible, such as having it be a distribution channel
for TAI-UTC data (current and possibly also historical), I don't
think it *has* to be changed.  (Indeed one of my goals is having
a leapsecond-aware OS that works seamlessly with stock NTP feeds.)

> I assume the file dates could be fixed with a flag day and one
> pass over the whole file system.
>
> I think the hard part would be time-stamps in network protocols
> or data bases.  Consider tar files or rsync.

Trying to rewrite every time_t-using protocol and file format,
and adjusting the timestamp on every file in every filesystem
everywhere, isn't just hard, it's impossible!

And in any case, most of them don't need fixing.  For example,
I believe file timestamps can continue to use Posix time_t.
There's a consequence that on leap-second days they'd be up to
one second off with respect to true UTC -- but since they have
only one-second granularity, they're always up to one second off
anyway.

As for network protocols, most of those don't need to (and won't)
change, either, can continue to use the Posix timestamps they use
today, either explicitly finessing leapsecond discontinuities
(if necessary) as suggested by RFC 7164, or simply not noticing
them (and having their performance improved, and their glitches
removed, by running on a true-UTC kernel that delivers smeared,
non-discontinuous Posix timestamps across leap seconds).
Those -- likely few -- protocols that truly care and that want to
introduce a true-UTC timestamp can begin to do so, once there's a
defined way to fetch true UTC from the kernel.

I think the most important thing to fix is the ill-defined timing
anomalies we're putting up with once every 18 months or so when
pure-Posix clocks meet true-UTC leap seconds in an unignorable
way.  The right compromise, I think, is to smear them in a
decently-defined and ignorable way.  Google and Amazon have
supposedly implemented this already in their data centers, by
modifying NTP and leaving the OSes alone.  But I think it's well
worth exploring the opposite approach, too.

Steve Summit