[LEAPSECS] presentations from AAS Future of Time sessions

[Warner Losh]

On Jan 11, 2014, at 8:28 PM, Brooks Harris wrote:


> On 2014-01-11 08:32 AM, Joseph Gwinn wrote:

>> On Fri, 10 Jan 2014 21:35:25 -0700, Warner Losh wrote:

>> 

>>> On Jan 10, 2014, at 8:35 PM, Skip Newhall wrote:

>>> 

>>> 

>>>> 'Proscribe’ and 'prescribe' are distinct words:

>>>>  

>>>> 'Proscribe' means to forbid, disallow, or prohibit. “School rules 

>>>> proscribe the use of pencils on exams.”

>>>>  

>>>> 'Prescribe' means to lay out specifications or rules about 

>>>> something. "In the manner prescribed by law.”

>>>>  

>>>> I don’t know the context of the sentence the Magnus refers to.

>>>> 

>>> Prescribe is the word I intended. POSIX mandates, requires, 

>>> prescribes that time is UTC.

>>> 

>> No.  While POSIX broken-down time resembles UTC, the underlying 

>> timescale (Seconds since the Epoch) is *not* UTC, in that every day 

>> contains exactly 60*60*24= 86,400 seconds.  Leap seconds are *not* 

>> applied.

>> 

>> The confusion arises because the POSIX Epoch is defined using UTC.  But 

>> the progression rule is a form of TAI (with unknown but constant 

>> offset).  POSIX background information follows.

>> 

>> 

>> URL of index page: 

>> <http://pubs.opengroup.org/onlinepubs/009695399/>

>> 

>> 

>> 

>> >From POSIX Base Definitions volume: 

>> 

>> 3.149 Epoch

>> The time zero hours, zero minutes, zero seconds, on January 1, 1970 

>> Coordinated Universal Time (UTC).

>> 

>> 

>> >From the General Concepts volume:

>> 

>> 4.14 Seconds Since the Epoch

>> A value that approximates the number of seconds that have elapsed since 

>> the Epoch. A Coordinated Universal Time name (specified in terms of 

>> seconds (tm_sec), minutes (tm_min), hours (tm_hour), days since January 

>> 1 of the year (tm_yday), and calendar year minus 1900 (tm_year)) is 

>> related to a time represented as seconds since the Epoch, according to 

>> the expression below.

>> 

>> If the year is <1970 or the value is negative, the relationship is 

>> undefined. If the year is >=1970 and the value is non-negative, the 

>> value is related to a Coordinated Universal Time name according to the 

>> C-language expression, where tm_sec, tm_min, tm_hour, tm_yday, and 

>> tm_year are all integer types:

>> 

>> tm_sec + tm_min*60 + tm_hour*3600 + tm_yday*86400 +

>>     (tm_year-70)*31536000 + ((tm_year-69)/4)*86400 -

>>     ((tm_year-1)/100)*86400 + ((tm_year+299)/400)*86400

>> 

>> The relationship between the actual time of day and the current value 

>> for seconds since the Epoch is unspecified.

>> How any changes to the value of seconds since the Epoch are made to 

>> align to a desired relationship with the current actual time is 

>> implementation-defined. As represented in seconds since the Epoch, each 

>> and every day shall be accounted for by exactly 86400 seconds.

>> 

>> Note:

>> The last three terms of the expression add in a day for each year that 

>> follows a leap year starting with the first leap year since the Epoch. 

>> The first term adds a day every 4 years starting in 1973, the second 

>> subtracts a day back out every 100 years starting in 2001, and the 

>> third adds a day back in every 400 years starting in 2001. The 

>> divisions in the formula are integer divisions; that is, the remainder 

>> is discarded leaving only the integer quotient.

>> 

>> 

>> History:  

>> <http://en.wikipedia.org/wiki/Unix_time>

>> 

>> 

>> Joe Gwinn  

>> _______________________________________________

>> LEAPSECS mailing list

>> 

>> LEAPSECS at leapsecond.com

>> http://six.pairlist.net/mailman/listinfo/leapsecs

>> 

>> 

>> 

>> 

> 

> Yes, in my opinion its unfortunate they chose to use the term "UTC" in that context.

> 

> What I've never seen written anywhere is something rather obvious - that the formula shown in 4.14 Seconds Since the Epoch is the formula for the Gregorian calendar timescale.. mktime() and gmtime() (and related) treat the time_t Seconds value as an uncompensated-for-Leap-Seconds Gregorian calendar timescale as defined and codified in ISO 8601, section 3.2.1 The Gregorian calendar.

> 

> Its actually explained better in the "Rationale: Base Definitions", section, A.4.15 Seconds Since the Epoch. -

> 

> IEEE Std 1003.1, 2013 Edition

> http://pubs.opengroup.org/onlinepubs/9699919799/

> (Select "Rationale" in the upper left pane, then "Rationale for Base Definitions" in the lower-left pane, scroll down to  A.4.15 Seconds Since the Epoch) 

> 

> There they say "Broken-down POSIX time is therefore not necessarily UTC, despite its appearance." Note the use of the phrase "POSIX time", a much better term for it, in my opinion., since that phrase encompasses the counting method and the definition of its origin - "the Epoch".  But that's not the end of the story.

> 

> In the absence of any Leap Second compensation applied to the time_t value, the "appearance" is the Gregorian calendar timescale. But the phrase "not necessarily UTC" signals that *if* Leap Second compensation has been applied to time_t and proper adjustments are made, mktime() and gmtime() may indeed return proper UTC. But this is true *only* on the "GMT" timezone timescale because how Leap Seconds are applied to, or accrued to, or instantiated on, other "timezones" is undefined.

> 

> "the Epoch" is another critical point about the use, or misuse, of "POSIX time". Section 3.150 Epoch defines it.

> (Select "Base Definitions" in upper-left pane, "Definitions" in lower-left pane, scroll down to Section 3.150 Epoch)

> There we see "3.150 Epoch The time zero hours, zero minutes, zero seconds, on January 1, 1970 Coordinated Universal Time (UTC).."

> 

> This is a very vague definition of the origin called "the Epoch". When, exactly, was January 1, 1970 Coordinated Universal Time (UTC).?

> 

> Currently there are three standards bodies responsible for the definition, maintenance of, and dissemination of UTC (proper) - BIPM,, ITU, and IERS. These bodies have inherited the work and responsibilities form many previous standards bodies involved in the history of the development of UTC as we know it today.

> 

> Here, we're interested in the definition of the origin of UTC to help us understand what the POSIX "Epoch" might actually be.

> 

> We know the offset between TAI and UTC was "steered" such that the offset became a integral Seconds offset of 10 Seconds in 1972 (1972-01-01T00:00:00Z). That's when Leap Seconds began.

> 

> 1972-01-01T00:00:00Z (UTC) = 1972-01-01T00:00:10 (TAI)

> 

> We know this from studying the history and reading the specs very carefully. But there's not a clear statement of that fact in any one place.

> 

> The origin of TAI was *retroactively* set at 1958-01-01 00:00:00. This is described in metrologia-leapsecond: The leap second: its history and possible future, and mentioned in Steve Allen's "Time Scales" http://www.ucolick.org/~sla/leapsecs/timescales.html, see "proleptic "TAI" on 1958-01-01". (By the way, I cannot locate the actual document that normatively states that, any assistence in finding the reference appreciated...)

> 

> So that essentially establishes a proleptic TAI timescale from 1958-01-0100:00:00 (TAI) to 1972-01-01T00:00:10 (TAI).


I don't think TAI is proleptic during that time.

I now that Loran C specifies Jan 1, 1958 as its epic (http://www.navcen.uscg.gov/pdf/loran/sigSpec/chapt2c.pdf). 

Loran was in operation from the 1960's onward. It follows UTC  through 1972, but never did leap seconds.

http://hpiers.obspm.fr/eop-pc/earthor/utc/TAI-UTC_tab.html looks to be the official TAI UTC offset table, which suggests strongly TAI and UTC diverged prior to 1961 (when the offset was 1.4228180). This also shows that from 1961-1971 TAI and UTC ticked at different rates.


> So when was POSIX "The time zero hours, zero minutes, zero seconds, on January 1, 1970 Coordinated Universal Time (UTC)"? 1970 is *before* the establishment UTC proper in 1972.. So what does 1970-01-01 00:00:00 mean?



> There's two choices - either the non-integral second value in the historical record as indicated in the BIPM tables (very tricky and controversial to calculate that correctly), or exactly 2 (uncompensated) years before 1972-01-01T00:00:00Z (UTC) on a proleptic UTC timescale.

> 

> Apparently, the POSIX specification *intentionally* allows either. Section 4.15 Seconds Since the Epoch states -

> 

> "The relationship between the actual time of day and the current value for seconds since the Epoch is unspecified.

> How any changes to the value of seconds since the Epoch are made to align to a desired relationship with the current actual time is implementation-defined."

> 

> That's a lot of wriggle room. And intentionally so, because so many Unix/Linux implementations are in the field with different, perhaps unknown, "relationship with the current actual time". It depends on the systems time source, the kernel implementation, and administrator policy.

> 

> So now I've spent weeks, months, studying standards and I'm still not sure of the answer! In that environment is it any wonder there's a desire to "kill Leap Seconds"?


Try to implement a system that gets them right when you have dependence on third parties to get them right....


> It seems like UTC and Leap Seconds are the problem, since its pretty tricky. But notice - even if you kill Leap Seconds you have not solved the problem - "the Epoch" remains undefined, so, ah, what time did you say it was, again? "Kill Leap Seconds" doesn't really have anything to do with the problem at all - its firstly a definitive definition of "the Epoch".


Yes. That precision doesn't stop people from getting the current time right, but there's a bit of an ambiguity of a few seconds of the TAI time of the start of the UNIX epoch...


> There *is* a specification we can reach to to resolve this dilemma - the NTP spec. 

> 

> NTP (IETF rfc5905) very clearly establishes its "prime epoch" as 1900-01-01T00:00:00Z (UTC) on a proleptic UTC timescale -

> 

> "In the date and timestamp formats, the prime epoch, or base date of era 0, is 0 h 1 January 1900 UTC, when all bits are zero.  It should be noted that strictly speaking, UTC did not exist prior to 1 January 1972, but it is convenient to assume it has existed for all eternity, even if all knowledge of historic leap seconds has been lost.  Dates are relative to the prime epoch; values greater than zero represent    times after that date; values less than zero represent times before it. "

> 

> Then, and very importantly,  Figure 4: Interesting Historic NTP Dates states the relationship to "First day UNIX" - 

> 

>   +-------------+------------+-----+---------------+------------------+

>    | Date        | MJD        | NTP | NTP Timestamp | Epoch            |

>    |             |            | Era | Era Offset    |                  |

>    +-------------+------------+-----+---------------+------------------+

>    | 1 Jan -4712 | -2,400,001 | -49 | 1,795,583,104 | 1st day Julian   |

>    | 1 Jan -1    | -679,306   | -14 | 139,775,744   | 2 BCE            |

>    | 1 Jan 0     | -678,491   | -14 | 171,311,744   | 1 BCE            |

>    | 1 Jan 1     | -678,575   | -14 | 202,939,144   | 1 CE             |

>    | 4 Oct 1582  | -100,851   | -3  | 2,873,647,488 | Last day Julian  |

>    | 15 Oct 1582 | -100,840   | -3  | 2,874,597,888 | First day        |

>    |             |            |     |               | Gregorian        |

>    | 31 Dec 1899 | 15019      | -1  | 4,294,880,896 | Last day NTP Era |

>    |             |            |     |               | -1               |

>    | 1 Jan 1900  | 15020      | 0   | 0             | First day NTP    |

>    |             |            |     |               | Era 0            |

>    | 1 Jan 1970  | 40,587     | 0   | 2,208,988,800 | First day UNIX   |

>    | 1 Jan 1972  | 41,317     | 0   | 2,272,060,800 | First day UTC    |

>    | 31 Dec 1999 | 51,543     | 0   | 3,155,587,200 | Last day 20th    |

>    |             |            |     |               | Century          |

>    | 8 Feb 2036  | 64,731     | 1   | 63,104        | First day NTP    |

>    |             |            |     |               | Era 1            |

>    +-------------+------------+-----+---------------+------------------+

> 

>                  Figure 4: Interesting Historic NTP Dates

> 

> This puts "Unix time" at exactly 2 (uncompensated) years (2 years * 365 days * 86400 Seconds) = 63072000 Seconds before 1972-01-01T00:00:00Z (UTC).


That's one possible definition, but I think the most common one.


> Eureka! If "Unix time" is the same as "POSIX time" (this seems to be a general assumption, but I know of nothing official that says so) we know when the POSIX "the Epoch" would ideally be set to "to align to a desired relationship with the current actual time".

> 

> I suspect many, maybe most, OSs and systems, including Unix/Linux seek to align to that definition, since its really the only one that makes any sense for a practical implementation. Perhaps others on the list could comment on that?

> 

> But the connection from the fractured UTC specs through NTP to POSIX is *unofficial*. 

> 

> So, through all this you can see one important part of the problem - fractured standards and inconsistent terminology. This confusion leads to misinterpretation and controversy, which in turn inhibits attempts to refine standards.


I'd agree with this...


> Can't someone please state this in a credible standard so I don't have to do months of detective work and wonder if I got it right?


I'd love to see that too...

Warner