[LEAPSECS] BBC radio Crowd Science

[Brooks Harris]

On 2017-01-31 04:44 PM, Warner Losh wrote:
> On Tue, Jan 31, 2017 at 2:20 PM, Brooks Harris <brooks at edlmax.com> wrote:
>>> It's the only way to encode one number such that the irregular radix
>>> math works out.
>> I think I'm going to need an example I can work through to fully understand
>> you here. When you get a chance, or can point me to something, thanks.
> OK.
>
>>> And treating a positive leap second 'special' that
>>> needs an extra decrement is isomorphic to changing the delta at the
>>> start of the second.
>> Ah, well the Leap Second is special, isn't it, even from the point of view
>> of your "irregular radix
>> math"? You are detecting it by whatever means and then treating it as if the
>> TAI-UTC value incremented and the start of the Leap Second so its convenient
>> to your methods, right? I mean, how its detected and treated could be seen
>> as an internal implementation choice.
>>
>> My "IsLeapSecond" flag can also be generated internally so long as there is
>> adequate metadata, either a IsLeapSecond or IsLeapSecondMinute, or
>> IsLeapSecondDay or something. But in any case you still need the TAI-UTC
>> value.
>>
>> The simple TAIToUTCDemoConsole program is not exactly how I do it in more
>> elaborate code, but, generally I aim to do everything with integer math and
>> logic.
>>
>> I wrote the little program to illustrate the resulting values, especially
>> the UTC seconds value, to explain what I meant by the YMDhms(TAI) and
>> YMDhms(UTC) and to stimulate just this convrsaion, thanks.
>>
>> I've found the IsLeapSecond flag to be very helpful metadata, especially
>> when a receiver of a timestamp is interpreting it. If its explicitly flagged
>> it saves any processing to detect it.
> I'm saying that the extra "is it a leap second" bit is a complication
> that's not needed for the computation. The fact that it may be useful
> for other things isn't really relevant.
>
> Let's take the TAI time 2017-01-01T00:00:36.5. Now, this is, as you
> know, the leap second. To compute what value this is, we take the new
> offset (37s) and subtract it, giving the following unreduced answer:
>
> 2017-01-01T00:00:(36.5-37)
> 2017-01-01T00:00:-0.5
>
> which we computed simply by subtracting 37 from the seconds field.
> We've not yet renormalized the notation. That's where the irregular
> radix stuff comes in. So, to normalize, we need to borrow one from the
> minutes column. To do that, we need to borrow one from the hours and
> so on. The minute we are borrowing from has 61 seconds, so we can
> rewrite this as
>
> 2016-12-31T23:59:(61-0.5)
> 2016-12-31T23.59:60.5
>
> which is the correct answer. We need only know the size of the minute
> we are borrowing from to make the computation.
>
> This is the same math, btw, you'd use to find an interval entirely
> inside UTC. What's the time difference between 2017-01-01T00:00:01 and
> 2016-12-31T23:59:58? The answer is 4. We can get this two ways. First,
> we can do it the tedious way of counting:
>
> 23:59:58
> 23:59:59 1s
> 23:59:60 2s
> 00:00:00 3s
> 00:00:01 4s --> Answer
>
> We can also do it by subtracting the two times:
>
>   2017-01-01T00:00:01-2016-12-31T23:59:58
>
> First step is to borrow from the previous minute, which rewrites it as
>
> 2016-12-31T23:59:(61+1)-2016-12031T23:59:58
> 2016-12-31T23:59:62-2016-12031T23:59:58
> proceeding to the subtraction, term by term, we get 00:00:04.
>
> In both cases, we did need to know that 23:59 had 61 seconds, but we
> didn't need to have a special leap second flag associated with any of
> the timestamps for the math to work out right. Knowing that it has 61
> seconds sometimes is exactly the same thing as knowing that sometimes
> February has 29 days some years. The only difference is that one is a
> math formula for thousands of years, and the other is a table lookup.
Oh. I see what you're doing now. Thanks much.

I was imagining things a little. I've been dealing with folks who are 
trying to run and prove these algorithms with tools like MatLab, so when 
you said "radix math" I was thinking "vector math". I've also spent 
months with Excel, so then everything is floating point and this really 
complicates things with handling precision errors. Excel also doesn't do 
%mod the way you need, so I had to write plugins. Ugh. These 
environments and tools are not well suited to timekeeping the way we're 
discussing it, in my opinion, but it becomes an important issue when 
grappling with communication of the subtleties. Each participant is 
approaching it from their particular environment.

(As a side note, there is complexity and miscommunication between the 
open source world and the Microsoft world, which doesn't help matters 
much. For example, I've got a big chore ahead sometime porting Tz 
Database to MSVC/Windows. We can't win or lose these OS wars, we just 
have to cope.)

So, approaching it your way will probably work fine, I don't doubt you. 
I see it as complicated, but I guess that's a matter of familiarity.

In my career I've dealt with these (crazy) SMPTE timecode timestamps. 
The nasty thing about them is they must support the NTSC frame rate of 
30000/1001. They span only the so-called "24 our working period", so no 
date is (usually) involved. But that rate has two nasty ramifications 1) 
that rate is a repeating decimal, 29.(970029), making accurate 
calculations tricky, and B) the hh:mm:ss portion of the 
hh:mm:ss:videoframes representation does not indicate true time. 
Uncompensated, the hh:mm:ss will get ahead of true time by about 3.5 
seconds per hour (it runs "slow" - 30000/1001 is 30 frames per 1.001 
seconds). SMPTE timecode has a "drop-frame count mode" to compensate. It 
"drops", or omits, two frame labels each minute except the tenth 
minutes. With that, the hh:mm:ss portion runs very nearly (not exactly) 
true hh:mm:ss running time. (It analogous to a *negative* Leap Second - 
labels are omitted, or "dropped").

I wrote a paper and c code demonstration about SMPTE timecode because 
there was a lot of confusion out there in the computer world about it. 
Note this code is written for illustration, usefully fast for general 
purposes, but not optimum for production code.

Conversion between SMPTE hh:mm:ss:ff Time Code and Frames
http://edlmax.com/SMPTETimeCodeConversion.htm

Now, SMPTE timecode in its native form encodes the address as 
hh:mm:ss:frames. So, typically, an application is receiving it in this 
form from some transport protocol (SMPTE timecode, in one form, is 
itself a synchronizing transport protocol), or embedded in some digital 
media, or something. To do something useful with it the best approach is 
to immediately convert it to a binary count of video frames. From there, 
all the many internal operations, usually involving duration 
calculations, can be done with simple integer math. In an editing system 
there are lots, and I mean lots, of calculations required, and they need 
to be *fast*. Of course you need to display these values to a human 
user, so you need to convert back to hh:mm:ss:ff, and eventually you'll 
probably need to generate native SMPTE timecode output, also need the 
address-to-hh:mm:ss:ff conversion.

So, I approach the general timekeeping topic from that perspective. I 
learned long ago not to mess with hh:mm:ss:ff based duration 
calculations. Those rates and that drop-frame count mode are really 
tricky to get exactly correct, and there are other frame rates (25/1 
PAL, 24/1 film, 24000/1001 film, audio 48000k sample rates, and many 
other things) and other complications that make hh:mm:ss:ff math 
entirely unmanageable. Just convert hh:mm:ss:ff to integer counts right 
away, convert back when you need to.

In general timekeeping we're faced with similar issues. The time may 
arrive as seconds (and decimal fractions) from protocols like NTP, PTP, 
or GPS, or as YMDhms from protocols like IRIG, WWVB, DCF77, and such. 
(By the way, SMPTE timecode was originally derived fro IRIG). In either 
case we need to perform some seconds-to-YMDhms (or the reverse) 
conversion. And there we encounter our friendly Gregorian calendar 
algorithm our favorite subject UTC with Leap Seconds.

Now, the Gregorian calendar algorithm is tricky, I think. Its much more 
complicated than SMPTE timecode and media manipulations, and more 
deceptive and confusing. Add the "fog of UTC" and here we are in this 
discussion.

So, you might see where I just wouldn't have considered doing it as 
hh:mm:ss manipulation. In most situations, and in the general case, your 
faced with running something like gmtime() and mktime() somewhere, and 
these need to be modified, or augmented, to support UTC. I suppose these 
could be implemented by some methods as you're using, but it looks like 
brain damage to me. Again, something of a matter of familiarity. So I'm 
gonna think about it more.

>
>> I'm not sure of that. It seems to me you are viewing it through a "irregular
>> radix math" lens, which I'm sure isn't bad, but its not the only approach.
> Sure, you can do it some crazy way that maps to how the numbers are
> counted, but is more complicated. My entire criticism of your approach
> boils down to "it's more complicated to separate out isALeap stuff"
Well, we need TAI-UTC and some sort of metadata to flag the Leap Second, 
either day, or hour, or minute, or second, right?
> since as I've conceded, the two methods are isomorphic since the +1
> that's needed on the offset with your method is added specially.
OK.
> It's
> on that basis that I say without equivocation the offset changes at
> the start of the leap second.
I see now what you're saying, I think. I see that *IF* the TAI-UTC 
increments and the start of the Leap Second you may not need 
"IsLeapSecond", that TAI-UTC holds the value conveniently; it 
essentially, implicitly, flags "is leap second".

But, if the TAI-UTC value increments *after* the Leap Second you need 
some flag to mark it, or to be able to figure which it is. Since I've 
come to the conclusion the specs say "TAI-UTC increments *after* the 
Leap Second" I've had to support that fact in the code. I still think 
that's how it is, but I'd sure like to be more confident of it.

>
>>> How you encode your internal tables is
>>> up to you, so long as they are isomorphic to what I'm saying.
>> Yes, I agree, any implementation could do it whatever way appropriate, but
>> I'm not sure about the TAI-UTC update topic yet.
>>
>> I'll bet we're getting the same results, at least as far as TAI
>> seconds-to-YMDhms(UTC) is concerned. If we're not there's an interoperabiliy
>> problem. Of course testing and verifying timekeeping algorithms is tedious.
>> But maybe we could make up a couple test examples and give generating a few
>> listings a go?
> If we get the same answer, we get the same answer. There's no
> interoperability issue. If your answers differ from mine, at least one
> of us is doing wrong. However, the internal representation is likely
> not too interesting.
Yes, well, I'm still concerned this "TAI-UTC increment" topic is an 
interoperability issue. In the design of any UTC capable timestamp, or 
in any distribution protocol, the TAI-UTC value is required, and when it 
updates seems crucial to me.

1861920036 TAI seconds can't be converted to YMDhms(UTC) without TAI-UTC 
and 2016-12-31 23:59:60 can't be converted to TAI seconds without 
TAI-UTC. If TAI-UTC = 37 you've got your answer, but if TAI-UTC = 36 you 
need something else. I think the spec says its TAI-UTC = 36.

-Brooks


>
> Warner
> _______________________________________________
> LEAPSECS mailing list
> LEAPSECS at leapsecond.com
> https://pairlist6.pair.net/mailman/listinfo/leapsecs
>
>