Critical Our monitoring reported that two zones were no longer being re-signed on Monday. We investigated our logs and found the following:
We noticed in our logs that the signer complained that it is unable to nsecify two of our zones. Further inspection of the logs showed that the Enforcer had generated new NSEC3 salt values prior to the signer showing this behaviour. Here are excerpts from our logs:
Enforcer reporting new signer configs with new salt values:
Jan 7 13:46:33 whitfield ods-signerd: [signconf] zone surfnet.nl signconf: RESIGN[PT7200S] REFRESH[PT432000S] VALIDITY[PT1209600S] DENIAL[PT1209600S] JITTER[PT43200S] OFFSET[PT3600S] NSEC[50] DNSKEYTTL[PT3600S] SOATTL[PT3600S] MINIMUM[PT3600S] SERIAL[datecounter] AUDIT[0]
Jan 7 13:46:33 whitfield ods-signerd: [signconf] zone surfnet.nl nsec3: OPTOUT[0] ALGORITHM[1] ITERATIONS[5] SALT[497b9bfc400211558792a583f9ebab3e4bae69ab]
Jan 7 13:46:34 whitfield ods-signerd: [signconf] zone surfnt.org signconf: RESIGN[PT7200S] REFRESH[PT432000S] VALIDITY[PT1209600S] DENIAL[PT1209600S] JITTER[PT43200S] OFFSET[PT3600S] NSEC[50] DNSKEYTTL[PT3600S] SOATTL[PT3600S] MINIMUM[PT3600S] SERIAL[datecounter] AUDIT[0]
Jan 7 13:46:34 whitfield ods-signerd: [signconf] zone surfnt.org nsec3: OPTOUT[0] ALGORITHM[1] ITERATIONS[5] SALT[497b9bfc400211558792a583f9ebab3e4bae69ab]
Signer complaining that it cannot nsecify:
Jan 7 13:47:40 whitfield ods-signerd: [denial] unable to create NSEC3 RR: failed to create bitmap
Jan 7 13:47:40 whitfield ods-signerd: [denial] unable to nsecify3: failed to create NSEC3 RR
Jan 7 13:47:40 whitfield ods-signerd: [data] unable to nsecify3: failed to add NSEC3 record
Jan 7 13:47:40 whitfield ods-signerd: [worker[2]] backoff task [nsecify] for zone surfnet.nl with 120 seconds
Jan 7 13:47:46 whitfield ods-signerd: [denial] unable to create NSEC3 RR: failed to create bitmap
Jan 7 13:47:46 whitfield ods-signerd: [denial] unable to nsecify3: failed to create NSEC3 RR
Jan 7 13:47:46 whitfield ods-signerd: [data] unable to nsecify3: failed to add NSEC3 record
Jan 7 13:47:46 whitfield ods-signerd: [worker[2]] backoff task [nsecify] for zone surfnt.org with 120 seconds
The signer queue looks like this:
It is now Wed Jan 9 11:51:57 2013
I have 12 tasks scheduled.
On Wed Jan 9 11:52:45 2013 I will [nsecify] zone surfnet.nl
On Wed Jan 9 11:52:46 2013 I will [nsecify] zone surfnt.org
On Wed Jan 9 13:37:44 2013 I will [sign] zone dnshealth.info
On Wed Jan 9 13:37:44 2013 I will [sign] zone tuxed.net
On Wed Jan 9 13:37:44 2013 I will [sign] zone dnssechealth.nl
On Wed Jan 9 13:37:44 2013 I will [sign] zone dnssecmonitor.nl
On Wed Jan 9 13:37:44 2013 I will [sign] zone gigaport.nl
On Wed Jan 9 13:37:44 2013 I will [sign] zone gigaport3.nl
On Wed Jan 9 13:37:44 2013 I will [sign] zone dnshealth.org
On Wed Jan 9 13:37:44 2013 I will [sign] zone buildfarm.opendnssec.org
On Wed Jan 9 13:37:44 2013 I will [sign] zone surfdnssec.org
On Wed Jan 9 13:46:51 2013 I will [sign] zone breedbandzuil.nl
We noticed that there are no longer tasks for signing the two zones concerned (surfnt.org and surfnet.nl) so we assume that the zone is not getting re-signed (which explains our monitoring complaining about signature expiration).
I've attached an archive with the zone files, signed files, intermediate files, old signer config and new signer config.
NOTE: several other zones received new salt values as well but were nsecified and signed without problems 