Withdrawing issued certificates - approach and possibilities

[wisbech]

Withdraw approach

As described in issue #317

We are currently facing an issue with the withdrawal functionality of a certificate in case of faulty readings. The problem arises if there have been state changes on the certificates in the form of transactions. These transactions are, by nature, immutable in Project Origin.

This immutability presents a challenge when we need to withdraw a certificate due to faulty readings. We are unable to reverse or modify the state changes once they have been made.

This diagram shows the sequence of events:

• A state change occurs on the certificate in the form of a transaction.
• The transaction is immutable, meaning it cannot be changed once it has been made.
• The system detects a faulty reading on the certificate.
• The system attempts to withdraw the certificate.
• The system is unable to reverse or modify the state changes on the certificate, leading to the issue.

sequenceDiagram
    participant C as Certificate
    participant T as Transaction
    participant S as System
    C->>T: State change (Transaction)
    Note over C,T: Transaction is immutable
    S->>C: Detects faulty reading
    S->>C: Attempts to withdraw certificate
    Note over S,C: Unable to reverse or modify state changes

Loading

We are actively exploring different strategies to address this issue. The strategies, their feasibility, and our verdict on each are outlined in the table below:

Strategy	Description	Feasibility	Verdict
Transaction Reversal	Implement a new transaction type that signifies the reversal of a previous transaction. This would not alter the original transaction but would effectively negate its impact.	TBD	TBD
Faulty Certificate Flagging	Introduce a mechanism to flag certificates that have been issued based on faulty readings. This would not withdraw the certificate but would indicate to any potential users or systems that the certificate is not valid.	TBD	TBD
Certificate Replacement	Issue a new, corrected certificate to replace the faulty one. The original certificate could be flagged as faulty to prevent its use.	TBD	TBD
Dual Transaction Approach	For every transaction, create a 'shadow' transaction that can be used to reverse the original transaction if necessary. This would double the number of transactions but would provide a straightforward way to handle faulty readings.	TBD	TBD
Certificate Replacement with Claim ID Reference	Issue a new, corrected certificate to replace the faulty one. The original certificate could be flagged as faulty to prevent its use. The new certificate would reference the claim IDs of the original certificate to maintain a link between the two. This would allow users or systems to trace the new certificate back to the original one, providing a clear audit trail.	TBD	TBD

[wisbech]

At a meetup we have the following notes on the issue of the wicked problem of withdrawal of a certificate.

Constrains:

• Expiry 2 months
• Measurements can be corrected up to 3 years post observation
  • The measurement can be either
    • Lower 👎 in volume
    • Higher 👍 in volume
    • Wrongly issued

Requirements:

• Certificates can only be issued on observed measures, not estimates.
• An actual measure of withdrawals in % of total measures in a system would help the business case.

Measures to mitigate:

• In case of faulty measurements in the underlying data, the certificate is withdrawn from and a new is issued to the producer
  • With an expiry of two months from issuance
• That all consumption certificates never expires
• Recall / revocation procedure
  • A broadcast of invalid certificates
    • Terms and conditions for handling broadcast
    • Chronicler can store records of all individual certificates for reporting purposes
• Disputes should be handled by market actors

Handling certificate in withdrawal process of a certificate

• complete revocation
• revoke in case of 👎 lower volume measurement
• adding a new certificate in case of 👍 higher volume measurement
  • passing a reference to the newly issued certificate

[tnickelsen]

I see that EnergyTag does corrections the same way that I understand the GO's do. If we choose to do it differently than EnergyTag, does that mean that we are not EnergyTag compliant like Michael Gimm wants us to be?

[wisbech]

@wisbech 'That all production certifcates never expires' - you mean consumption, right?

You are right @tnickelsen - I will correct it right away

[wisbech]

I see that EnergyTag does corrections the same way that I understand the GO's do. If we choose to do it differently than EnergyTag, does that mean that we are not EnergyTag compliant like Michael Gimm wants us to be?

If we are breaching the premise of time-volume based certification we have an issue yes - this should be adressed in the conversations we have with Energytag.

[tnickelsen]

I see that EnergyTag does corrections the same way that I understand the GO's do. If we choose to do it differently than EnergyTag, does that mean that we are not EnergyTag compliant like Michael Gimm wants us to be?

If we are breaching the premise of time-volume based certification we have an issue yes - this should be adressed in the conversations we have with Energytag.

Alright so we aim to change this in the EnergyTag standard, if we decide on withdraw as we discussed?

[Soped42]

I see that EnergyTag does corrections the same way that I understand the GO's do. If we choose to do it differently than EnergyTag, does that mean that we are not EnergyTag compliant like Michael Gimm wants us to be?

Reading "Annex 4 Error Handling for Ex-Post Corrections of Meter Data (Informative)" I see that they are generally following the same directions as we are intending to go.
However as we work with a fix-forward system, we will have to deviate slightly.

I don't think comparing too much to GO's is going to get us far, as they are very different in their nature in terms of volume vs location and time based GC's.

I guess it's also worth noticing that EnergyTags approach is a recommendation not a requirement to be compliant.

Page 37 of https://energytag.org/wp-content/uploads/2023/09/Granular-Certificate-Scheme-Standard-V2.pdf
"GCs are always issued after the time of production
and based on measurement data. In practice however,
errors can be detected in this meter data. When this
occurs, the procedure below and further detailed in
Annex 4 Error Handling for Ex-post Corrections of
Meter Data is recommended."

[wisbech]

Bulletpoints around data quality discussion from the meetup "Project Origin gettogether" 26.06.2024

Data Quality

• Corrections up to 3 years after measurement
  • Within 3 months, 5% of all measurements are corrected
  • After 3 months, there are less than 1% corrections

Balancing Market

• After 1 month (Refixing), there are no changes - the window is locked.

Issuance of Certificates

• Certificates should be issued based on settlement (observations and estimates) and not solely on measurements (observations)
• Settlement and certification of energy would be incomprehensible if they were solely based on measurements, as they can be incomplete and in some cases impossible to obtain.

Measurement Data

Estimate over a day

• Over 24 hours
• If 23 hours are measured and 1 hour is not, the 1 hour is estimated
  • All 24 hours are labeled as estimated with the lowest data quality

Wholesale Settlement

• 5 working days after
• 1 month afterwards

Balance Fixing

• Almost no changes from 8 to 13 calendar days - moving towards 8.

Correction Data Closure

1. 3 months
2. 18 months
3. 36 months

Master Data Errors

• Such as incorrect meter reported

• +- 1% error on existing meter

• Transmission loss of 4%

Best Effort

• Sliding window for certificate lifespan and making it data-driven based on the amount of corrections
• We will not achieve complete accuracy as we cannot track electrons
• Minimum thresholds for corrections need to be set
  • Minimum amounts of Wh
  • Corrections of master data errors
  • Corrections should be done through the data hub's time series
  • After balance fixing

Greater Insight into Consumption and Production

• The data hub is benchmarked
  • E17 - production
  • E18 - consumption
• Requires flexible settlement (hourly based)