docs/obs-implementation.rst
author Matt Harbison <matt_harbison@yahoo.com>
Thu, 05 Mar 2015 20:02:07 -0500
branchstable
changeset 1209 fa35aeb64d32
parent 519 9825c7da5b54
permissions -rw-r--r--
evolve: prevent a crash in httpclient_pushobsmarkers() when pushing I've been running into a crash when pushing from my hg repo in a Fedora 16 VM to Win7 running 'hg serve', even with extensions disabled on both sides: ../hg push -r . pc pushing to http://192.168.1.4:8000/ searching for changes no changes found pushing 2 obsolescence markers (263 bytes) ** unknown exception encountered, please report by visiting ... File "hg-evolve/hgext/evolve.py", line 2482, in _pushobsolete remote.evoext_pushobsmarkers_0(obsdata) File "hg-evolve/hgext/evolve.py", line 2522, in httpclient_pushobsmarkers ret, output = self._call('evoext_pushobsmarkers_0', data=obsfile) ValueError: too many values to unpack I'm not sure how this repo differs from the one in the test suite, so I'm not sure how to craft a test for this. The failure occurs even when there _are_ csets to push. There was no crash if no obsolete markers needed to be pushed. At any rate, this code was stolen from httppeer._callpush(), where it calls self._call(). The socket exception handling wasn't necessary to fix the crash, but the calling code might as well be duplicated in its entirety. A successful push with this patch looks like this. Note the final line is _not_ in the output of the http push in test-simple4server.t: ../hg push -r . pc pushing to http://192.168.1.4:8000/ searching for changes remote has heads on branch 'default' that are not known locally: 3af110194a0c 56000e3ae44d 57ac6e51d290 7da4355c21b8 and 8 others remote: adding changesets remote: adding manifests remote: adding file changes remote: added 1 changesets with 0 changes to 1 files (+1 heads) pushing 4 obsolescence markers (525 bytes) remote: 2 obsolescence markers added

.. Copyright 2011 Pierre-Yves David <pierre-yves.david@ens-lyon.org>
..                Logilab SA        <contact@logilab.fr>

-----------------------------------------------------
Implementation of Obsolete Marker
-----------------------------------------------------
.. warning:: This document is still in heavy work in progress

Main questions about Obsolete Marker Implementation
-----------------------------------------------------




How shall we exchange Marker over the Wire ?
`````````````````````````````````````````````````````````

We can have a lot of markers. We do not want to exchange data for the one we
already know. Listkey() is not very appropriate there as you get everything.

Moreover, we might want to only hear about Marker that impact changeset we are
pulling.

pushkey is not batchable yet (could be fixed)

A dedicated discovery and exchange protocol seems mandatory here.


Various technical details
-----------------------------------------------------

Some stuff that worse to note. some may deserve their own section later.

storing old changeset
``````````````````````

The new general delta format allows a very efficient storage of two very similar
changesets. Storing obsolete children using general delta takes no more place
than storing the obsolete diff. Reverted file will even we reused. The whole
operation will take much less space the strip backup.


Abstraction from history rewriting UI
```````````````````````````````````````````

How Mercurial handles obsolete marker is independent from what decides
to create them and what actual operation solves the error case. Any of
the existing history rewriting UI (rebase, mq, histedit) can lay
obsolete markers and resolve situation created by others. To go
further, a hook system of obsolete marker creation would allow each
mechanism to collaborate with other though a standard and central
mechanism.


Obsolete marker storage
```````````````````````````

The Obsolete marker will most likely be stored outside standard
history. They are multiple reasons for this:

First, obsolete markers are really perpendicular to standard history
there is no strong reason to include it here other than convenience.

Second, storing obsolete marker inside standard history means:

* A changeset must be created every time an obsolete relation is added. Very
  inconvenient for delete operation.

* Obsolete marker must be forged at the creation of the new changeset. This
  is very inconvenient for split operation. And in general it becomes
  complicated to fix history afterward in particular when working with older
  clients.

Storing obsolete marker outside history have several pros:

* It eases Exchange of obsolete markers without unnecessary obsolete
  changeset contents.

* It allows tuning the actual storage and protocol exchange while maintaining
  compatibility with older clients through the wire (as we do the repository
  format).

* It eases the exchange of obsolete related information during
  discovery to exchange obsolete changeset relevant to conflict
  resolution. Exchanging such information deserves a dedicated
  protocol.

Persistent
```````````````````````

*Extinct* changeset and obsolete marker will most likely be garbage collected as
some point. However, archive server may decide to keep them forever in order to
keep a fully auditable history in its finest conception.


Current status
-----------------------------------------------------

Obsolete marker are partialy in core.

2.3:

- storage over obsolete marker
- exchange suing pushkey
- extinct changeset are properly hidden
- extinct changeset are excluded from exchange