docs/obs-implementation.rst
author Pierre-Yves David <pierre-yves.david@fb.com>
Mon, 17 Nov 2014 16:04:48 +0100
branchstable
changeset 1167 a1f5a948420d
parent 519 9825c7da5b54
permissions -rw-r--r--
test: ensure we echo prompt Tool running using the shipped 'run-tests.py' instead of the mercurial one would get different result otherwise.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
519
9825c7da5b54 ensure all file have a copyright notice
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 501
diff changeset
     1
.. Copyright 2011 Pierre-Yves David <pierre-yves.david@ens-lyon.org>
9825c7da5b54 ensure all file have a copyright notice
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 501
diff changeset
     2
..                Logilab SA        <contact@logilab.fr>
161
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
     3
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
     4
-----------------------------------------------------
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
     5
Implementation of Obsolete Marker
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
     6
-----------------------------------------------------
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
     7
.. warning:: This document is still in heavy work in progress
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
     8
244
9d37254031fc [doc] add Question and Answer about Obsolet Marker Implementation
Pierre-Yves.David@ens-lyon.org
parents: 228
diff changeset
     9
Main questions about Obsolete Marker Implementation
9d37254031fc [doc] add Question and Answer about Obsolet Marker Implementation
Pierre-Yves.David@ens-lyon.org
parents: 228
diff changeset
    10
-----------------------------------------------------
9d37254031fc [doc] add Question and Answer about Obsolet Marker Implementation
Pierre-Yves.David@ens-lyon.org
parents: 228
diff changeset
    11
9d37254031fc [doc] add Question and Answer about Obsolet Marker Implementation
Pierre-Yves.David@ens-lyon.org
parents: 228
diff changeset
    12
246
1e8e32d3871c doc: idea about OnDisk Storage
Pierre-Yves.David@ens-lyon.org
parents: 244
diff changeset
    13
244
9d37254031fc [doc] add Question and Answer about Obsolet Marker Implementation
Pierre-Yves.David@ens-lyon.org
parents: 228
diff changeset
    14
9d37254031fc [doc] add Question and Answer about Obsolet Marker Implementation
Pierre-Yves.David@ens-lyon.org
parents: 228
diff changeset
    15
How shall we exchange Marker over the Wire ?
9d37254031fc [doc] add Question and Answer about Obsolet Marker Implementation
Pierre-Yves.David@ens-lyon.org
parents: 228
diff changeset
    16
`````````````````````````````````````````````````````````
9d37254031fc [doc] add Question and Answer about Obsolet Marker Implementation
Pierre-Yves.David@ens-lyon.org
parents: 228
diff changeset
    17
9d37254031fc [doc] add Question and Answer about Obsolet Marker Implementation
Pierre-Yves.David@ens-lyon.org
parents: 228
diff changeset
    18
We can have a lot of markers. We do not want to exchange data for the one we
9d37254031fc [doc] add Question and Answer about Obsolet Marker Implementation
Pierre-Yves.David@ens-lyon.org
parents: 228
diff changeset
    19
already know. Listkey() is not very appropriate there as you get everything.
9d37254031fc [doc] add Question and Answer about Obsolet Marker Implementation
Pierre-Yves.David@ens-lyon.org
parents: 228
diff changeset
    20
9d37254031fc [doc] add Question and Answer about Obsolet Marker Implementation
Pierre-Yves.David@ens-lyon.org
parents: 228
diff changeset
    21
Moreover, we might want to only hear about Marker that impact changeset we are
9d37254031fc [doc] add Question and Answer about Obsolet Marker Implementation
Pierre-Yves.David@ens-lyon.org
parents: 228
diff changeset
    22
pulling.
9d37254031fc [doc] add Question and Answer about Obsolet Marker Implementation
Pierre-Yves.David@ens-lyon.org
parents: 228
diff changeset
    23
9d37254031fc [doc] add Question and Answer about Obsolet Marker Implementation
Pierre-Yves.David@ens-lyon.org
parents: 228
diff changeset
    24
pushkey is not batchable yet (could be fixed)
9d37254031fc [doc] add Question and Answer about Obsolet Marker Implementation
Pierre-Yves.David@ens-lyon.org
parents: 228
diff changeset
    25
9d37254031fc [doc] add Question and Answer about Obsolet Marker Implementation
Pierre-Yves.David@ens-lyon.org
parents: 228
diff changeset
    26
A dedicated discovery and exchange protocol seems mandatory here.
9d37254031fc [doc] add Question and Answer about Obsolet Marker Implementation
Pierre-Yves.David@ens-lyon.org
parents: 228
diff changeset
    27
9d37254031fc [doc] add Question and Answer about Obsolet Marker Implementation
Pierre-Yves.David@ens-lyon.org
parents: 228
diff changeset
    28
161
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    29
Various technical details
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    30
-----------------------------------------------------
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    31
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    32
Some stuff that worse to note. some may deserve their own section later.
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    33
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    34
storing old changeset
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    35
``````````````````````
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    36
244
9d37254031fc [doc] add Question and Answer about Obsolet Marker Implementation
Pierre-Yves.David@ens-lyon.org
parents: 228
diff changeset
    37
The new general delta format allows a very efficient storage of two very similar
357
b398e9c2dbd1 doc: English proofreading
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 311
diff changeset
    38
changesets. Storing obsolete children using general delta takes no more place
161
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    39
than storing the obsolete diff. Reverted file will even we reused. The whole
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    40
operation will take much less space the strip backup.
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    41
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    42
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    43
Abstraction from history rewriting UI
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    44
```````````````````````````````````````````
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    45
357
b398e9c2dbd1 doc: English proofreading
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 311
diff changeset
    46
How Mercurial handles obsolete marker is independent from what decides
b398e9c2dbd1 doc: English proofreading
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 311
diff changeset
    47
to create them and what actual operation solves the error case. Any of
b398e9c2dbd1 doc: English proofreading
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 311
diff changeset
    48
the existing history rewriting UI (rebase, mq, histedit) can lay
b398e9c2dbd1 doc: English proofreading
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 311
diff changeset
    49
obsolete markers and resolve situation created by others. To go
b398e9c2dbd1 doc: English proofreading
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 311
diff changeset
    50
further, a hook system of obsolete marker creation would allow each
b398e9c2dbd1 doc: English proofreading
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 311
diff changeset
    51
mechanism to collaborate with other though a standard and central
b398e9c2dbd1 doc: English proofreading
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 311
diff changeset
    52
mechanism.
161
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    53
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    54
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    55
Obsolete marker storage
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    56
```````````````````````````
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    57
357
b398e9c2dbd1 doc: English proofreading
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 311
diff changeset
    58
The Obsolete marker will most likely be stored outside standard
b398e9c2dbd1 doc: English proofreading
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 311
diff changeset
    59
history. They are multiple reasons for this:
161
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    60
357
b398e9c2dbd1 doc: English proofreading
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 311
diff changeset
    61
First, obsolete markers are really perpendicular to standard history
b398e9c2dbd1 doc: English proofreading
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 311
diff changeset
    62
there is no strong reason to include it here other than convenience.
161
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    63
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    64
Second, storing obsolete marker inside standard history means:
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    65
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    66
* A changeset must be created every time an obsolete relation is added. Very
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    67
  inconvenient for delete operation.
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    68
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    69
* Obsolete marker must be forged at the creation of the new changeset. This
357
b398e9c2dbd1 doc: English proofreading
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 311
diff changeset
    70
  is very inconvenient for split operation. And in general it becomes
161
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    71
  complicated to fix history afterward in particular when working with older
357
b398e9c2dbd1 doc: English proofreading
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 311
diff changeset
    72
  clients.
161
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    73
357
b398e9c2dbd1 doc: English proofreading
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 311
diff changeset
    74
Storing obsolete marker outside history have several pros:
161
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    75
357
b398e9c2dbd1 doc: English proofreading
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 311
diff changeset
    76
* It eases Exchange of obsolete markers without unnecessary obsolete
b398e9c2dbd1 doc: English proofreading
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 311
diff changeset
    77
  changeset contents.
161
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    78
357
b398e9c2dbd1 doc: English proofreading
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 311
diff changeset
    79
* It allows tuning the actual storage and protocol exchange while maintaining
b398e9c2dbd1 doc: English proofreading
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 311
diff changeset
    80
  compatibility with older clients through the wire (as we do the repository
b398e9c2dbd1 doc: English proofreading
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 311
diff changeset
    81
  format).
161
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    82
357
b398e9c2dbd1 doc: English proofreading
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 311
diff changeset
    83
* It eases the exchange of obsolete related information during
b398e9c2dbd1 doc: English proofreading
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 311
diff changeset
    84
  discovery to exchange obsolete changeset relevant to conflict
b398e9c2dbd1 doc: English proofreading
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 311
diff changeset
    85
  resolution. Exchanging such information deserves a dedicated
b398e9c2dbd1 doc: English proofreading
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 311
diff changeset
    86
  protocol.
161
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    87
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    88
Persistent
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    89
```````````````````````
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    90
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    91
*Extinct* changeset and obsolete marker will most likely be garbage collected as
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    92
some point. However, archive server may decide to keep them forever in order to
357
b398e9c2dbd1 doc: English proofreading
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 311
diff changeset
    93
keep a fully auditable history in its finest conception.
161
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    94
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    95
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    96
Current status
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    97
-----------------------------------------------------
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
    98
501
9a17c48c2099 update implementation details
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 357
diff changeset
    99
Obsolete marker are partialy in core.
161
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
   100
501
9a17c48c2099 update implementation details
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 357
diff changeset
   101
2.3:
161
4e3f25ba5401 More doc and index with sphynx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
diff changeset
   102
501
9a17c48c2099 update implementation details
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 357
diff changeset
   103
- storage over obsolete marker
9a17c48c2099 update implementation details
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 357
diff changeset
   104
- exchange suing pushkey
9a17c48c2099 update implementation details
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 357
diff changeset
   105
- extinct changeset are properly hidden
9a17c48c2099 update implementation details
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 357
diff changeset
   106
- extinct changeset are excluded from exchange