made the pack completely portable and wrote relevent bat files to go with it

This commit is contained in:
Draqoken
2025-04-09 17:04:56 +03:00
parent 5e77d7e9cf
commit 5e4144c3c0
7417 changed files with 2181044 additions and 19 deletions

View File

@@ -0,0 +1,103 @@
Error reporting in git
======================
`BUG`, `bug`, `die`, `usage`, `error`, and `warning` report errors of
various kinds.
- `BUG` is for failed internal assertions that should never happen,
i.e. a bug in git itself.
- `bug` (lower-case, not `BUG`) is supposed to be used like `BUG` but
prints a "BUG" message instead of calling `abort()`.
+
A call to `bug()` will then result in a "real" call to the `BUG()`
function, either explicitly by invoking `BUG_if_bug()` after call(s)
to `bug()`, or implicitly at `exit()` time where we'll check if we
encountered any outstanding `bug()` invocations.
+
If there were no prior calls to `bug()` before invoking `BUG_if_bug()`
the latter is a NOOP. The `BUG_if_bug()` function takes the same
arguments as `BUG()` itself. Calling `BUG_if_bug()` explicitly isn't
necessary, but ensures that we die as soon as possible.
+
If you know you had prior calls to `bug()` then calling `BUG()` itself
is equivalent to calling `BUG_if_bug()`, the latter being a wrapper
calling `BUG()` if we've set a flag indicating that we've called
`bug()`.
+
This is for the convenience of APIs who'd like to potentially report
more than one "bug", such as the optbug() validation in
parse-options.c.
- `die` is for fatal application errors. It prints a message to
the user and exits with status 128.
- `usage` is for errors in command line usage. After printing its
message, it exits with status 129. (See also `usage_with_options`
in the link:api-parse-options.html[parse-options API].)
- `error` is for non-fatal library errors. It prints a message
to the user and returns -1 for convenience in signaling the error
to the caller.
- `warning` is for reporting situations that probably should not
occur but which the user (and Git) can continue to work around
without running into too many problems. Like `error`, it
returns -1 after reporting the situation to the caller.
These reports will be logged via the trace2 facility. See the "error"
event in link:api-trace2.html[trace2 API].
Customizable error handlers
---------------------------
The default behavior of `die` and `error` is to write a message to
stderr and then exit or return as appropriate. This behavior can be
overridden using `set_die_routine` and `set_error_routine`. For
example, "git daemon" uses set_die_routine to write the reason `die`
was called to syslog before exiting.
Library errors
--------------
Functions return a negative integer on error. Details beyond that
vary from function to function:
- Some functions return -1 for all errors. Others return a more
specific value depending on how the caller might want to react
to the error.
- Some functions report the error to stderr with `error`,
while others leave that for the caller to do.
- errno is not meaningful on return from most functions (except
for thin wrappers for system calls).
Check the function's API documentation to be sure.
Caller-handled errors
---------------------
An increasing number of functions take a parameter 'struct strbuf *err'.
On error, such functions append a message about what went wrong to the
'err' strbuf. The message is meant to be complete enough to be passed
to `die` or `error` as-is. For example:
if (ref_transaction_commit(transaction, &err))
die("%s", err.buf);
The 'err' parameter will be untouched if no error occurred, so multiple
function calls can be chained:
t = ref_transaction_begin(&err);
if (!t ||
ref_transaction_update(t, "HEAD", ..., &err) ||
ret_transaction_commit(t, &err))
die("%s", err.buf);
The 'err' parameter must be a pointer to a valid strbuf. To silence
a message, pass a strbuf that is explicitly ignored:
if (thing_that_can_fail_in_an_ignorable_way(..., &err))
/* This failure is okay. */
strbuf_reset(&err);

View File

@@ -0,0 +1,598 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="en">
<head>
<meta charset="UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=edge"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<meta name="generator" content="Asciidoctor 2.0.23"/>
<title>Error reporting in git</title>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"/>
<style>
/*! Asciidoctor default stylesheet | MIT License | https://asciidoctor.org */
/* Uncomment the following line when using as a custom stylesheet */
/* @import "https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"; */
html{font-family:sans-serif;-webkit-text-size-adjust:100%}
a{background:none}
a:focus{outline:thin dotted}
a:active,a:hover{outline:0}
h1{font-size:2em;margin:.67em 0}
b,strong{font-weight:bold}
abbr{font-size:.9em}
abbr[title]{cursor:help;border-bottom:1px dotted #dddddf;text-decoration:none}
dfn{font-style:italic}
hr{height:0}
mark{background:#ff0;color:#000}
code,kbd,pre,samp{font-family:monospace;font-size:1em}
pre{white-space:pre-wrap}
q{quotes:"\201C" "\201D" "\2018" "\2019"}
small{font-size:80%}
sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline}
sup{top:-.5em}
sub{bottom:-.25em}
img{border:0}
svg:not(:root){overflow:hidden}
figure{margin:0}
audio,video{display:inline-block}
audio:not([controls]){display:none;height:0}
fieldset{border:1px solid silver;margin:0 2px;padding:.35em .625em .75em}
legend{border:0;padding:0}
button,input,select,textarea{font-family:inherit;font-size:100%;margin:0}
button,input{line-height:normal}
button,select{text-transform:none}
button,html input[type=button],input[type=reset],input[type=submit]{-webkit-appearance:button;cursor:pointer}
button[disabled],html input[disabled]{cursor:default}
input[type=checkbox],input[type=radio]{padding:0}
button::-moz-focus-inner,input::-moz-focus-inner{border:0;padding:0}
textarea{overflow:auto;vertical-align:top}
table{border-collapse:collapse;border-spacing:0}
*,::before,::after{box-sizing:border-box}
html,body{font-size:100%}
body{background:#fff;color:rgba(0,0,0,.8);padding:0;margin:0;font-family:"Noto Serif","DejaVu Serif",serif;line-height:1;position:relative;cursor:auto;-moz-tab-size:4;-o-tab-size:4;tab-size:4;word-wrap:anywhere;-moz-osx-font-smoothing:grayscale;-webkit-font-smoothing:antialiased}
a:hover{cursor:pointer}
img,object,embed{max-width:100%;height:auto}
object,embed{height:100%}
img{-ms-interpolation-mode:bicubic}
.left{float:left!important}
.right{float:right!important}
.text-left{text-align:left!important}
.text-right{text-align:right!important}
.text-center{text-align:center!important}
.text-justify{text-align:justify!important}
.hide{display:none}
img,object,svg{display:inline-block;vertical-align:middle}
textarea{height:auto;min-height:50px}
select{width:100%}
.subheader,.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{line-height:1.45;color:#7a2518;font-weight:400;margin-top:0;margin-bottom:.25em}
div,dl,dt,dd,ul,ol,li,h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6,pre,form,p,blockquote,th,td{margin:0;padding:0}
a{color:#2156a5;text-decoration:underline;line-height:inherit}
a:hover,a:focus{color:#1d4b8f}
a img{border:0}
p{line-height:1.6;margin-bottom:1.25em;text-rendering:optimizeLegibility}
p aside{font-size:.875em;line-height:1.35;font-style:italic}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{font-family:"Open Sans","DejaVu Sans",sans-serif;font-weight:300;font-style:normal;color:#ba3925;text-rendering:optimizeLegibility;margin-top:1em;margin-bottom:.5em;line-height:1.0125em}
h1 small,h2 small,h3 small,#toctitle small,.sidebarblock>.content>.title small,h4 small,h5 small,h6 small{font-size:60%;color:#e99b8f;line-height:0}
h1{font-size:2.125em}
h2{font-size:1.6875em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.375em}
h4,h5{font-size:1.125em}
h6{font-size:1em}
hr{border:solid #dddddf;border-width:1px 0 0;clear:both;margin:1.25em 0 1.1875em}
em,i{font-style:italic;line-height:inherit}
strong,b{font-weight:bold;line-height:inherit}
small{font-size:60%;line-height:inherit}
code{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:400;color:rgba(0,0,0,.9)}
ul,ol,dl{line-height:1.6;margin-bottom:1.25em;list-style-position:outside;font-family:inherit}
ul,ol{margin-left:1.5em}
ul li ul,ul li ol{margin-left:1.25em;margin-bottom:0}
ul.circle{list-style-type:circle}
ul.disc{list-style-type:disc}
ul.square{list-style-type:square}
ul.circle ul:not([class]),ul.disc ul:not([class]),ul.square ul:not([class]){list-style:inherit}
ol li ul,ol li ol{margin-left:1.25em;margin-bottom:0}
dl dt{margin-bottom:.3125em;font-weight:bold}
dl dd{margin-bottom:1.25em}
blockquote{margin:0 0 1.25em;padding:.5625em 1.25em 0 1.1875em;border-left:1px solid #ddd}
blockquote,blockquote p{line-height:1.6;color:rgba(0,0,0,.85)}
@media screen and (min-width:768px){h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2}
h1{font-size:2.75em}
h2{font-size:2.3125em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.6875em}
h4{font-size:1.4375em}}
table{background:#fff;margin-bottom:1.25em;border:1px solid #dedede;word-wrap:normal}
table thead,table tfoot{background:#f7f8f7}
table thead tr th,table thead tr td,table tfoot tr th,table tfoot tr td{padding:.5em .625em .625em;font-size:inherit;color:rgba(0,0,0,.8);text-align:left}
table tr th,table tr td{padding:.5625em .625em;font-size:inherit;color:rgba(0,0,0,.8)}
table tr.even,table tr.alt{background:#f8f8f7}
table thead tr th,table tfoot tr th,table tbody tr td,table tr td,table tfoot tr td{line-height:1.6}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2;word-spacing:-.05em}
h1 strong,h2 strong,h3 strong,#toctitle strong,.sidebarblock>.content>.title strong,h4 strong,h5 strong,h6 strong{font-weight:400}
.center{margin-left:auto;margin-right:auto}
.stretch{width:100%}
.clearfix::before,.clearfix::after,.float-group::before,.float-group::after{content:" ";display:table}
.clearfix::after,.float-group::after{clear:both}
:not(pre).nobreak{word-wrap:normal}
:not(pre).nowrap{white-space:nowrap}
:not(pre).pre-wrap{white-space:pre-wrap}
:not(pre):not([class^=L])>code{font-size:.9375em;font-style:normal!important;letter-spacing:0;padding:.1em .5ex;word-spacing:-.15em;background:#f7f7f8;border-radius:4px;line-height:1.45;text-rendering:optimizeSpeed}
pre{color:rgba(0,0,0,.9);font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;line-height:1.45;text-rendering:optimizeSpeed}
pre code,pre pre{color:inherit;font-size:inherit;line-height:inherit}
pre>code{display:block}
pre.nowrap,pre.nowrap pre{white-space:pre;word-wrap:normal}
em em{font-style:normal}
strong strong{font-weight:400}
.keyseq{color:rgba(51,51,51,.8)}
kbd{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;display:inline-block;color:rgba(0,0,0,.8);font-size:.65em;line-height:1.45;background:#f7f7f7;border:1px solid #ccc;border-radius:3px;box-shadow:0 1px 0 rgba(0,0,0,.2),inset 0 0 0 .1em #fff;margin:0 .15em;padding:.2em .5em;vertical-align:middle;position:relative;top:-.1em;white-space:nowrap}
.keyseq kbd:first-child{margin-left:0}
.keyseq kbd:last-child{margin-right:0}
.menuseq,.menuref{color:#000}
.menuseq b:not(.caret),.menuref{font-weight:inherit}
.menuseq{word-spacing:-.02em}
.menuseq b.caret{font-size:1.25em;line-height:.8}
.menuseq i.caret{font-weight:bold;text-align:center;width:.45em}
b.button::before,b.button::after{position:relative;top:-1px;font-weight:400}
b.button::before{content:"[";padding:0 3px 0 2px}
b.button::after{content:"]";padding:0 2px 0 3px}
p a>code:hover{color:rgba(0,0,0,.9)}
#header,#content,#footnotes,#footer{width:100%;margin:0 auto;max-width:62.5em;*zoom:1;position:relative;padding-left:.9375em;padding-right:.9375em}
#header::before,#header::after,#content::before,#content::after,#footnotes::before,#footnotes::after,#footer::before,#footer::after{content:" ";display:table}
#header::after,#content::after,#footnotes::after,#footer::after{clear:both}
#content{margin-top:1.25em}
#content::before{content:none}
#header>h1:first-child{color:rgba(0,0,0,.85);margin-top:2.25rem;margin-bottom:0}
#header>h1:first-child+#toc{margin-top:8px;border-top:1px solid #dddddf}
#header>h1:only-child{border-bottom:1px solid #dddddf;padding-bottom:8px}
#header .details{border-bottom:1px solid #dddddf;line-height:1.45;padding-top:.25em;padding-bottom:.25em;padding-left:.25em;color:rgba(0,0,0,.6);display:flex;flex-flow:row wrap}
#header .details span:first-child{margin-left:-.125em}
#header .details span.email a{color:rgba(0,0,0,.85)}
#header .details br{display:none}
#header .details br+span::before{content:"\00a0\2013\00a0"}
#header .details br+span.author::before{content:"\00a0\22c5\00a0";color:rgba(0,0,0,.85)}
#header .details br+span#revremark::before{content:"\00a0|\00a0"}
#header #revnumber{text-transform:capitalize}
#header #revnumber::after{content:"\00a0"}
#content>h1:first-child:not([class]){color:rgba(0,0,0,.85);border-bottom:1px solid #dddddf;padding-bottom:8px;margin-top:0;padding-top:1rem;margin-bottom:1.25rem}
#toc{border-bottom:1px solid #e7e7e9;padding-bottom:.5em}
#toc>ul{margin-left:.125em}
#toc ul.sectlevel0>li>a{font-style:italic}
#toc ul.sectlevel0 ul.sectlevel1{margin:.5em 0}
#toc ul{font-family:"Open Sans","DejaVu Sans",sans-serif;list-style-type:none}
#toc li{line-height:1.3334;margin-top:.3334em}
#toc a{text-decoration:none}
#toc a:active{text-decoration:underline}
#toctitle{color:#7a2518;font-size:1.2em}
@media screen and (min-width:768px){#toctitle{font-size:1.375em}
body.toc2{padding-left:15em;padding-right:0}
body.toc2 #header>h1:nth-last-child(2){border-bottom:1px solid #dddddf;padding-bottom:8px}
#toc.toc2{margin-top:0!important;background:#f8f8f7;position:fixed;width:15em;left:0;top:0;border-right:1px solid #e7e7e9;border-top-width:0!important;border-bottom-width:0!important;z-index:1000;padding:1.25em 1em;height:100%;overflow:auto}
#toc.toc2 #toctitle{margin-top:0;margin-bottom:.8rem;font-size:1.2em}
#toc.toc2>ul{font-size:.9em;margin-bottom:0}
#toc.toc2 ul ul{margin-left:0;padding-left:1em}
#toc.toc2 ul.sectlevel0 ul.sectlevel1{padding-left:0;margin-top:.5em;margin-bottom:.5em}
body.toc2.toc-right{padding-left:0;padding-right:15em}
body.toc2.toc-right #toc.toc2{border-right-width:0;border-left:1px solid #e7e7e9;left:auto;right:0}}
@media screen and (min-width:1280px){body.toc2{padding-left:20em;padding-right:0}
#toc.toc2{width:20em}
#toc.toc2 #toctitle{font-size:1.375em}
#toc.toc2>ul{font-size:.95em}
#toc.toc2 ul ul{padding-left:1.25em}
body.toc2.toc-right{padding-left:0;padding-right:20em}}
#content #toc{border:1px solid #e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;border-radius:4px}
#content #toc>:first-child{margin-top:0}
#content #toc>:last-child{margin-bottom:0}
#footer{max-width:none;background:rgba(0,0,0,.8);padding:1.25em}
#footer-text{color:hsla(0,0%,100%,.8);line-height:1.44}
#content{margin-bottom:.625em}
.sect1{padding-bottom:.625em}
@media screen and (min-width:768px){#content{margin-bottom:1.25em}
.sect1{padding-bottom:1.25em}}
.sect1:last-child{padding-bottom:0}
.sect1+.sect1{border-top:1px solid #e7e7e9}
#content h1>a.anchor,h2>a.anchor,h3>a.anchor,#toctitle>a.anchor,.sidebarblock>.content>.title>a.anchor,h4>a.anchor,h5>a.anchor,h6>a.anchor{position:absolute;z-index:1001;width:1.5ex;margin-left:-1.5ex;display:block;text-decoration:none!important;visibility:hidden;text-align:center;font-weight:400}
#content h1>a.anchor::before,h2>a.anchor::before,h3>a.anchor::before,#toctitle>a.anchor::before,.sidebarblock>.content>.title>a.anchor::before,h4>a.anchor::before,h5>a.anchor::before,h6>a.anchor::before{content:"\00A7";font-size:.85em;display:block;padding-top:.1em}
#content h1:hover>a.anchor,#content h1>a.anchor:hover,h2:hover>a.anchor,h2>a.anchor:hover,h3:hover>a.anchor,#toctitle:hover>a.anchor,.sidebarblock>.content>.title:hover>a.anchor,h3>a.anchor:hover,#toctitle>a.anchor:hover,.sidebarblock>.content>.title>a.anchor:hover,h4:hover>a.anchor,h4>a.anchor:hover,h5:hover>a.anchor,h5>a.anchor:hover,h6:hover>a.anchor,h6>a.anchor:hover{visibility:visible}
#content h1>a.link,h2>a.link,h3>a.link,#toctitle>a.link,.sidebarblock>.content>.title>a.link,h4>a.link,h5>a.link,h6>a.link{color:#ba3925;text-decoration:none}
#content h1>a.link:hover,h2>a.link:hover,h3>a.link:hover,#toctitle>a.link:hover,.sidebarblock>.content>.title>a.link:hover,h4>a.link:hover,h5>a.link:hover,h6>a.link:hover{color:#a53221}
details,.audioblock,.imageblock,.literalblock,.listingblock,.stemblock,.videoblock{margin-bottom:1.25em}
details{margin-left:1.25rem}
details>summary{cursor:pointer;display:block;position:relative;line-height:1.6;margin-bottom:.625rem;outline:none;-webkit-tap-highlight-color:transparent}
details>summary::-webkit-details-marker{display:none}
details>summary::before{content:"";border:solid transparent;border-left:solid;border-width:.3em 0 .3em .5em;position:absolute;top:.5em;left:-1.25rem;transform:translateX(15%)}
details[open]>summary::before{border:solid transparent;border-top:solid;border-width:.5em .3em 0;transform:translateY(15%)}
details>summary::after{content:"";width:1.25rem;height:1em;position:absolute;top:.3em;left:-1.25rem}
.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{text-rendering:optimizeLegibility;text-align:left;font-family:"Noto Serif","DejaVu Serif",serif;font-size:1rem;font-style:italic}
table.tableblock.fit-content>caption.title{white-space:nowrap;width:0}
.paragraph.lead>p,#preamble>.sectionbody>[class=paragraph]:first-of-type p{font-size:1.21875em;line-height:1.6;color:rgba(0,0,0,.85)}
.admonitionblock>table{border-collapse:separate;border:0;background:none;width:100%}
.admonitionblock>table td.icon{text-align:center;width:80px}
.admonitionblock>table td.icon img{max-width:none}
.admonitionblock>table td.icon .title{font-weight:bold;font-family:"Open Sans","DejaVu Sans",sans-serif;text-transform:uppercase}
.admonitionblock>table td.content{padding-left:1.125em;padding-right:1.25em;border-left:1px solid #dddddf;color:rgba(0,0,0,.6);word-wrap:anywhere}
.admonitionblock>table td.content>:last-child>:last-child{margin-bottom:0}
.exampleblock>.content{border:1px solid #e6e6e6;margin-bottom:1.25em;padding:1.25em;background:#fff;border-radius:4px}
.sidebarblock{border:1px solid #dbdbd6;margin-bottom:1.25em;padding:1.25em;background:#f3f3f2;border-radius:4px}
.sidebarblock>.content>.title{color:#7a2518;margin-top:0;text-align:center}
.exampleblock>.content>:first-child,.sidebarblock>.content>:first-child{margin-top:0}
.exampleblock>.content>:last-child,.exampleblock>.content>:last-child>:last-child,.exampleblock>.content .olist>ol>li:last-child>:last-child,.exampleblock>.content .ulist>ul>li:last-child>:last-child,.exampleblock>.content .qlist>ol>li:last-child>:last-child,.sidebarblock>.content>:last-child,.sidebarblock>.content>:last-child>:last-child,.sidebarblock>.content .olist>ol>li:last-child>:last-child,.sidebarblock>.content .ulist>ul>li:last-child>:last-child,.sidebarblock>.content .qlist>ol>li:last-child>:last-child{margin-bottom:0}
.literalblock pre,.listingblock>.content>pre{border-radius:4px;overflow-x:auto;padding:1em;font-size:.8125em}
@media screen and (min-width:768px){.literalblock pre,.listingblock>.content>pre{font-size:.90625em}}
@media screen and (min-width:1280px){.literalblock pre,.listingblock>.content>pre{font-size:1em}}
.literalblock pre,.listingblock>.content>pre:not(.highlight),.listingblock>.content>pre[class=highlight],.listingblock>.content>pre[class^="highlight "]{background:#f7f7f8}
.literalblock.output pre{color:#f7f7f8;background:rgba(0,0,0,.9)}
.listingblock>.content{position:relative}
.listingblock code[data-lang]::before{display:none;content:attr(data-lang);position:absolute;font-size:.75em;top:.425rem;right:.5rem;line-height:1;text-transform:uppercase;color:inherit;opacity:.5}
.listingblock:hover code[data-lang]::before{display:block}
.listingblock.terminal pre .command::before{content:attr(data-prompt);padding-right:.5em;color:inherit;opacity:.5}
.listingblock.terminal pre .command:not([data-prompt])::before{content:"$"}
.listingblock pre.highlightjs{padding:0}
.listingblock pre.highlightjs>code{padding:1em;border-radius:4px}
.listingblock pre.prettyprint{border-width:0}
.prettyprint{background:#f7f7f8}
pre.prettyprint .linenums{line-height:1.45;margin-left:2em}
pre.prettyprint li{background:none;list-style-type:inherit;padding-left:0}
pre.prettyprint li code[data-lang]::before{opacity:1}
pre.prettyprint li:not(:first-child) code[data-lang]::before{display:none}
table.linenotable{border-collapse:separate;border:0;margin-bottom:0;background:none}
table.linenotable td[class]{color:inherit;vertical-align:top;padding:0;line-height:inherit;white-space:normal}
table.linenotable td.code{padding-left:.75em}
table.linenotable td.linenos,pre.pygments .linenos{border-right:1px solid;opacity:.35;padding-right:.5em;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}
pre.pygments span.linenos{display:inline-block;margin-right:.75em}
.quoteblock{margin:0 1em 1.25em 1.5em;display:table}
.quoteblock:not(.excerpt)>.title{margin-left:-1.5em;margin-bottom:.75em}
.quoteblock blockquote,.quoteblock p{color:rgba(0,0,0,.85);font-size:1.15rem;line-height:1.75;word-spacing:.1em;letter-spacing:0;font-style:italic;text-align:justify}
.quoteblock blockquote{margin:0;padding:0;border:0}
.quoteblock blockquote::before{content:"\201c";float:left;font-size:2.75em;font-weight:bold;line-height:.6em;margin-left:-.6em;color:#7a2518;text-shadow:0 1px 2px rgba(0,0,0,.1)}
.quoteblock blockquote>.paragraph:last-child p{margin-bottom:0}
.quoteblock .attribution{margin-top:.75em;margin-right:.5ex;text-align:right}
.verseblock{margin:0 1em 1.25em}
.verseblock pre{font-family:"Open Sans","DejaVu Sans",sans-serif;font-size:1.15rem;color:rgba(0,0,0,.85);font-weight:300;text-rendering:optimizeLegibility}
.verseblock pre strong{font-weight:400}
.verseblock .attribution{margin-top:1.25rem;margin-left:.5ex}
.quoteblock .attribution,.verseblock .attribution{font-size:.9375em;line-height:1.45;font-style:italic}
.quoteblock .attribution br,.verseblock .attribution br{display:none}
.quoteblock .attribution cite,.verseblock .attribution cite{display:block;letter-spacing:-.025em;color:rgba(0,0,0,.6)}
.quoteblock.abstract blockquote::before,.quoteblock.excerpt blockquote::before,.quoteblock .quoteblock blockquote::before{display:none}
.quoteblock.abstract blockquote,.quoteblock.abstract p,.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{line-height:1.6;word-spacing:0}
.quoteblock.abstract{margin:0 1em 1.25em;display:block}
.quoteblock.abstract>.title{margin:0 0 .375em;font-size:1.15em;text-align:center}
.quoteblock.excerpt>blockquote,.quoteblock .quoteblock{padding:0 0 .25em 1em;border-left:.25em solid #dddddf}
.quoteblock.excerpt,.quoteblock .quoteblock{margin-left:0}
.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{color:inherit;font-size:1.0625rem}
.quoteblock.excerpt .attribution,.quoteblock .quoteblock .attribution{color:inherit;font-size:.85rem;text-align:left;margin-right:0}
p.tableblock:last-child{margin-bottom:0}
td.tableblock>.content{margin-bottom:1.25em;word-wrap:anywhere}
td.tableblock>.content>:last-child{margin-bottom:-1.25em}
table.tableblock,th.tableblock,td.tableblock{border:0 solid #dedede}
table.grid-all>*>tr>*{border-width:1px}
table.grid-cols>*>tr>*{border-width:0 1px}
table.grid-rows>*>tr>*{border-width:1px 0}
table.frame-all{border-width:1px}
table.frame-ends{border-width:1px 0}
table.frame-sides{border-width:0 1px}
table.frame-none>colgroup+*>:first-child>*,table.frame-sides>colgroup+*>:first-child>*{border-top-width:0}
table.frame-none>:last-child>:last-child>*,table.frame-sides>:last-child>:last-child>*{border-bottom-width:0}
table.frame-none>*>tr>:first-child,table.frame-ends>*>tr>:first-child{border-left-width:0}
table.frame-none>*>tr>:last-child,table.frame-ends>*>tr>:last-child{border-right-width:0}
table.stripes-all>*>tr,table.stripes-odd>*>tr:nth-of-type(odd),table.stripes-even>*>tr:nth-of-type(even),table.stripes-hover>*>tr:hover{background:#f8f8f7}
th.halign-left,td.halign-left{text-align:left}
th.halign-right,td.halign-right{text-align:right}
th.halign-center,td.halign-center{text-align:center}
th.valign-top,td.valign-top{vertical-align:top}
th.valign-bottom,td.valign-bottom{vertical-align:bottom}
th.valign-middle,td.valign-middle{vertical-align:middle}
table thead th,table tfoot th{font-weight:bold}
tbody tr th{background:#f7f8f7}
tbody tr th,tbody tr th p,tfoot tr th,tfoot tr th p{color:rgba(0,0,0,.8);font-weight:bold}
p.tableblock>code:only-child{background:none;padding:0}
p.tableblock{font-size:1em}
ol{margin-left:1.75em}
ul li ol{margin-left:1.5em}
dl dd{margin-left:1.125em}
dl dd:last-child,dl dd:last-child>:last-child{margin-bottom:0}
li p,ul dd,ol dd,.olist .olist,.ulist .ulist,.ulist .olist,.olist .ulist{margin-bottom:.625em}
ul.checklist,ul.none,ol.none,ul.no-bullet,ol.no-bullet,ol.unnumbered,ul.unstyled,ol.unstyled{list-style-type:none}
ul.no-bullet,ol.no-bullet,ol.unnumbered{margin-left:.625em}
ul.unstyled,ol.unstyled{margin-left:0}
li>p:empty:only-child::before{content:"";display:inline-block}
ul.checklist>li>p:first-child{margin-left:-1em}
ul.checklist>li>p:first-child>.fa-square-o:first-child,ul.checklist>li>p:first-child>.fa-check-square-o:first-child{width:1.25em;font-size:.8em;position:relative;bottom:.125em}
ul.checklist>li>p:first-child>input[type=checkbox]:first-child{margin-right:.25em}
ul.inline{display:flex;flex-flow:row wrap;list-style:none;margin:0 0 .625em -1.25em}
ul.inline>li{margin-left:1.25em}
.unstyled dl dt{font-weight:400;font-style:normal}
ol.arabic{list-style-type:decimal}
ol.decimal{list-style-type:decimal-leading-zero}
ol.loweralpha{list-style-type:lower-alpha}
ol.upperalpha{list-style-type:upper-alpha}
ol.lowerroman{list-style-type:lower-roman}
ol.upperroman{list-style-type:upper-roman}
ol.lowergreek{list-style-type:lower-greek}
.hdlist>table,.colist>table{border:0;background:none}
.hdlist>table>tbody>tr,.colist>table>tbody>tr{background:none}
td.hdlist1,td.hdlist2{vertical-align:top;padding:0 .625em}
td.hdlist1{font-weight:bold;padding-bottom:1.25em}
td.hdlist2{word-wrap:anywhere}
.literalblock+.colist,.listingblock+.colist{margin-top:-.5em}
.colist td:not([class]):first-child{padding:.4em .75em 0;line-height:1;vertical-align:top}
.colist td:not([class]):first-child img{max-width:none}
.colist td:not([class]):last-child{padding:.25em 0}
.thumb,.th{line-height:0;display:inline-block;border:4px solid #fff;box-shadow:0 0 0 1px #ddd}
.imageblock.left{margin:.25em .625em 1.25em 0}
.imageblock.right{margin:.25em 0 1.25em .625em}
.imageblock>.title{margin-bottom:0}
.imageblock.thumb,.imageblock.th{border-width:6px}
.imageblock.thumb>.title,.imageblock.th>.title{padding:0 .125em}
.image.left,.image.right{margin-top:.25em;margin-bottom:.25em;display:inline-block;line-height:0}
.image.left{margin-right:.625em}
.image.right{margin-left:.625em}
a.image{text-decoration:none;display:inline-block}
a.image object{pointer-events:none}
sup.footnote,sup.footnoteref{font-size:.875em;position:static;vertical-align:super}
sup.footnote a,sup.footnoteref a{text-decoration:none}
sup.footnote a:active,sup.footnoteref a:active,#footnotes .footnote a:first-of-type:active{text-decoration:underline}
#footnotes{padding-top:.75em;padding-bottom:.75em;margin-bottom:.625em}
#footnotes hr{width:20%;min-width:6.25em;margin:-.25em 0 .75em;border-width:1px 0 0}
#footnotes .footnote{padding:0 .375em 0 .225em;line-height:1.3334;font-size:.875em;margin-left:1.2em;margin-bottom:.2em}
#footnotes .footnote a:first-of-type{font-weight:bold;text-decoration:none;margin-left:-1.05em}
#footnotes .footnote:last-of-type{margin-bottom:0}
#content #footnotes{margin-top:-.625em;margin-bottom:0;padding:.75em 0}
div.unbreakable{page-break-inside:avoid}
.big{font-size:larger}
.small{font-size:smaller}
.underline{text-decoration:underline}
.overline{text-decoration:overline}
.line-through{text-decoration:line-through}
.aqua{color:#00bfbf}
.aqua-background{background:#00fafa}
.black{color:#000}
.black-background{background:#000}
.blue{color:#0000bf}
.blue-background{background:#0000fa}
.fuchsia{color:#bf00bf}
.fuchsia-background{background:#fa00fa}
.gray{color:#606060}
.gray-background{background:#7d7d7d}
.green{color:#006000}
.green-background{background:#007d00}
.lime{color:#00bf00}
.lime-background{background:#00fa00}
.maroon{color:#600000}
.maroon-background{background:#7d0000}
.navy{color:#000060}
.navy-background{background:#00007d}
.olive{color:#606000}
.olive-background{background:#7d7d00}
.purple{color:#600060}
.purple-background{background:#7d007d}
.red{color:#bf0000}
.red-background{background:#fa0000}
.silver{color:#909090}
.silver-background{background:#bcbcbc}
.teal{color:#006060}
.teal-background{background:#007d7d}
.white{color:#bfbfbf}
.white-background{background:#fafafa}
.yellow{color:#bfbf00}
.yellow-background{background:#fafa00}
span.icon>.fa{cursor:default}
a span.icon>.fa{cursor:inherit}
.admonitionblock td.icon [class^="fa icon-"]{font-size:2.5em;text-shadow:1px 1px 2px rgba(0,0,0,.5);cursor:default}
.admonitionblock td.icon .icon-note::before{content:"\f05a";color:#19407c}
.admonitionblock td.icon .icon-tip::before{content:"\f0eb";text-shadow:1px 1px 2px rgba(155,155,0,.8);color:#111}
.admonitionblock td.icon .icon-warning::before{content:"\f071";color:#bf6900}
.admonitionblock td.icon .icon-caution::before{content:"\f06d";color:#bf3400}
.admonitionblock td.icon .icon-important::before{content:"\f06a";color:#bf0000}
.conum[data-value]{display:inline-block;color:#fff!important;background:rgba(0,0,0,.8);border-radius:50%;text-align:center;font-size:.75em;width:1.67em;height:1.67em;line-height:1.67em;font-family:"Open Sans","DejaVu Sans",sans-serif;font-style:normal;font-weight:bold}
.conum[data-value] *{color:#fff!important}
.conum[data-value]+b{display:none}
.conum[data-value]::after{content:attr(data-value)}
pre .conum[data-value]{position:relative;top:-.125em}
b.conum *{color:inherit!important}
.conum:not([data-value]):empty{display:none}
dt,th.tableblock,td.content,div.footnote{text-rendering:optimizeLegibility}
h1,h2,p,td.content,span.alt,summary{letter-spacing:-.01em}
p strong,td.content strong,div.footnote strong{letter-spacing:-.005em}
p,blockquote,dt,td.content,td.hdlist1,span.alt,summary{font-size:1.0625rem}
p{margin-bottom:1.25rem}
.sidebarblock p,.sidebarblock dt,.sidebarblock td.content,p.tableblock{font-size:1em}
.exampleblock>.content{background:#fffef7;border-color:#e0e0dc;box-shadow:0 1px 4px #e0e0dc}
.print-only{display:none!important}
@page{margin:1.25cm .75cm}
@media print{*{box-shadow:none!important;text-shadow:none!important}
html{font-size:80%}
a{color:inherit!important;text-decoration:underline!important}
a.bare,a[href^="#"],a[href^="mailto:"]{text-decoration:none!important}
a[href^="http:"]:not(.bare)::after,a[href^="https:"]:not(.bare)::after{content:"(" attr(href) ")";display:inline-block;font-size:.875em;padding-left:.25em}
abbr[title]{border-bottom:1px dotted}
abbr[title]::after{content:" (" attr(title) ")"}
pre,blockquote,tr,img,object,svg{page-break-inside:avoid}
thead{display:table-header-group}
svg{max-width:100%}
p,blockquote,dt,td.content{font-size:1em;orphans:3;widows:3}
h2,h3,#toctitle,.sidebarblock>.content>.title{page-break-after:avoid}
#header,#content,#footnotes,#footer{max-width:none}
#toc,.sidebarblock,.exampleblock>.content{background:none!important}
#toc{border-bottom:1px solid #dddddf!important;padding-bottom:0!important}
body.book #header{text-align:center}
body.book #header>h1:first-child{border:0!important;margin:2.5em 0 1em}
body.book #header .details{border:0!important;display:block;padding:0!important}
body.book #header .details span:first-child{margin-left:0!important}
body.book #header .details br{display:block}
body.book #header .details br+span::before{content:none!important}
body.book #toc{border:0!important;text-align:left!important;padding:0!important;margin:0!important}
body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-break-before:always}
.listingblock code[data-lang]::before{display:block}
#footer{padding:0 .9375em}
.hide-on-print{display:none!important}
.print-only{display:block!important}
.hide-for-print{display:none!important}
.show-for-print{display:inherit!important}}
@media amzn-kf8,print{#header>h1:first-child{margin-top:1.25rem}
.sect1{padding:0!important}
.sect1+.sect1{border:0}
#footer{background:none}
#footer-text{color:rgba(0,0,0,.6);font-size:.9em}}
@media amzn-kf8{#header,#content,#footnotes,#footer{padding:0}}
</style>
</head>
<body class="article">
<div id="header">
<h1>Error reporting in git</h1>
</div>
<div id="content">
<div id="preamble">
<div class="sectionbody">
<div class="paragraph">
<p><code>BUG</code>, <code>bug</code>, <code>die</code>, <code>usage</code>, <code>error</code>, and <code>warning</code> report errors of
various kinds.</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>BUG</code> is for failed internal assertions that should never happen,
i.e. a bug in git itself.</p>
</li>
<li>
<p><code>bug</code> (lower-case, not <code>BUG</code>) is supposed to be used like <code>BUG</code> but
prints a "BUG" message instead of calling <code>abort</code>().</p>
<div class="paragraph">
<p>A call to <code>bug</code>() will then result in a "real" call to the <code>BUG</code>()
function, either explicitly by invoking <code>BUG_if_bug</code>() after call(s)
to <code>bug</code>(), or implicitly at <code>exit</code>() time where we&#8217;ll check if we
encountered any outstanding <code>bug</code>() invocations.</p>
</div>
<div class="paragraph">
<p>If there were no prior calls to <code>bug</code>() before invoking <code>BUG_if_bug</code>()
the latter is a NOOP. The <code>BUG_if_bug</code>() function takes the same
arguments as <code>BUG</code>() itself. Calling <code>BUG_if_bug</code>() explicitly isn&#8217;t
necessary, but ensures that we die as soon as possible.</p>
</div>
<div class="paragraph">
<p>If you know you had prior calls to <code>bug</code>() then calling <code>BUG</code>() itself
is equivalent to calling <code>BUG_if_bug</code>(), the latter being a wrapper
calling <code>BUG</code>() if we&#8217;ve set a flag indicating that we&#8217;ve called
<code>bug</code>().</p>
</div>
<div class="paragraph">
<p>This is for the convenience of APIs who&#8217;d like to potentially report
more than one "bug", such as the optbug() validation in
parse-options.c.</p>
</div>
</li>
<li>
<p><code>die</code> is for fatal application errors. It prints a message to
the user and exits with status 128.</p>
</li>
<li>
<p><code>usage</code> is for errors in command line usage. After printing its
message, it exits with status 129. (See also <code>usage_with_options</code>
in the <a href="api-parse-options.html">parse-options API</a>.)</p>
</li>
<li>
<p><code>error</code> is for non-fatal library errors. It prints a message
to the user and returns -1 for convenience in signaling the error
to the caller.</p>
</li>
<li>
<p><code>warning</code> is for reporting situations that probably should not
occur but which the user (and Git) can continue to work around
without running into too many problems. Like <code>error</code>, it
returns -1 after reporting the situation to the caller.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>These reports will be logged via the trace2 facility. See the "error"
event in <a href="api-trace2.html">trace2 API</a>.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_customizable_error_handlers">Customizable error handlers</h2>
<div class="sectionbody">
<div class="paragraph">
<p>The default behavior of <code>die</code> and <code>error</code> is to write a message to
stderr and then exit or return as appropriate. This behavior can be
overridden using <code>set_die_routine</code> and <code>set_error_routine</code>. For
example, "git daemon" uses set_die_routine to write the reason <code>die</code>
was called to syslog before exiting.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_library_errors">Library errors</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Functions return a negative integer on error. Details beyond that
vary from function to function:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Some functions return -1 for all errors. Others return a more
specific value depending on how the caller might want to react
to the error.</p>
</li>
<li>
<p>Some functions report the error to stderr with <code>error</code>,
while others leave that for the caller to do.</p>
</li>
<li>
<p>errno is not meaningful on return from most functions (except
for thin wrappers for system calls).</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Check the function&#8217;s API documentation to be sure.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_caller_handled_errors">Caller-handled errors</h2>
<div class="sectionbody">
<div class="paragraph">
<p>An increasing number of functions take a parameter <em>struct strbuf *err</em>.
On error, such functions append a message about what went wrong to the
<em>err</em> strbuf. The message is meant to be complete enough to be passed
to <code>die</code> or <code>error</code> as-is. For example:</p>
</div>
<div class="literalblock">
<div class="content">
<pre>if (ref_transaction_commit(transaction, &amp;err))
die("%s", err.buf);</pre>
</div>
</div>
<div class="paragraph">
<p>The <em>err</em> parameter will be untouched if no error occurred, so multiple
function calls can be chained:</p>
</div>
<div class="literalblock">
<div class="content">
<pre>t = ref_transaction_begin(&amp;err);
if (!t ||
ref_transaction_update(t, "HEAD", ..., &amp;err) ||
ret_transaction_commit(t, &amp;err))
die("%s", err.buf);</pre>
</div>
</div>
<div class="paragraph">
<p>The <em>err</em> parameter must be a pointer to a valid strbuf. To silence
a message, pass a strbuf that is explicitly ignored:</p>
</div>
<div class="literalblock">
<div class="content">
<pre>if (thing_that_can_fail_in_an_ignorable_way(..., &amp;err))
/* This failure is okay. */
strbuf_reset(&amp;err);</pre>
</div>
</div>
</div>
</div>
</div>
<div id="footer">
<div id="footer-text">
Last updated 2025-03-17 07:09:38 UTC
</div>
</div>
</body>
</html>

View File

@@ -0,0 +1,13 @@
Git API Documents
=================
Git has grown a set of internal APIs over time. This collection
documents them.
////////////////////////////////////////////////////////////////
// table of contents begin
////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////
// table of contents end
////////////////////////////////////////////////////////////////

View File

@@ -0,0 +1,18 @@
Git API Documents
=================
Git has grown a set of internal APIs over time. This collection
documents them.
////////////////////////////////////////////////////////////////
// table of contents begin
////////////////////////////////////////////////////////////////
* link:api-error-handling.html[Error reporting in git]
* link:api-merge.html[merge API]
* link:api-parse-options.html[parse-options API]
* link:api-path-walk.html[Path-Walk API]
* link:api-simple-ipc.html[Simple-IPC API]
* link:api-trace2.html[= Trace2 API]
////////////////////////////////////////////////////////////////
// table of contents end
////////////////////////////////////////////////////////////////

View File

@@ -0,0 +1,475 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="en">
<head>
<meta charset="UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=edge"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<meta name="generator" content="Asciidoctor 2.0.23"/>
<title>Git API Documents</title>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"/>
<style>
/*! Asciidoctor default stylesheet | MIT License | https://asciidoctor.org */
/* Uncomment the following line when using as a custom stylesheet */
/* @import "https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"; */
html{font-family:sans-serif;-webkit-text-size-adjust:100%}
a{background:none}
a:focus{outline:thin dotted}
a:active,a:hover{outline:0}
h1{font-size:2em;margin:.67em 0}
b,strong{font-weight:bold}
abbr{font-size:.9em}
abbr[title]{cursor:help;border-bottom:1px dotted #dddddf;text-decoration:none}
dfn{font-style:italic}
hr{height:0}
mark{background:#ff0;color:#000}
code,kbd,pre,samp{font-family:monospace;font-size:1em}
pre{white-space:pre-wrap}
q{quotes:"\201C" "\201D" "\2018" "\2019"}
small{font-size:80%}
sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline}
sup{top:-.5em}
sub{bottom:-.25em}
img{border:0}
svg:not(:root){overflow:hidden}
figure{margin:0}
audio,video{display:inline-block}
audio:not([controls]){display:none;height:0}
fieldset{border:1px solid silver;margin:0 2px;padding:.35em .625em .75em}
legend{border:0;padding:0}
button,input,select,textarea{font-family:inherit;font-size:100%;margin:0}
button,input{line-height:normal}
button,select{text-transform:none}
button,html input[type=button],input[type=reset],input[type=submit]{-webkit-appearance:button;cursor:pointer}
button[disabled],html input[disabled]{cursor:default}
input[type=checkbox],input[type=radio]{padding:0}
button::-moz-focus-inner,input::-moz-focus-inner{border:0;padding:0}
textarea{overflow:auto;vertical-align:top}
table{border-collapse:collapse;border-spacing:0}
*,::before,::after{box-sizing:border-box}
html,body{font-size:100%}
body{background:#fff;color:rgba(0,0,0,.8);padding:0;margin:0;font-family:"Noto Serif","DejaVu Serif",serif;line-height:1;position:relative;cursor:auto;-moz-tab-size:4;-o-tab-size:4;tab-size:4;word-wrap:anywhere;-moz-osx-font-smoothing:grayscale;-webkit-font-smoothing:antialiased}
a:hover{cursor:pointer}
img,object,embed{max-width:100%;height:auto}
object,embed{height:100%}
img{-ms-interpolation-mode:bicubic}
.left{float:left!important}
.right{float:right!important}
.text-left{text-align:left!important}
.text-right{text-align:right!important}
.text-center{text-align:center!important}
.text-justify{text-align:justify!important}
.hide{display:none}
img,object,svg{display:inline-block;vertical-align:middle}
textarea{height:auto;min-height:50px}
select{width:100%}
.subheader,.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{line-height:1.45;color:#7a2518;font-weight:400;margin-top:0;margin-bottom:.25em}
div,dl,dt,dd,ul,ol,li,h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6,pre,form,p,blockquote,th,td{margin:0;padding:0}
a{color:#2156a5;text-decoration:underline;line-height:inherit}
a:hover,a:focus{color:#1d4b8f}
a img{border:0}
p{line-height:1.6;margin-bottom:1.25em;text-rendering:optimizeLegibility}
p aside{font-size:.875em;line-height:1.35;font-style:italic}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{font-family:"Open Sans","DejaVu Sans",sans-serif;font-weight:300;font-style:normal;color:#ba3925;text-rendering:optimizeLegibility;margin-top:1em;margin-bottom:.5em;line-height:1.0125em}
h1 small,h2 small,h3 small,#toctitle small,.sidebarblock>.content>.title small,h4 small,h5 small,h6 small{font-size:60%;color:#e99b8f;line-height:0}
h1{font-size:2.125em}
h2{font-size:1.6875em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.375em}
h4,h5{font-size:1.125em}
h6{font-size:1em}
hr{border:solid #dddddf;border-width:1px 0 0;clear:both;margin:1.25em 0 1.1875em}
em,i{font-style:italic;line-height:inherit}
strong,b{font-weight:bold;line-height:inherit}
small{font-size:60%;line-height:inherit}
code{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:400;color:rgba(0,0,0,.9)}
ul,ol,dl{line-height:1.6;margin-bottom:1.25em;list-style-position:outside;font-family:inherit}
ul,ol{margin-left:1.5em}
ul li ul,ul li ol{margin-left:1.25em;margin-bottom:0}
ul.circle{list-style-type:circle}
ul.disc{list-style-type:disc}
ul.square{list-style-type:square}
ul.circle ul:not([class]),ul.disc ul:not([class]),ul.square ul:not([class]){list-style:inherit}
ol li ul,ol li ol{margin-left:1.25em;margin-bottom:0}
dl dt{margin-bottom:.3125em;font-weight:bold}
dl dd{margin-bottom:1.25em}
blockquote{margin:0 0 1.25em;padding:.5625em 1.25em 0 1.1875em;border-left:1px solid #ddd}
blockquote,blockquote p{line-height:1.6;color:rgba(0,0,0,.85)}
@media screen and (min-width:768px){h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2}
h1{font-size:2.75em}
h2{font-size:2.3125em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.6875em}
h4{font-size:1.4375em}}
table{background:#fff;margin-bottom:1.25em;border:1px solid #dedede;word-wrap:normal}
table thead,table tfoot{background:#f7f8f7}
table thead tr th,table thead tr td,table tfoot tr th,table tfoot tr td{padding:.5em .625em .625em;font-size:inherit;color:rgba(0,0,0,.8);text-align:left}
table tr th,table tr td{padding:.5625em .625em;font-size:inherit;color:rgba(0,0,0,.8)}
table tr.even,table tr.alt{background:#f8f8f7}
table thead tr th,table tfoot tr th,table tbody tr td,table tr td,table tfoot tr td{line-height:1.6}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2;word-spacing:-.05em}
h1 strong,h2 strong,h3 strong,#toctitle strong,.sidebarblock>.content>.title strong,h4 strong,h5 strong,h6 strong{font-weight:400}
.center{margin-left:auto;margin-right:auto}
.stretch{width:100%}
.clearfix::before,.clearfix::after,.float-group::before,.float-group::after{content:" ";display:table}
.clearfix::after,.float-group::after{clear:both}
:not(pre).nobreak{word-wrap:normal}
:not(pre).nowrap{white-space:nowrap}
:not(pre).pre-wrap{white-space:pre-wrap}
:not(pre):not([class^=L])>code{font-size:.9375em;font-style:normal!important;letter-spacing:0;padding:.1em .5ex;word-spacing:-.15em;background:#f7f7f8;border-radius:4px;line-height:1.45;text-rendering:optimizeSpeed}
pre{color:rgba(0,0,0,.9);font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;line-height:1.45;text-rendering:optimizeSpeed}
pre code,pre pre{color:inherit;font-size:inherit;line-height:inherit}
pre>code{display:block}
pre.nowrap,pre.nowrap pre{white-space:pre;word-wrap:normal}
em em{font-style:normal}
strong strong{font-weight:400}
.keyseq{color:rgba(51,51,51,.8)}
kbd{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;display:inline-block;color:rgba(0,0,0,.8);font-size:.65em;line-height:1.45;background:#f7f7f7;border:1px solid #ccc;border-radius:3px;box-shadow:0 1px 0 rgba(0,0,0,.2),inset 0 0 0 .1em #fff;margin:0 .15em;padding:.2em .5em;vertical-align:middle;position:relative;top:-.1em;white-space:nowrap}
.keyseq kbd:first-child{margin-left:0}
.keyseq kbd:last-child{margin-right:0}
.menuseq,.menuref{color:#000}
.menuseq b:not(.caret),.menuref{font-weight:inherit}
.menuseq{word-spacing:-.02em}
.menuseq b.caret{font-size:1.25em;line-height:.8}
.menuseq i.caret{font-weight:bold;text-align:center;width:.45em}
b.button::before,b.button::after{position:relative;top:-1px;font-weight:400}
b.button::before{content:"[";padding:0 3px 0 2px}
b.button::after{content:"]";padding:0 2px 0 3px}
p a>code:hover{color:rgba(0,0,0,.9)}
#header,#content,#footnotes,#footer{width:100%;margin:0 auto;max-width:62.5em;*zoom:1;position:relative;padding-left:.9375em;padding-right:.9375em}
#header::before,#header::after,#content::before,#content::after,#footnotes::before,#footnotes::after,#footer::before,#footer::after{content:" ";display:table}
#header::after,#content::after,#footnotes::after,#footer::after{clear:both}
#content{margin-top:1.25em}
#content::before{content:none}
#header>h1:first-child{color:rgba(0,0,0,.85);margin-top:2.25rem;margin-bottom:0}
#header>h1:first-child+#toc{margin-top:8px;border-top:1px solid #dddddf}
#header>h1:only-child{border-bottom:1px solid #dddddf;padding-bottom:8px}
#header .details{border-bottom:1px solid #dddddf;line-height:1.45;padding-top:.25em;padding-bottom:.25em;padding-left:.25em;color:rgba(0,0,0,.6);display:flex;flex-flow:row wrap}
#header .details span:first-child{margin-left:-.125em}
#header .details span.email a{color:rgba(0,0,0,.85)}
#header .details br{display:none}
#header .details br+span::before{content:"\00a0\2013\00a0"}
#header .details br+span.author::before{content:"\00a0\22c5\00a0";color:rgba(0,0,0,.85)}
#header .details br+span#revremark::before{content:"\00a0|\00a0"}
#header #revnumber{text-transform:capitalize}
#header #revnumber::after{content:"\00a0"}
#content>h1:first-child:not([class]){color:rgba(0,0,0,.85);border-bottom:1px solid #dddddf;padding-bottom:8px;margin-top:0;padding-top:1rem;margin-bottom:1.25rem}
#toc{border-bottom:1px solid #e7e7e9;padding-bottom:.5em}
#toc>ul{margin-left:.125em}
#toc ul.sectlevel0>li>a{font-style:italic}
#toc ul.sectlevel0 ul.sectlevel1{margin:.5em 0}
#toc ul{font-family:"Open Sans","DejaVu Sans",sans-serif;list-style-type:none}
#toc li{line-height:1.3334;margin-top:.3334em}
#toc a{text-decoration:none}
#toc a:active{text-decoration:underline}
#toctitle{color:#7a2518;font-size:1.2em}
@media screen and (min-width:768px){#toctitle{font-size:1.375em}
body.toc2{padding-left:15em;padding-right:0}
body.toc2 #header>h1:nth-last-child(2){border-bottom:1px solid #dddddf;padding-bottom:8px}
#toc.toc2{margin-top:0!important;background:#f8f8f7;position:fixed;width:15em;left:0;top:0;border-right:1px solid #e7e7e9;border-top-width:0!important;border-bottom-width:0!important;z-index:1000;padding:1.25em 1em;height:100%;overflow:auto}
#toc.toc2 #toctitle{margin-top:0;margin-bottom:.8rem;font-size:1.2em}
#toc.toc2>ul{font-size:.9em;margin-bottom:0}
#toc.toc2 ul ul{margin-left:0;padding-left:1em}
#toc.toc2 ul.sectlevel0 ul.sectlevel1{padding-left:0;margin-top:.5em;margin-bottom:.5em}
body.toc2.toc-right{padding-left:0;padding-right:15em}
body.toc2.toc-right #toc.toc2{border-right-width:0;border-left:1px solid #e7e7e9;left:auto;right:0}}
@media screen and (min-width:1280px){body.toc2{padding-left:20em;padding-right:0}
#toc.toc2{width:20em}
#toc.toc2 #toctitle{font-size:1.375em}
#toc.toc2>ul{font-size:.95em}
#toc.toc2 ul ul{padding-left:1.25em}
body.toc2.toc-right{padding-left:0;padding-right:20em}}
#content #toc{border:1px solid #e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;border-radius:4px}
#content #toc>:first-child{margin-top:0}
#content #toc>:last-child{margin-bottom:0}
#footer{max-width:none;background:rgba(0,0,0,.8);padding:1.25em}
#footer-text{color:hsla(0,0%,100%,.8);line-height:1.44}
#content{margin-bottom:.625em}
.sect1{padding-bottom:.625em}
@media screen and (min-width:768px){#content{margin-bottom:1.25em}
.sect1{padding-bottom:1.25em}}
.sect1:last-child{padding-bottom:0}
.sect1+.sect1{border-top:1px solid #e7e7e9}
#content h1>a.anchor,h2>a.anchor,h3>a.anchor,#toctitle>a.anchor,.sidebarblock>.content>.title>a.anchor,h4>a.anchor,h5>a.anchor,h6>a.anchor{position:absolute;z-index:1001;width:1.5ex;margin-left:-1.5ex;display:block;text-decoration:none!important;visibility:hidden;text-align:center;font-weight:400}
#content h1>a.anchor::before,h2>a.anchor::before,h3>a.anchor::before,#toctitle>a.anchor::before,.sidebarblock>.content>.title>a.anchor::before,h4>a.anchor::before,h5>a.anchor::before,h6>a.anchor::before{content:"\00A7";font-size:.85em;display:block;padding-top:.1em}
#content h1:hover>a.anchor,#content h1>a.anchor:hover,h2:hover>a.anchor,h2>a.anchor:hover,h3:hover>a.anchor,#toctitle:hover>a.anchor,.sidebarblock>.content>.title:hover>a.anchor,h3>a.anchor:hover,#toctitle>a.anchor:hover,.sidebarblock>.content>.title>a.anchor:hover,h4:hover>a.anchor,h4>a.anchor:hover,h5:hover>a.anchor,h5>a.anchor:hover,h6:hover>a.anchor,h6>a.anchor:hover{visibility:visible}
#content h1>a.link,h2>a.link,h3>a.link,#toctitle>a.link,.sidebarblock>.content>.title>a.link,h4>a.link,h5>a.link,h6>a.link{color:#ba3925;text-decoration:none}
#content h1>a.link:hover,h2>a.link:hover,h3>a.link:hover,#toctitle>a.link:hover,.sidebarblock>.content>.title>a.link:hover,h4>a.link:hover,h5>a.link:hover,h6>a.link:hover{color:#a53221}
details,.audioblock,.imageblock,.literalblock,.listingblock,.stemblock,.videoblock{margin-bottom:1.25em}
details{margin-left:1.25rem}
details>summary{cursor:pointer;display:block;position:relative;line-height:1.6;margin-bottom:.625rem;outline:none;-webkit-tap-highlight-color:transparent}
details>summary::-webkit-details-marker{display:none}
details>summary::before{content:"";border:solid transparent;border-left:solid;border-width:.3em 0 .3em .5em;position:absolute;top:.5em;left:-1.25rem;transform:translateX(15%)}
details[open]>summary::before{border:solid transparent;border-top:solid;border-width:.5em .3em 0;transform:translateY(15%)}
details>summary::after{content:"";width:1.25rem;height:1em;position:absolute;top:.3em;left:-1.25rem}
.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{text-rendering:optimizeLegibility;text-align:left;font-family:"Noto Serif","DejaVu Serif",serif;font-size:1rem;font-style:italic}
table.tableblock.fit-content>caption.title{white-space:nowrap;width:0}
.paragraph.lead>p,#preamble>.sectionbody>[class=paragraph]:first-of-type p{font-size:1.21875em;line-height:1.6;color:rgba(0,0,0,.85)}
.admonitionblock>table{border-collapse:separate;border:0;background:none;width:100%}
.admonitionblock>table td.icon{text-align:center;width:80px}
.admonitionblock>table td.icon img{max-width:none}
.admonitionblock>table td.icon .title{font-weight:bold;font-family:"Open Sans","DejaVu Sans",sans-serif;text-transform:uppercase}
.admonitionblock>table td.content{padding-left:1.125em;padding-right:1.25em;border-left:1px solid #dddddf;color:rgba(0,0,0,.6);word-wrap:anywhere}
.admonitionblock>table td.content>:last-child>:last-child{margin-bottom:0}
.exampleblock>.content{border:1px solid #e6e6e6;margin-bottom:1.25em;padding:1.25em;background:#fff;border-radius:4px}
.sidebarblock{border:1px solid #dbdbd6;margin-bottom:1.25em;padding:1.25em;background:#f3f3f2;border-radius:4px}
.sidebarblock>.content>.title{color:#7a2518;margin-top:0;text-align:center}
.exampleblock>.content>:first-child,.sidebarblock>.content>:first-child{margin-top:0}
.exampleblock>.content>:last-child,.exampleblock>.content>:last-child>:last-child,.exampleblock>.content .olist>ol>li:last-child>:last-child,.exampleblock>.content .ulist>ul>li:last-child>:last-child,.exampleblock>.content .qlist>ol>li:last-child>:last-child,.sidebarblock>.content>:last-child,.sidebarblock>.content>:last-child>:last-child,.sidebarblock>.content .olist>ol>li:last-child>:last-child,.sidebarblock>.content .ulist>ul>li:last-child>:last-child,.sidebarblock>.content .qlist>ol>li:last-child>:last-child{margin-bottom:0}
.literalblock pre,.listingblock>.content>pre{border-radius:4px;overflow-x:auto;padding:1em;font-size:.8125em}
@media screen and (min-width:768px){.literalblock pre,.listingblock>.content>pre{font-size:.90625em}}
@media screen and (min-width:1280px){.literalblock pre,.listingblock>.content>pre{font-size:1em}}
.literalblock pre,.listingblock>.content>pre:not(.highlight),.listingblock>.content>pre[class=highlight],.listingblock>.content>pre[class^="highlight "]{background:#f7f7f8}
.literalblock.output pre{color:#f7f7f8;background:rgba(0,0,0,.9)}
.listingblock>.content{position:relative}
.listingblock code[data-lang]::before{display:none;content:attr(data-lang);position:absolute;font-size:.75em;top:.425rem;right:.5rem;line-height:1;text-transform:uppercase;color:inherit;opacity:.5}
.listingblock:hover code[data-lang]::before{display:block}
.listingblock.terminal pre .command::before{content:attr(data-prompt);padding-right:.5em;color:inherit;opacity:.5}
.listingblock.terminal pre .command:not([data-prompt])::before{content:"$"}
.listingblock pre.highlightjs{padding:0}
.listingblock pre.highlightjs>code{padding:1em;border-radius:4px}
.listingblock pre.prettyprint{border-width:0}
.prettyprint{background:#f7f7f8}
pre.prettyprint .linenums{line-height:1.45;margin-left:2em}
pre.prettyprint li{background:none;list-style-type:inherit;padding-left:0}
pre.prettyprint li code[data-lang]::before{opacity:1}
pre.prettyprint li:not(:first-child) code[data-lang]::before{display:none}
table.linenotable{border-collapse:separate;border:0;margin-bottom:0;background:none}
table.linenotable td[class]{color:inherit;vertical-align:top;padding:0;line-height:inherit;white-space:normal}
table.linenotable td.code{padding-left:.75em}
table.linenotable td.linenos,pre.pygments .linenos{border-right:1px solid;opacity:.35;padding-right:.5em;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}
pre.pygments span.linenos{display:inline-block;margin-right:.75em}
.quoteblock{margin:0 1em 1.25em 1.5em;display:table}
.quoteblock:not(.excerpt)>.title{margin-left:-1.5em;margin-bottom:.75em}
.quoteblock blockquote,.quoteblock p{color:rgba(0,0,0,.85);font-size:1.15rem;line-height:1.75;word-spacing:.1em;letter-spacing:0;font-style:italic;text-align:justify}
.quoteblock blockquote{margin:0;padding:0;border:0}
.quoteblock blockquote::before{content:"\201c";float:left;font-size:2.75em;font-weight:bold;line-height:.6em;margin-left:-.6em;color:#7a2518;text-shadow:0 1px 2px rgba(0,0,0,.1)}
.quoteblock blockquote>.paragraph:last-child p{margin-bottom:0}
.quoteblock .attribution{margin-top:.75em;margin-right:.5ex;text-align:right}
.verseblock{margin:0 1em 1.25em}
.verseblock pre{font-family:"Open Sans","DejaVu Sans",sans-serif;font-size:1.15rem;color:rgba(0,0,0,.85);font-weight:300;text-rendering:optimizeLegibility}
.verseblock pre strong{font-weight:400}
.verseblock .attribution{margin-top:1.25rem;margin-left:.5ex}
.quoteblock .attribution,.verseblock .attribution{font-size:.9375em;line-height:1.45;font-style:italic}
.quoteblock .attribution br,.verseblock .attribution br{display:none}
.quoteblock .attribution cite,.verseblock .attribution cite{display:block;letter-spacing:-.025em;color:rgba(0,0,0,.6)}
.quoteblock.abstract blockquote::before,.quoteblock.excerpt blockquote::before,.quoteblock .quoteblock blockquote::before{display:none}
.quoteblock.abstract blockquote,.quoteblock.abstract p,.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{line-height:1.6;word-spacing:0}
.quoteblock.abstract{margin:0 1em 1.25em;display:block}
.quoteblock.abstract>.title{margin:0 0 .375em;font-size:1.15em;text-align:center}
.quoteblock.excerpt>blockquote,.quoteblock .quoteblock{padding:0 0 .25em 1em;border-left:.25em solid #dddddf}
.quoteblock.excerpt,.quoteblock .quoteblock{margin-left:0}
.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{color:inherit;font-size:1.0625rem}
.quoteblock.excerpt .attribution,.quoteblock .quoteblock .attribution{color:inherit;font-size:.85rem;text-align:left;margin-right:0}
p.tableblock:last-child{margin-bottom:0}
td.tableblock>.content{margin-bottom:1.25em;word-wrap:anywhere}
td.tableblock>.content>:last-child{margin-bottom:-1.25em}
table.tableblock,th.tableblock,td.tableblock{border:0 solid #dedede}
table.grid-all>*>tr>*{border-width:1px}
table.grid-cols>*>tr>*{border-width:0 1px}
table.grid-rows>*>tr>*{border-width:1px 0}
table.frame-all{border-width:1px}
table.frame-ends{border-width:1px 0}
table.frame-sides{border-width:0 1px}
table.frame-none>colgroup+*>:first-child>*,table.frame-sides>colgroup+*>:first-child>*{border-top-width:0}
table.frame-none>:last-child>:last-child>*,table.frame-sides>:last-child>:last-child>*{border-bottom-width:0}
table.frame-none>*>tr>:first-child,table.frame-ends>*>tr>:first-child{border-left-width:0}
table.frame-none>*>tr>:last-child,table.frame-ends>*>tr>:last-child{border-right-width:0}
table.stripes-all>*>tr,table.stripes-odd>*>tr:nth-of-type(odd),table.stripes-even>*>tr:nth-of-type(even),table.stripes-hover>*>tr:hover{background:#f8f8f7}
th.halign-left,td.halign-left{text-align:left}
th.halign-right,td.halign-right{text-align:right}
th.halign-center,td.halign-center{text-align:center}
th.valign-top,td.valign-top{vertical-align:top}
th.valign-bottom,td.valign-bottom{vertical-align:bottom}
th.valign-middle,td.valign-middle{vertical-align:middle}
table thead th,table tfoot th{font-weight:bold}
tbody tr th{background:#f7f8f7}
tbody tr th,tbody tr th p,tfoot tr th,tfoot tr th p{color:rgba(0,0,0,.8);font-weight:bold}
p.tableblock>code:only-child{background:none;padding:0}
p.tableblock{font-size:1em}
ol{margin-left:1.75em}
ul li ol{margin-left:1.5em}
dl dd{margin-left:1.125em}
dl dd:last-child,dl dd:last-child>:last-child{margin-bottom:0}
li p,ul dd,ol dd,.olist .olist,.ulist .ulist,.ulist .olist,.olist .ulist{margin-bottom:.625em}
ul.checklist,ul.none,ol.none,ul.no-bullet,ol.no-bullet,ol.unnumbered,ul.unstyled,ol.unstyled{list-style-type:none}
ul.no-bullet,ol.no-bullet,ol.unnumbered{margin-left:.625em}
ul.unstyled,ol.unstyled{margin-left:0}
li>p:empty:only-child::before{content:"";display:inline-block}
ul.checklist>li>p:first-child{margin-left:-1em}
ul.checklist>li>p:first-child>.fa-square-o:first-child,ul.checklist>li>p:first-child>.fa-check-square-o:first-child{width:1.25em;font-size:.8em;position:relative;bottom:.125em}
ul.checklist>li>p:first-child>input[type=checkbox]:first-child{margin-right:.25em}
ul.inline{display:flex;flex-flow:row wrap;list-style:none;margin:0 0 .625em -1.25em}
ul.inline>li{margin-left:1.25em}
.unstyled dl dt{font-weight:400;font-style:normal}
ol.arabic{list-style-type:decimal}
ol.decimal{list-style-type:decimal-leading-zero}
ol.loweralpha{list-style-type:lower-alpha}
ol.upperalpha{list-style-type:upper-alpha}
ol.lowerroman{list-style-type:lower-roman}
ol.upperroman{list-style-type:upper-roman}
ol.lowergreek{list-style-type:lower-greek}
.hdlist>table,.colist>table{border:0;background:none}
.hdlist>table>tbody>tr,.colist>table>tbody>tr{background:none}
td.hdlist1,td.hdlist2{vertical-align:top;padding:0 .625em}
td.hdlist1{font-weight:bold;padding-bottom:1.25em}
td.hdlist2{word-wrap:anywhere}
.literalblock+.colist,.listingblock+.colist{margin-top:-.5em}
.colist td:not([class]):first-child{padding:.4em .75em 0;line-height:1;vertical-align:top}
.colist td:not([class]):first-child img{max-width:none}
.colist td:not([class]):last-child{padding:.25em 0}
.thumb,.th{line-height:0;display:inline-block;border:4px solid #fff;box-shadow:0 0 0 1px #ddd}
.imageblock.left{margin:.25em .625em 1.25em 0}
.imageblock.right{margin:.25em 0 1.25em .625em}
.imageblock>.title{margin-bottom:0}
.imageblock.thumb,.imageblock.th{border-width:6px}
.imageblock.thumb>.title,.imageblock.th>.title{padding:0 .125em}
.image.left,.image.right{margin-top:.25em;margin-bottom:.25em;display:inline-block;line-height:0}
.image.left{margin-right:.625em}
.image.right{margin-left:.625em}
a.image{text-decoration:none;display:inline-block}
a.image object{pointer-events:none}
sup.footnote,sup.footnoteref{font-size:.875em;position:static;vertical-align:super}
sup.footnote a,sup.footnoteref a{text-decoration:none}
sup.footnote a:active,sup.footnoteref a:active,#footnotes .footnote a:first-of-type:active{text-decoration:underline}
#footnotes{padding-top:.75em;padding-bottom:.75em;margin-bottom:.625em}
#footnotes hr{width:20%;min-width:6.25em;margin:-.25em 0 .75em;border-width:1px 0 0}
#footnotes .footnote{padding:0 .375em 0 .225em;line-height:1.3334;font-size:.875em;margin-left:1.2em;margin-bottom:.2em}
#footnotes .footnote a:first-of-type{font-weight:bold;text-decoration:none;margin-left:-1.05em}
#footnotes .footnote:last-of-type{margin-bottom:0}
#content #footnotes{margin-top:-.625em;margin-bottom:0;padding:.75em 0}
div.unbreakable{page-break-inside:avoid}
.big{font-size:larger}
.small{font-size:smaller}
.underline{text-decoration:underline}
.overline{text-decoration:overline}
.line-through{text-decoration:line-through}
.aqua{color:#00bfbf}
.aqua-background{background:#00fafa}
.black{color:#000}
.black-background{background:#000}
.blue{color:#0000bf}
.blue-background{background:#0000fa}
.fuchsia{color:#bf00bf}
.fuchsia-background{background:#fa00fa}
.gray{color:#606060}
.gray-background{background:#7d7d7d}
.green{color:#006000}
.green-background{background:#007d00}
.lime{color:#00bf00}
.lime-background{background:#00fa00}
.maroon{color:#600000}
.maroon-background{background:#7d0000}
.navy{color:#000060}
.navy-background{background:#00007d}
.olive{color:#606000}
.olive-background{background:#7d7d00}
.purple{color:#600060}
.purple-background{background:#7d007d}
.red{color:#bf0000}
.red-background{background:#fa0000}
.silver{color:#909090}
.silver-background{background:#bcbcbc}
.teal{color:#006060}
.teal-background{background:#007d7d}
.white{color:#bfbfbf}
.white-background{background:#fafafa}
.yellow{color:#bfbf00}
.yellow-background{background:#fafa00}
span.icon>.fa{cursor:default}
a span.icon>.fa{cursor:inherit}
.admonitionblock td.icon [class^="fa icon-"]{font-size:2.5em;text-shadow:1px 1px 2px rgba(0,0,0,.5);cursor:default}
.admonitionblock td.icon .icon-note::before{content:"\f05a";color:#19407c}
.admonitionblock td.icon .icon-tip::before{content:"\f0eb";text-shadow:1px 1px 2px rgba(155,155,0,.8);color:#111}
.admonitionblock td.icon .icon-warning::before{content:"\f071";color:#bf6900}
.admonitionblock td.icon .icon-caution::before{content:"\f06d";color:#bf3400}
.admonitionblock td.icon .icon-important::before{content:"\f06a";color:#bf0000}
.conum[data-value]{display:inline-block;color:#fff!important;background:rgba(0,0,0,.8);border-radius:50%;text-align:center;font-size:.75em;width:1.67em;height:1.67em;line-height:1.67em;font-family:"Open Sans","DejaVu Sans",sans-serif;font-style:normal;font-weight:bold}
.conum[data-value] *{color:#fff!important}
.conum[data-value]+b{display:none}
.conum[data-value]::after{content:attr(data-value)}
pre .conum[data-value]{position:relative;top:-.125em}
b.conum *{color:inherit!important}
.conum:not([data-value]):empty{display:none}
dt,th.tableblock,td.content,div.footnote{text-rendering:optimizeLegibility}
h1,h2,p,td.content,span.alt,summary{letter-spacing:-.01em}
p strong,td.content strong,div.footnote strong{letter-spacing:-.005em}
p,blockquote,dt,td.content,td.hdlist1,span.alt,summary{font-size:1.0625rem}
p{margin-bottom:1.25rem}
.sidebarblock p,.sidebarblock dt,.sidebarblock td.content,p.tableblock{font-size:1em}
.exampleblock>.content{background:#fffef7;border-color:#e0e0dc;box-shadow:0 1px 4px #e0e0dc}
.print-only{display:none!important}
@page{margin:1.25cm .75cm}
@media print{*{box-shadow:none!important;text-shadow:none!important}
html{font-size:80%}
a{color:inherit!important;text-decoration:underline!important}
a.bare,a[href^="#"],a[href^="mailto:"]{text-decoration:none!important}
a[href^="http:"]:not(.bare)::after,a[href^="https:"]:not(.bare)::after{content:"(" attr(href) ")";display:inline-block;font-size:.875em;padding-left:.25em}
abbr[title]{border-bottom:1px dotted}
abbr[title]::after{content:" (" attr(title) ")"}
pre,blockquote,tr,img,object,svg{page-break-inside:avoid}
thead{display:table-header-group}
svg{max-width:100%}
p,blockquote,dt,td.content{font-size:1em;orphans:3;widows:3}
h2,h3,#toctitle,.sidebarblock>.content>.title{page-break-after:avoid}
#header,#content,#footnotes,#footer{max-width:none}
#toc,.sidebarblock,.exampleblock>.content{background:none!important}
#toc{border-bottom:1px solid #dddddf!important;padding-bottom:0!important}
body.book #header{text-align:center}
body.book #header>h1:first-child{border:0!important;margin:2.5em 0 1em}
body.book #header .details{border:0!important;display:block;padding:0!important}
body.book #header .details span:first-child{margin-left:0!important}
body.book #header .details br{display:block}
body.book #header .details br+span::before{content:none!important}
body.book #toc{border:0!important;text-align:left!important;padding:0!important;margin:0!important}
body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-break-before:always}
.listingblock code[data-lang]::before{display:block}
#footer{padding:0 .9375em}
.hide-on-print{display:none!important}
.print-only{display:block!important}
.hide-for-print{display:none!important}
.show-for-print{display:inherit!important}}
@media amzn-kf8,print{#header>h1:first-child{margin-top:1.25rem}
.sect1{padding:0!important}
.sect1+.sect1{border:0}
#footer{background:none}
#footer-text{color:rgba(0,0,0,.6);font-size:.9em}}
@media amzn-kf8{#header,#content,#footnotes,#footer{padding:0}}
</style>
</head>
<body class="article">
<div id="header">
<h1>Git API Documents</h1>
</div>
<div id="content">
<div class="paragraph">
<p>Git has grown a set of internal APIs over time. This collection
documents them.</p>
</div>
<div class="ulist">
<ul>
<li>
<p><a href="api-error-handling.html">Error reporting in git</a></p>
</li>
<li>
<p><a href="api-merge.html">merge API</a></p>
</li>
<li>
<p><a href="api-parse-options.html">parse-options API</a></p>
</li>
<li>
<p><a href="api-path-walk.html">Path-Walk API</a></p>
</li>
<li>
<p><a href="api-simple-ipc.html">Simple-IPC API</a></p>
</li>
<li>
<p><a href="api-trace2.html">= Trace2 API</a></p>
</li>
</ul>
</div>
</div>
<div id="footer">
<div id="footer-text">
Last updated 2025-03-17 07:09:38 UTC
</div>
</div>
</body>
</html>

View File

@@ -0,0 +1,36 @@
merge API
=========
The merge API helps a program to reconcile two competing sets of
improvements to some files (e.g., unregistered changes from the work
tree versus changes involved in switching to a new branch), reporting
conflicts if found. The library called through this API is
responsible for a few things.
* determining which trees to merge (recursive ancestor consolidation);
* lining up corresponding files in the trees to be merged (rename
detection, subtree shifting), reporting edge cases like add/add
and rename/rename conflicts to the user;
* performing a three-way merge of corresponding files, taking
path-specific merge drivers (specified in `.gitattributes`)
into account.
Data structures
---------------
* `mmbuffer_t`, `mmfile_t`
These store data usable for use by the xdiff backend, for writing and
for reading, respectively. See `xdiff/xdiff.h` for the definitions
and `diff.c` for examples.
* `struct ll_merge_options`
Check merge-ll.h for details.
Low-level (single file) merge
-----------------------------
Check merge-ll.h for details.

View File

@@ -0,0 +1,512 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="en">
<head>
<meta charset="UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=edge"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<meta name="generator" content="Asciidoctor 2.0.23"/>
<title>merge API</title>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"/>
<style>
/*! Asciidoctor default stylesheet | MIT License | https://asciidoctor.org */
/* Uncomment the following line when using as a custom stylesheet */
/* @import "https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"; */
html{font-family:sans-serif;-webkit-text-size-adjust:100%}
a{background:none}
a:focus{outline:thin dotted}
a:active,a:hover{outline:0}
h1{font-size:2em;margin:.67em 0}
b,strong{font-weight:bold}
abbr{font-size:.9em}
abbr[title]{cursor:help;border-bottom:1px dotted #dddddf;text-decoration:none}
dfn{font-style:italic}
hr{height:0}
mark{background:#ff0;color:#000}
code,kbd,pre,samp{font-family:monospace;font-size:1em}
pre{white-space:pre-wrap}
q{quotes:"\201C" "\201D" "\2018" "\2019"}
small{font-size:80%}
sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline}
sup{top:-.5em}
sub{bottom:-.25em}
img{border:0}
svg:not(:root){overflow:hidden}
figure{margin:0}
audio,video{display:inline-block}
audio:not([controls]){display:none;height:0}
fieldset{border:1px solid silver;margin:0 2px;padding:.35em .625em .75em}
legend{border:0;padding:0}
button,input,select,textarea{font-family:inherit;font-size:100%;margin:0}
button,input{line-height:normal}
button,select{text-transform:none}
button,html input[type=button],input[type=reset],input[type=submit]{-webkit-appearance:button;cursor:pointer}
button[disabled],html input[disabled]{cursor:default}
input[type=checkbox],input[type=radio]{padding:0}
button::-moz-focus-inner,input::-moz-focus-inner{border:0;padding:0}
textarea{overflow:auto;vertical-align:top}
table{border-collapse:collapse;border-spacing:0}
*,::before,::after{box-sizing:border-box}
html,body{font-size:100%}
body{background:#fff;color:rgba(0,0,0,.8);padding:0;margin:0;font-family:"Noto Serif","DejaVu Serif",serif;line-height:1;position:relative;cursor:auto;-moz-tab-size:4;-o-tab-size:4;tab-size:4;word-wrap:anywhere;-moz-osx-font-smoothing:grayscale;-webkit-font-smoothing:antialiased}
a:hover{cursor:pointer}
img,object,embed{max-width:100%;height:auto}
object,embed{height:100%}
img{-ms-interpolation-mode:bicubic}
.left{float:left!important}
.right{float:right!important}
.text-left{text-align:left!important}
.text-right{text-align:right!important}
.text-center{text-align:center!important}
.text-justify{text-align:justify!important}
.hide{display:none}
img,object,svg{display:inline-block;vertical-align:middle}
textarea{height:auto;min-height:50px}
select{width:100%}
.subheader,.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{line-height:1.45;color:#7a2518;font-weight:400;margin-top:0;margin-bottom:.25em}
div,dl,dt,dd,ul,ol,li,h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6,pre,form,p,blockquote,th,td{margin:0;padding:0}
a{color:#2156a5;text-decoration:underline;line-height:inherit}
a:hover,a:focus{color:#1d4b8f}
a img{border:0}
p{line-height:1.6;margin-bottom:1.25em;text-rendering:optimizeLegibility}
p aside{font-size:.875em;line-height:1.35;font-style:italic}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{font-family:"Open Sans","DejaVu Sans",sans-serif;font-weight:300;font-style:normal;color:#ba3925;text-rendering:optimizeLegibility;margin-top:1em;margin-bottom:.5em;line-height:1.0125em}
h1 small,h2 small,h3 small,#toctitle small,.sidebarblock>.content>.title small,h4 small,h5 small,h6 small{font-size:60%;color:#e99b8f;line-height:0}
h1{font-size:2.125em}
h2{font-size:1.6875em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.375em}
h4,h5{font-size:1.125em}
h6{font-size:1em}
hr{border:solid #dddddf;border-width:1px 0 0;clear:both;margin:1.25em 0 1.1875em}
em,i{font-style:italic;line-height:inherit}
strong,b{font-weight:bold;line-height:inherit}
small{font-size:60%;line-height:inherit}
code{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:400;color:rgba(0,0,0,.9)}
ul,ol,dl{line-height:1.6;margin-bottom:1.25em;list-style-position:outside;font-family:inherit}
ul,ol{margin-left:1.5em}
ul li ul,ul li ol{margin-left:1.25em;margin-bottom:0}
ul.circle{list-style-type:circle}
ul.disc{list-style-type:disc}
ul.square{list-style-type:square}
ul.circle ul:not([class]),ul.disc ul:not([class]),ul.square ul:not([class]){list-style:inherit}
ol li ul,ol li ol{margin-left:1.25em;margin-bottom:0}
dl dt{margin-bottom:.3125em;font-weight:bold}
dl dd{margin-bottom:1.25em}
blockquote{margin:0 0 1.25em;padding:.5625em 1.25em 0 1.1875em;border-left:1px solid #ddd}
blockquote,blockquote p{line-height:1.6;color:rgba(0,0,0,.85)}
@media screen and (min-width:768px){h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2}
h1{font-size:2.75em}
h2{font-size:2.3125em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.6875em}
h4{font-size:1.4375em}}
table{background:#fff;margin-bottom:1.25em;border:1px solid #dedede;word-wrap:normal}
table thead,table tfoot{background:#f7f8f7}
table thead tr th,table thead tr td,table tfoot tr th,table tfoot tr td{padding:.5em .625em .625em;font-size:inherit;color:rgba(0,0,0,.8);text-align:left}
table tr th,table tr td{padding:.5625em .625em;font-size:inherit;color:rgba(0,0,0,.8)}
table tr.even,table tr.alt{background:#f8f8f7}
table thead tr th,table tfoot tr th,table tbody tr td,table tr td,table tfoot tr td{line-height:1.6}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2;word-spacing:-.05em}
h1 strong,h2 strong,h3 strong,#toctitle strong,.sidebarblock>.content>.title strong,h4 strong,h5 strong,h6 strong{font-weight:400}
.center{margin-left:auto;margin-right:auto}
.stretch{width:100%}
.clearfix::before,.clearfix::after,.float-group::before,.float-group::after{content:" ";display:table}
.clearfix::after,.float-group::after{clear:both}
:not(pre).nobreak{word-wrap:normal}
:not(pre).nowrap{white-space:nowrap}
:not(pre).pre-wrap{white-space:pre-wrap}
:not(pre):not([class^=L])>code{font-size:.9375em;font-style:normal!important;letter-spacing:0;padding:.1em .5ex;word-spacing:-.15em;background:#f7f7f8;border-radius:4px;line-height:1.45;text-rendering:optimizeSpeed}
pre{color:rgba(0,0,0,.9);font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;line-height:1.45;text-rendering:optimizeSpeed}
pre code,pre pre{color:inherit;font-size:inherit;line-height:inherit}
pre>code{display:block}
pre.nowrap,pre.nowrap pre{white-space:pre;word-wrap:normal}
em em{font-style:normal}
strong strong{font-weight:400}
.keyseq{color:rgba(51,51,51,.8)}
kbd{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;display:inline-block;color:rgba(0,0,0,.8);font-size:.65em;line-height:1.45;background:#f7f7f7;border:1px solid #ccc;border-radius:3px;box-shadow:0 1px 0 rgba(0,0,0,.2),inset 0 0 0 .1em #fff;margin:0 .15em;padding:.2em .5em;vertical-align:middle;position:relative;top:-.1em;white-space:nowrap}
.keyseq kbd:first-child{margin-left:0}
.keyseq kbd:last-child{margin-right:0}
.menuseq,.menuref{color:#000}
.menuseq b:not(.caret),.menuref{font-weight:inherit}
.menuseq{word-spacing:-.02em}
.menuseq b.caret{font-size:1.25em;line-height:.8}
.menuseq i.caret{font-weight:bold;text-align:center;width:.45em}
b.button::before,b.button::after{position:relative;top:-1px;font-weight:400}
b.button::before{content:"[";padding:0 3px 0 2px}
b.button::after{content:"]";padding:0 2px 0 3px}
p a>code:hover{color:rgba(0,0,0,.9)}
#header,#content,#footnotes,#footer{width:100%;margin:0 auto;max-width:62.5em;*zoom:1;position:relative;padding-left:.9375em;padding-right:.9375em}
#header::before,#header::after,#content::before,#content::after,#footnotes::before,#footnotes::after,#footer::before,#footer::after{content:" ";display:table}
#header::after,#content::after,#footnotes::after,#footer::after{clear:both}
#content{margin-top:1.25em}
#content::before{content:none}
#header>h1:first-child{color:rgba(0,0,0,.85);margin-top:2.25rem;margin-bottom:0}
#header>h1:first-child+#toc{margin-top:8px;border-top:1px solid #dddddf}
#header>h1:only-child{border-bottom:1px solid #dddddf;padding-bottom:8px}
#header .details{border-bottom:1px solid #dddddf;line-height:1.45;padding-top:.25em;padding-bottom:.25em;padding-left:.25em;color:rgba(0,0,0,.6);display:flex;flex-flow:row wrap}
#header .details span:first-child{margin-left:-.125em}
#header .details span.email a{color:rgba(0,0,0,.85)}
#header .details br{display:none}
#header .details br+span::before{content:"\00a0\2013\00a0"}
#header .details br+span.author::before{content:"\00a0\22c5\00a0";color:rgba(0,0,0,.85)}
#header .details br+span#revremark::before{content:"\00a0|\00a0"}
#header #revnumber{text-transform:capitalize}
#header #revnumber::after{content:"\00a0"}
#content>h1:first-child:not([class]){color:rgba(0,0,0,.85);border-bottom:1px solid #dddddf;padding-bottom:8px;margin-top:0;padding-top:1rem;margin-bottom:1.25rem}
#toc{border-bottom:1px solid #e7e7e9;padding-bottom:.5em}
#toc>ul{margin-left:.125em}
#toc ul.sectlevel0>li>a{font-style:italic}
#toc ul.sectlevel0 ul.sectlevel1{margin:.5em 0}
#toc ul{font-family:"Open Sans","DejaVu Sans",sans-serif;list-style-type:none}
#toc li{line-height:1.3334;margin-top:.3334em}
#toc a{text-decoration:none}
#toc a:active{text-decoration:underline}
#toctitle{color:#7a2518;font-size:1.2em}
@media screen and (min-width:768px){#toctitle{font-size:1.375em}
body.toc2{padding-left:15em;padding-right:0}
body.toc2 #header>h1:nth-last-child(2){border-bottom:1px solid #dddddf;padding-bottom:8px}
#toc.toc2{margin-top:0!important;background:#f8f8f7;position:fixed;width:15em;left:0;top:0;border-right:1px solid #e7e7e9;border-top-width:0!important;border-bottom-width:0!important;z-index:1000;padding:1.25em 1em;height:100%;overflow:auto}
#toc.toc2 #toctitle{margin-top:0;margin-bottom:.8rem;font-size:1.2em}
#toc.toc2>ul{font-size:.9em;margin-bottom:0}
#toc.toc2 ul ul{margin-left:0;padding-left:1em}
#toc.toc2 ul.sectlevel0 ul.sectlevel1{padding-left:0;margin-top:.5em;margin-bottom:.5em}
body.toc2.toc-right{padding-left:0;padding-right:15em}
body.toc2.toc-right #toc.toc2{border-right-width:0;border-left:1px solid #e7e7e9;left:auto;right:0}}
@media screen and (min-width:1280px){body.toc2{padding-left:20em;padding-right:0}
#toc.toc2{width:20em}
#toc.toc2 #toctitle{font-size:1.375em}
#toc.toc2>ul{font-size:.95em}
#toc.toc2 ul ul{padding-left:1.25em}
body.toc2.toc-right{padding-left:0;padding-right:20em}}
#content #toc{border:1px solid #e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;border-radius:4px}
#content #toc>:first-child{margin-top:0}
#content #toc>:last-child{margin-bottom:0}
#footer{max-width:none;background:rgba(0,0,0,.8);padding:1.25em}
#footer-text{color:hsla(0,0%,100%,.8);line-height:1.44}
#content{margin-bottom:.625em}
.sect1{padding-bottom:.625em}
@media screen and (min-width:768px){#content{margin-bottom:1.25em}
.sect1{padding-bottom:1.25em}}
.sect1:last-child{padding-bottom:0}
.sect1+.sect1{border-top:1px solid #e7e7e9}
#content h1>a.anchor,h2>a.anchor,h3>a.anchor,#toctitle>a.anchor,.sidebarblock>.content>.title>a.anchor,h4>a.anchor,h5>a.anchor,h6>a.anchor{position:absolute;z-index:1001;width:1.5ex;margin-left:-1.5ex;display:block;text-decoration:none!important;visibility:hidden;text-align:center;font-weight:400}
#content h1>a.anchor::before,h2>a.anchor::before,h3>a.anchor::before,#toctitle>a.anchor::before,.sidebarblock>.content>.title>a.anchor::before,h4>a.anchor::before,h5>a.anchor::before,h6>a.anchor::before{content:"\00A7";font-size:.85em;display:block;padding-top:.1em}
#content h1:hover>a.anchor,#content h1>a.anchor:hover,h2:hover>a.anchor,h2>a.anchor:hover,h3:hover>a.anchor,#toctitle:hover>a.anchor,.sidebarblock>.content>.title:hover>a.anchor,h3>a.anchor:hover,#toctitle>a.anchor:hover,.sidebarblock>.content>.title>a.anchor:hover,h4:hover>a.anchor,h4>a.anchor:hover,h5:hover>a.anchor,h5>a.anchor:hover,h6:hover>a.anchor,h6>a.anchor:hover{visibility:visible}
#content h1>a.link,h2>a.link,h3>a.link,#toctitle>a.link,.sidebarblock>.content>.title>a.link,h4>a.link,h5>a.link,h6>a.link{color:#ba3925;text-decoration:none}
#content h1>a.link:hover,h2>a.link:hover,h3>a.link:hover,#toctitle>a.link:hover,.sidebarblock>.content>.title>a.link:hover,h4>a.link:hover,h5>a.link:hover,h6>a.link:hover{color:#a53221}
details,.audioblock,.imageblock,.literalblock,.listingblock,.stemblock,.videoblock{margin-bottom:1.25em}
details{margin-left:1.25rem}
details>summary{cursor:pointer;display:block;position:relative;line-height:1.6;margin-bottom:.625rem;outline:none;-webkit-tap-highlight-color:transparent}
details>summary::-webkit-details-marker{display:none}
details>summary::before{content:"";border:solid transparent;border-left:solid;border-width:.3em 0 .3em .5em;position:absolute;top:.5em;left:-1.25rem;transform:translateX(15%)}
details[open]>summary::before{border:solid transparent;border-top:solid;border-width:.5em .3em 0;transform:translateY(15%)}
details>summary::after{content:"";width:1.25rem;height:1em;position:absolute;top:.3em;left:-1.25rem}
.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{text-rendering:optimizeLegibility;text-align:left;font-family:"Noto Serif","DejaVu Serif",serif;font-size:1rem;font-style:italic}
table.tableblock.fit-content>caption.title{white-space:nowrap;width:0}
.paragraph.lead>p,#preamble>.sectionbody>[class=paragraph]:first-of-type p{font-size:1.21875em;line-height:1.6;color:rgba(0,0,0,.85)}
.admonitionblock>table{border-collapse:separate;border:0;background:none;width:100%}
.admonitionblock>table td.icon{text-align:center;width:80px}
.admonitionblock>table td.icon img{max-width:none}
.admonitionblock>table td.icon .title{font-weight:bold;font-family:"Open Sans","DejaVu Sans",sans-serif;text-transform:uppercase}
.admonitionblock>table td.content{padding-left:1.125em;padding-right:1.25em;border-left:1px solid #dddddf;color:rgba(0,0,0,.6);word-wrap:anywhere}
.admonitionblock>table td.content>:last-child>:last-child{margin-bottom:0}
.exampleblock>.content{border:1px solid #e6e6e6;margin-bottom:1.25em;padding:1.25em;background:#fff;border-radius:4px}
.sidebarblock{border:1px solid #dbdbd6;margin-bottom:1.25em;padding:1.25em;background:#f3f3f2;border-radius:4px}
.sidebarblock>.content>.title{color:#7a2518;margin-top:0;text-align:center}
.exampleblock>.content>:first-child,.sidebarblock>.content>:first-child{margin-top:0}
.exampleblock>.content>:last-child,.exampleblock>.content>:last-child>:last-child,.exampleblock>.content .olist>ol>li:last-child>:last-child,.exampleblock>.content .ulist>ul>li:last-child>:last-child,.exampleblock>.content .qlist>ol>li:last-child>:last-child,.sidebarblock>.content>:last-child,.sidebarblock>.content>:last-child>:last-child,.sidebarblock>.content .olist>ol>li:last-child>:last-child,.sidebarblock>.content .ulist>ul>li:last-child>:last-child,.sidebarblock>.content .qlist>ol>li:last-child>:last-child{margin-bottom:0}
.literalblock pre,.listingblock>.content>pre{border-radius:4px;overflow-x:auto;padding:1em;font-size:.8125em}
@media screen and (min-width:768px){.literalblock pre,.listingblock>.content>pre{font-size:.90625em}}
@media screen and (min-width:1280px){.literalblock pre,.listingblock>.content>pre{font-size:1em}}
.literalblock pre,.listingblock>.content>pre:not(.highlight),.listingblock>.content>pre[class=highlight],.listingblock>.content>pre[class^="highlight "]{background:#f7f7f8}
.literalblock.output pre{color:#f7f7f8;background:rgba(0,0,0,.9)}
.listingblock>.content{position:relative}
.listingblock code[data-lang]::before{display:none;content:attr(data-lang);position:absolute;font-size:.75em;top:.425rem;right:.5rem;line-height:1;text-transform:uppercase;color:inherit;opacity:.5}
.listingblock:hover code[data-lang]::before{display:block}
.listingblock.terminal pre .command::before{content:attr(data-prompt);padding-right:.5em;color:inherit;opacity:.5}
.listingblock.terminal pre .command:not([data-prompt])::before{content:"$"}
.listingblock pre.highlightjs{padding:0}
.listingblock pre.highlightjs>code{padding:1em;border-radius:4px}
.listingblock pre.prettyprint{border-width:0}
.prettyprint{background:#f7f7f8}
pre.prettyprint .linenums{line-height:1.45;margin-left:2em}
pre.prettyprint li{background:none;list-style-type:inherit;padding-left:0}
pre.prettyprint li code[data-lang]::before{opacity:1}
pre.prettyprint li:not(:first-child) code[data-lang]::before{display:none}
table.linenotable{border-collapse:separate;border:0;margin-bottom:0;background:none}
table.linenotable td[class]{color:inherit;vertical-align:top;padding:0;line-height:inherit;white-space:normal}
table.linenotable td.code{padding-left:.75em}
table.linenotable td.linenos,pre.pygments .linenos{border-right:1px solid;opacity:.35;padding-right:.5em;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}
pre.pygments span.linenos{display:inline-block;margin-right:.75em}
.quoteblock{margin:0 1em 1.25em 1.5em;display:table}
.quoteblock:not(.excerpt)>.title{margin-left:-1.5em;margin-bottom:.75em}
.quoteblock blockquote,.quoteblock p{color:rgba(0,0,0,.85);font-size:1.15rem;line-height:1.75;word-spacing:.1em;letter-spacing:0;font-style:italic;text-align:justify}
.quoteblock blockquote{margin:0;padding:0;border:0}
.quoteblock blockquote::before{content:"\201c";float:left;font-size:2.75em;font-weight:bold;line-height:.6em;margin-left:-.6em;color:#7a2518;text-shadow:0 1px 2px rgba(0,0,0,.1)}
.quoteblock blockquote>.paragraph:last-child p{margin-bottom:0}
.quoteblock .attribution{margin-top:.75em;margin-right:.5ex;text-align:right}
.verseblock{margin:0 1em 1.25em}
.verseblock pre{font-family:"Open Sans","DejaVu Sans",sans-serif;font-size:1.15rem;color:rgba(0,0,0,.85);font-weight:300;text-rendering:optimizeLegibility}
.verseblock pre strong{font-weight:400}
.verseblock .attribution{margin-top:1.25rem;margin-left:.5ex}
.quoteblock .attribution,.verseblock .attribution{font-size:.9375em;line-height:1.45;font-style:italic}
.quoteblock .attribution br,.verseblock .attribution br{display:none}
.quoteblock .attribution cite,.verseblock .attribution cite{display:block;letter-spacing:-.025em;color:rgba(0,0,0,.6)}
.quoteblock.abstract blockquote::before,.quoteblock.excerpt blockquote::before,.quoteblock .quoteblock blockquote::before{display:none}
.quoteblock.abstract blockquote,.quoteblock.abstract p,.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{line-height:1.6;word-spacing:0}
.quoteblock.abstract{margin:0 1em 1.25em;display:block}
.quoteblock.abstract>.title{margin:0 0 .375em;font-size:1.15em;text-align:center}
.quoteblock.excerpt>blockquote,.quoteblock .quoteblock{padding:0 0 .25em 1em;border-left:.25em solid #dddddf}
.quoteblock.excerpt,.quoteblock .quoteblock{margin-left:0}
.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{color:inherit;font-size:1.0625rem}
.quoteblock.excerpt .attribution,.quoteblock .quoteblock .attribution{color:inherit;font-size:.85rem;text-align:left;margin-right:0}
p.tableblock:last-child{margin-bottom:0}
td.tableblock>.content{margin-bottom:1.25em;word-wrap:anywhere}
td.tableblock>.content>:last-child{margin-bottom:-1.25em}
table.tableblock,th.tableblock,td.tableblock{border:0 solid #dedede}
table.grid-all>*>tr>*{border-width:1px}
table.grid-cols>*>tr>*{border-width:0 1px}
table.grid-rows>*>tr>*{border-width:1px 0}
table.frame-all{border-width:1px}
table.frame-ends{border-width:1px 0}
table.frame-sides{border-width:0 1px}
table.frame-none>colgroup+*>:first-child>*,table.frame-sides>colgroup+*>:first-child>*{border-top-width:0}
table.frame-none>:last-child>:last-child>*,table.frame-sides>:last-child>:last-child>*{border-bottom-width:0}
table.frame-none>*>tr>:first-child,table.frame-ends>*>tr>:first-child{border-left-width:0}
table.frame-none>*>tr>:last-child,table.frame-ends>*>tr>:last-child{border-right-width:0}
table.stripes-all>*>tr,table.stripes-odd>*>tr:nth-of-type(odd),table.stripes-even>*>tr:nth-of-type(even),table.stripes-hover>*>tr:hover{background:#f8f8f7}
th.halign-left,td.halign-left{text-align:left}
th.halign-right,td.halign-right{text-align:right}
th.halign-center,td.halign-center{text-align:center}
th.valign-top,td.valign-top{vertical-align:top}
th.valign-bottom,td.valign-bottom{vertical-align:bottom}
th.valign-middle,td.valign-middle{vertical-align:middle}
table thead th,table tfoot th{font-weight:bold}
tbody tr th{background:#f7f8f7}
tbody tr th,tbody tr th p,tfoot tr th,tfoot tr th p{color:rgba(0,0,0,.8);font-weight:bold}
p.tableblock>code:only-child{background:none;padding:0}
p.tableblock{font-size:1em}
ol{margin-left:1.75em}
ul li ol{margin-left:1.5em}
dl dd{margin-left:1.125em}
dl dd:last-child,dl dd:last-child>:last-child{margin-bottom:0}
li p,ul dd,ol dd,.olist .olist,.ulist .ulist,.ulist .olist,.olist .ulist{margin-bottom:.625em}
ul.checklist,ul.none,ol.none,ul.no-bullet,ol.no-bullet,ol.unnumbered,ul.unstyled,ol.unstyled{list-style-type:none}
ul.no-bullet,ol.no-bullet,ol.unnumbered{margin-left:.625em}
ul.unstyled,ol.unstyled{margin-left:0}
li>p:empty:only-child::before{content:"";display:inline-block}
ul.checklist>li>p:first-child{margin-left:-1em}
ul.checklist>li>p:first-child>.fa-square-o:first-child,ul.checklist>li>p:first-child>.fa-check-square-o:first-child{width:1.25em;font-size:.8em;position:relative;bottom:.125em}
ul.checklist>li>p:first-child>input[type=checkbox]:first-child{margin-right:.25em}
ul.inline{display:flex;flex-flow:row wrap;list-style:none;margin:0 0 .625em -1.25em}
ul.inline>li{margin-left:1.25em}
.unstyled dl dt{font-weight:400;font-style:normal}
ol.arabic{list-style-type:decimal}
ol.decimal{list-style-type:decimal-leading-zero}
ol.loweralpha{list-style-type:lower-alpha}
ol.upperalpha{list-style-type:upper-alpha}
ol.lowerroman{list-style-type:lower-roman}
ol.upperroman{list-style-type:upper-roman}
ol.lowergreek{list-style-type:lower-greek}
.hdlist>table,.colist>table{border:0;background:none}
.hdlist>table>tbody>tr,.colist>table>tbody>tr{background:none}
td.hdlist1,td.hdlist2{vertical-align:top;padding:0 .625em}
td.hdlist1{font-weight:bold;padding-bottom:1.25em}
td.hdlist2{word-wrap:anywhere}
.literalblock+.colist,.listingblock+.colist{margin-top:-.5em}
.colist td:not([class]):first-child{padding:.4em .75em 0;line-height:1;vertical-align:top}
.colist td:not([class]):first-child img{max-width:none}
.colist td:not([class]):last-child{padding:.25em 0}
.thumb,.th{line-height:0;display:inline-block;border:4px solid #fff;box-shadow:0 0 0 1px #ddd}
.imageblock.left{margin:.25em .625em 1.25em 0}
.imageblock.right{margin:.25em 0 1.25em .625em}
.imageblock>.title{margin-bottom:0}
.imageblock.thumb,.imageblock.th{border-width:6px}
.imageblock.thumb>.title,.imageblock.th>.title{padding:0 .125em}
.image.left,.image.right{margin-top:.25em;margin-bottom:.25em;display:inline-block;line-height:0}
.image.left{margin-right:.625em}
.image.right{margin-left:.625em}
a.image{text-decoration:none;display:inline-block}
a.image object{pointer-events:none}
sup.footnote,sup.footnoteref{font-size:.875em;position:static;vertical-align:super}
sup.footnote a,sup.footnoteref a{text-decoration:none}
sup.footnote a:active,sup.footnoteref a:active,#footnotes .footnote a:first-of-type:active{text-decoration:underline}
#footnotes{padding-top:.75em;padding-bottom:.75em;margin-bottom:.625em}
#footnotes hr{width:20%;min-width:6.25em;margin:-.25em 0 .75em;border-width:1px 0 0}
#footnotes .footnote{padding:0 .375em 0 .225em;line-height:1.3334;font-size:.875em;margin-left:1.2em;margin-bottom:.2em}
#footnotes .footnote a:first-of-type{font-weight:bold;text-decoration:none;margin-left:-1.05em}
#footnotes .footnote:last-of-type{margin-bottom:0}
#content #footnotes{margin-top:-.625em;margin-bottom:0;padding:.75em 0}
div.unbreakable{page-break-inside:avoid}
.big{font-size:larger}
.small{font-size:smaller}
.underline{text-decoration:underline}
.overline{text-decoration:overline}
.line-through{text-decoration:line-through}
.aqua{color:#00bfbf}
.aqua-background{background:#00fafa}
.black{color:#000}
.black-background{background:#000}
.blue{color:#0000bf}
.blue-background{background:#0000fa}
.fuchsia{color:#bf00bf}
.fuchsia-background{background:#fa00fa}
.gray{color:#606060}
.gray-background{background:#7d7d7d}
.green{color:#006000}
.green-background{background:#007d00}
.lime{color:#00bf00}
.lime-background{background:#00fa00}
.maroon{color:#600000}
.maroon-background{background:#7d0000}
.navy{color:#000060}
.navy-background{background:#00007d}
.olive{color:#606000}
.olive-background{background:#7d7d00}
.purple{color:#600060}
.purple-background{background:#7d007d}
.red{color:#bf0000}
.red-background{background:#fa0000}
.silver{color:#909090}
.silver-background{background:#bcbcbc}
.teal{color:#006060}
.teal-background{background:#007d7d}
.white{color:#bfbfbf}
.white-background{background:#fafafa}
.yellow{color:#bfbf00}
.yellow-background{background:#fafa00}
span.icon>.fa{cursor:default}
a span.icon>.fa{cursor:inherit}
.admonitionblock td.icon [class^="fa icon-"]{font-size:2.5em;text-shadow:1px 1px 2px rgba(0,0,0,.5);cursor:default}
.admonitionblock td.icon .icon-note::before{content:"\f05a";color:#19407c}
.admonitionblock td.icon .icon-tip::before{content:"\f0eb";text-shadow:1px 1px 2px rgba(155,155,0,.8);color:#111}
.admonitionblock td.icon .icon-warning::before{content:"\f071";color:#bf6900}
.admonitionblock td.icon .icon-caution::before{content:"\f06d";color:#bf3400}
.admonitionblock td.icon .icon-important::before{content:"\f06a";color:#bf0000}
.conum[data-value]{display:inline-block;color:#fff!important;background:rgba(0,0,0,.8);border-radius:50%;text-align:center;font-size:.75em;width:1.67em;height:1.67em;line-height:1.67em;font-family:"Open Sans","DejaVu Sans",sans-serif;font-style:normal;font-weight:bold}
.conum[data-value] *{color:#fff!important}
.conum[data-value]+b{display:none}
.conum[data-value]::after{content:attr(data-value)}
pre .conum[data-value]{position:relative;top:-.125em}
b.conum *{color:inherit!important}
.conum:not([data-value]):empty{display:none}
dt,th.tableblock,td.content,div.footnote{text-rendering:optimizeLegibility}
h1,h2,p,td.content,span.alt,summary{letter-spacing:-.01em}
p strong,td.content strong,div.footnote strong{letter-spacing:-.005em}
p,blockquote,dt,td.content,td.hdlist1,span.alt,summary{font-size:1.0625rem}
p{margin-bottom:1.25rem}
.sidebarblock p,.sidebarblock dt,.sidebarblock td.content,p.tableblock{font-size:1em}
.exampleblock>.content{background:#fffef7;border-color:#e0e0dc;box-shadow:0 1px 4px #e0e0dc}
.print-only{display:none!important}
@page{margin:1.25cm .75cm}
@media print{*{box-shadow:none!important;text-shadow:none!important}
html{font-size:80%}
a{color:inherit!important;text-decoration:underline!important}
a.bare,a[href^="#"],a[href^="mailto:"]{text-decoration:none!important}
a[href^="http:"]:not(.bare)::after,a[href^="https:"]:not(.bare)::after{content:"(" attr(href) ")";display:inline-block;font-size:.875em;padding-left:.25em}
abbr[title]{border-bottom:1px dotted}
abbr[title]::after{content:" (" attr(title) ")"}
pre,blockquote,tr,img,object,svg{page-break-inside:avoid}
thead{display:table-header-group}
svg{max-width:100%}
p,blockquote,dt,td.content{font-size:1em;orphans:3;widows:3}
h2,h3,#toctitle,.sidebarblock>.content>.title{page-break-after:avoid}
#header,#content,#footnotes,#footer{max-width:none}
#toc,.sidebarblock,.exampleblock>.content{background:none!important}
#toc{border-bottom:1px solid #dddddf!important;padding-bottom:0!important}
body.book #header{text-align:center}
body.book #header>h1:first-child{border:0!important;margin:2.5em 0 1em}
body.book #header .details{border:0!important;display:block;padding:0!important}
body.book #header .details span:first-child{margin-left:0!important}
body.book #header .details br{display:block}
body.book #header .details br+span::before{content:none!important}
body.book #toc{border:0!important;text-align:left!important;padding:0!important;margin:0!important}
body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-break-before:always}
.listingblock code[data-lang]::before{display:block}
#footer{padding:0 .9375em}
.hide-on-print{display:none!important}
.print-only{display:block!important}
.hide-for-print{display:none!important}
.show-for-print{display:inherit!important}}
@media amzn-kf8,print{#header>h1:first-child{margin-top:1.25rem}
.sect1{padding:0!important}
.sect1+.sect1{border:0}
#footer{background:none}
#footer-text{color:rgba(0,0,0,.6);font-size:.9em}}
@media amzn-kf8{#header,#content,#footnotes,#footer{padding:0}}
</style>
</head>
<body class="article">
<div id="header">
<h1>merge API</h1>
</div>
<div id="content">
<div id="preamble">
<div class="sectionbody">
<div class="paragraph">
<p>The merge API helps a program to reconcile two competing sets of
improvements to some files (e.g., unregistered changes from the work
tree versus changes involved in switching to a new branch), reporting
conflicts if found. The library called through this API is
responsible for a few things.</p>
</div>
<div class="ulist">
<ul>
<li>
<p>determining which trees to merge (recursive ancestor consolidation);</p>
</li>
<li>
<p>lining up corresponding files in the trees to be merged (rename
detection, subtree shifting), reporting edge cases like add/add
and rename/rename conflicts to the user;</p>
</li>
<li>
<p>performing a three-way merge of corresponding files, taking
path-specific merge drivers (specified in .<code>gitattributes</code>)
into account.</p>
</li>
</ul>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_data_structures">Data structures</h2>
<div class="sectionbody">
<div class="ulist">
<ul>
<li>
<p><code>mmbuffer_t</code>, <code>mmfile_t</code></p>
</li>
</ul>
</div>
<div class="paragraph">
<p>These store data usable for use by the xdiff backend, for writing and
for reading, respectively. See <code>xdiff/xdiff.h</code> for the definitions
and <code>diff.c</code> for examples.</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>struct</code> <code>ll_merge_options</code></p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Check merge-ll.h for details.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_low_level_single_file_merge">Low-level (single file) merge</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Check merge-ll.h for details.</p>
</div>
</div>
</div>
</div>
<div id="footer">
<div id="footer-text">
Last updated 2025-03-17 07:09:38 UTC
</div>
</div>
</body>
</html>

View File

@@ -0,0 +1,349 @@
parse-options API
=================
The parse-options API is used to parse and massage options in Git
and to provide a usage help with consistent look.
Basics
------
The argument vector `argv[]` may usually contain mandatory or optional
'non-option arguments', e.g. a filename or a branch, 'options', and
'subcommands'.
Options are optional arguments that start with a dash and
that allow to change the behavior of a command.
* There are basically three types of options:
'boolean' options,
options with (mandatory) 'arguments' and
options with 'optional arguments'
(i.e. a boolean option that can be adjusted).
* There are basically two forms of options:
'Short options' consist of one dash (`-`) and one alphanumeric
character.
'Long options' begin with two dashes (`--`) and some
alphanumeric characters.
* Options are case-sensitive.
Please define 'lower-case long options' only.
The parse-options API allows:
* 'stuck' and 'separate form' of options with arguments.
`-oArg` is stuck, `-o Arg` is separate form.
`--option=Arg` is stuck, `--option Arg` is separate form.
* Long options may be 'abbreviated', as long as the abbreviation
is unambiguous.
* Short options may be bundled, e.g. `-a -b` can be specified as `-ab`.
* Boolean long options can be 'negated' (or 'unset') by prepending
`no-`, e.g. `--no-abbrev` instead of `--abbrev`. Conversely,
options that begin with `no-` can be 'negated' by removing it.
Other long options can be unset (e.g., set string to NULL, set
integer to 0) by prepending `no-`.
* Options and non-option arguments can clearly be separated using the `--`
option, e.g. `-a -b --option -- --this-is-a-file` indicates that
`--this-is-a-file` must not be processed as an option.
Subcommands are special in a couple of ways:
* Subcommands only have long form, and they have no double dash prefix, no
negated form, and no description, and they don't take any arguments, and
can't be abbreviated.
* There must be exactly one subcommand among the arguments, or zero if the
command has a default operation mode.
* All arguments following the subcommand are considered to be arguments of
the subcommand, and, conversely, arguments meant for the subcommand may
not precede the subcommand.
Therefore, if the options array contains at least one subcommand and
`parse_options()` encounters the first dashless argument, it will either:
* stop and return, if that dashless argument is a known subcommand, setting
`value` to the function pointer associated with that subcommand, storing
the name of the subcommand in argv[0], and leaving the rest of the
arguments unprocessed, or
* stop and return, if it was invoked with the `PARSE_OPT_SUBCOMMAND_OPTIONAL`
flag and that dashless argument doesn't match any subcommands, leaving
`value` unchanged and the rest of the arguments unprocessed, or
* show error and usage, and abort.
Steps to parse options
----------------------
. `#include "parse-options.h"`
. define a NULL-terminated
`static const char * const builtin_foo_usage[]` array
containing alternative usage strings
. define `builtin_foo_options` array as described below
in section 'Data Structure'.
. in `cmd_foo(int argc, const char **argv, const char *prefix)`
call
argc = parse_options(argc, argv, prefix, builtin_foo_options, builtin_foo_usage, flags);
+
`parse_options()` will filter out the processed options of `argv[]` and leave the
non-option arguments in `argv[]`.
`argc` is updated appropriately because of the assignment.
+
You can also pass NULL instead of a usage array as the fifth parameter of
parse_options(), to avoid displaying a help screen with usage info and
option list. This should only be done if necessary, e.g. to implement
a limited parser for only a subset of the options that needs to be run
before the full parser, which in turn shows the full help message.
+
Flags are the bitwise-or of:
`PARSE_OPT_KEEP_DASHDASH`::
Keep the `--` that usually separates options from
non-option arguments.
`PARSE_OPT_STOP_AT_NON_OPTION`::
Usually the whole argument vector is massaged and reordered.
Using this flag, processing is stopped at the first non-option
argument.
`PARSE_OPT_KEEP_ARGV0`::
Keep the first argument, which contains the program name. It's
removed from argv[] by default.
`PARSE_OPT_KEEP_UNKNOWN_OPT`::
Keep unknown options instead of erroring out. This doesn't
work for all combinations of arguments as users might expect
it to do. E.g. if the first argument in `--unknown --known`
takes a value (which we can't know), the second one is
mistakenly interpreted as a known option. Similarly, if
`PARSE_OPT_STOP_AT_NON_OPTION` is set, the second argument in
`--unknown value` will be mistakenly interpreted as a
non-option, not as a value belonging to the unknown option,
the parser early. That's why parse_options() errors out if
both options are set.
Note that non-option arguments are always kept, even without
this flag.
`PARSE_OPT_NO_INTERNAL_HELP`::
By default, parse_options() handles `-h`, `--help` and
`--help-all` internally, by showing a help screen. This option
turns it off and allows one to add custom handlers for these
options, or to just leave them unknown.
`PARSE_OPT_SUBCOMMAND_OPTIONAL`::
Don't error out when no subcommand is specified.
Note that `PARSE_OPT_STOP_AT_NON_OPTION` is incompatible with subcommands;
while `PARSE_OPT_KEEP_DASHDASH` and `PARSE_OPT_KEEP_UNKNOWN_OPT` can only be
used with subcommands when combined with `PARSE_OPT_SUBCOMMAND_OPTIONAL`.
Data Structure
--------------
The main data structure is an array of the `option` struct,
say `static struct option builtin_add_options[]`.
There are some macros to easily define options:
`OPT__ABBREV(&int_var)`::
Add `--abbrev[=<n>]`.
`OPT__COLOR(&int_var, description)`::
Add `--color[=<when>]` and `--no-color`.
`OPT__DRY_RUN(&int_var, description)`::
Add `-n, --dry-run`.
`OPT__FORCE(&int_var, description)`::
Add `-f, --force`.
`OPT__QUIET(&int_var, description)`::
Add `-q, --quiet`.
`OPT__VERBOSE(&int_var, description)`::
Add `-v, --verbose`.
`OPT_GROUP(description)`::
Start an option group. `description` is a short string that
describes the group or an empty string.
Start the description with an upper-case letter.
`OPT_BOOL(short, long, &int_var, description)`::
Introduce a boolean option. `int_var` is set to one with
`--option` and set to zero with `--no-option`.
`OPT_COUNTUP(short, long, &int_var, description)`::
Introduce a count-up option.
Each use of `--option` increments `int_var`, starting from zero
(even if initially negative), and `--no-option` resets it to
zero. To determine if `--option` or `--no-option` was encountered at
all, initialize `int_var` to a negative value, and if it is still
negative after parse_options(), then neither `--option` nor
`--no-option` was seen.
`OPT_BIT(short, long, &int_var, description, mask)`::
Introduce a boolean option.
If used, `int_var` is bitwise-ored with `mask`.
`OPT_NEGBIT(short, long, &int_var, description, mask)`::
Introduce a boolean option.
If used, `int_var` is bitwise-anded with the inverted `mask`.
`OPT_SET_INT(short, long, &int_var, description, integer)`::
Introduce an integer option.
`int_var` is set to `integer` with `--option`, and
reset to zero with `--no-option`.
`OPT_STRING(short, long, &str_var, arg_str, description)`::
Introduce an option with string argument.
The string argument is put into `str_var`.
`OPT_STRING_LIST(short, long, &struct string_list, arg_str, description)`::
Introduce an option with string argument.
The string argument is stored as an element in `string_list`.
Use of `--no-option` will clear the list of preceding values.
`OPT_INTEGER(short, long, &int_var, description)`::
Introduce an option with integer argument.
The integer is put into `int_var`.
`OPT_MAGNITUDE(short, long, &unsigned_long_var, description)`::
Introduce an option with a size argument. The argument must be a
non-negative integer and may include a suffix of 'k', 'm' or 'g' to
scale the provided value by 1024, 1024^2 or 1024^3 respectively.
The scaled value is put into `unsigned_long_var`.
`OPT_EXPIRY_DATE(short, long, &timestamp_t_var, description)`::
Introduce an option with expiry date argument, see `parse_expiry_date()`.
The timestamp is put into `timestamp_t_var`.
`OPT_CALLBACK(short, long, &var, arg_str, description, func_ptr)`::
Introduce an option with argument.
The argument will be fed into the function given by `func_ptr`
and the result will be put into `var`.
See 'Option Callbacks' below for a more elaborate description.
`OPT_FILENAME(short, long, &var, description)`::
Introduce an option with a filename argument.
The filename will be prefixed by passing the filename along with
the prefix argument of `parse_options()` to `prefix_filename()`.
`OPT_NUMBER_CALLBACK(&var, description, func_ptr)`::
Recognize numerical options like -123 and feed the integer as
if it was an argument to the function given by `func_ptr`.
The result will be put into `var`. There can be only one such
option definition. It cannot be negated and it takes no
arguments. Short options that happen to be digits take
precedence over it.
`OPT_COLOR_FLAG(short, long, &int_var, description)`::
Introduce an option that takes an optional argument that can
have one of three values: "always", "never", or "auto". If the
argument is not given, it defaults to "always". The `--no-` form
works like `--long=never`; it cannot take an argument. If
"always", set `int_var` to 1; if "never", set `int_var` to 0; if
"auto", set `int_var` to 1 if stdout is a tty or a pager,
0 otherwise.
`OPT_NOOP_NOARG(short, long)`::
Introduce an option that has no effect and takes no arguments.
Use it to hide deprecated options that are still to be recognized
and ignored silently.
`OPT_PASSTHRU(short, long, &char_var, arg_str, description, flags)`::
Introduce an option that will be reconstructed into a char* string,
which must be initialized to NULL. This is useful when you need to
pass the command-line option to another command. Any previous value
will be overwritten, so this should only be used for options where
the last one specified on the command line wins.
`OPT_PASSTHRU_ARGV(short, long, &strvec_var, arg_str, description, flags)`::
Introduce an option where all instances of it on the command-line will
be reconstructed into a strvec. This is useful when you need to
pass the command-line option, which can be specified multiple times,
to another command.
`OPT_CMDMODE(short, long, &int_var, description, enum_val)`::
Define an "operation mode" option, only one of which in the same
group of "operating mode" options that share the same `int_var`
can be given by the user. `int_var` is set to `enum_val` when the
option is used, but an error is reported if other "operating mode"
option has already set its value to the same `int_var`.
In new commands consider using subcommands instead.
`OPT_SUBCOMMAND(long, &fn_ptr, subcommand_fn)`::
Define a subcommand. `subcommand_fn` is put into `fn_ptr` when
this subcommand is used.
The last element of the array must be `OPT_END()`.
If not stated otherwise, interpret the arguments as follows:
* `short` is a character for the short option
(e.g. `'e'` for `-e`, use `0` to omit),
* `long` is a string for the long option
(e.g. `"example"` for `--example`, use `NULL` to omit),
* `int_var` is an integer variable,
* `str_var` is a string variable (`char *`),
* `arg_str` is the string that is shown as argument
(e.g. `"branch"` will result in `<branch>`).
If set to `NULL`, three dots (`...`) will be displayed.
* `description` is a short string to describe the effect of the option.
It shall begin with a lower-case letter and a full stop (`.`) shall be
omitted at the end.
Option Callbacks
----------------
The function must be defined in this form:
int func(const struct option *opt, const char *arg, int unset)
The callback mechanism is as follows:
* Inside `func`, the only interesting member of the structure
given by `opt` is the void pointer `opt->value`.
`*opt->value` will be the value that is saved into `var`, if you
use `OPT_CALLBACK()`.
For example, do `*(unsigned long *)opt->value = 42;` to get 42
into an `unsigned long` variable.
* Return value `0` indicates success and non-zero return
value will invoke `usage_with_options()` and, thus, die.
* If the user negates the option, `arg` is `NULL` and `unset` is 1.
Sophisticated option parsing
----------------------------
If you need, for example, option callbacks with optional arguments
or without arguments at all, or if you need other special cases,
that are not handled by the macros above, you need to specify the
members of the `option` structure manually.
This is not covered in this document, but well documented
in `parse-options.h` itself.
Examples
--------
See `test-parse-options.c` and
`builtin/add.c`,
`builtin/clone.c`,
`builtin/commit.c`,
`builtin/fetch.c`,
`builtin/fsck.c`,
`builtin/rm.c`
for real-world examples.

View File

@@ -0,0 +1,930 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="en">
<head>
<meta charset="UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=edge"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<meta name="generator" content="Asciidoctor 2.0.23"/>
<title>parse-options API</title>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"/>
<style>
/*! Asciidoctor default stylesheet | MIT License | https://asciidoctor.org */
/* Uncomment the following line when using as a custom stylesheet */
/* @import "https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"; */
html{font-family:sans-serif;-webkit-text-size-adjust:100%}
a{background:none}
a:focus{outline:thin dotted}
a:active,a:hover{outline:0}
h1{font-size:2em;margin:.67em 0}
b,strong{font-weight:bold}
abbr{font-size:.9em}
abbr[title]{cursor:help;border-bottom:1px dotted #dddddf;text-decoration:none}
dfn{font-style:italic}
hr{height:0}
mark{background:#ff0;color:#000}
code,kbd,pre,samp{font-family:monospace;font-size:1em}
pre{white-space:pre-wrap}
q{quotes:"\201C" "\201D" "\2018" "\2019"}
small{font-size:80%}
sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline}
sup{top:-.5em}
sub{bottom:-.25em}
img{border:0}
svg:not(:root){overflow:hidden}
figure{margin:0}
audio,video{display:inline-block}
audio:not([controls]){display:none;height:0}
fieldset{border:1px solid silver;margin:0 2px;padding:.35em .625em .75em}
legend{border:0;padding:0}
button,input,select,textarea{font-family:inherit;font-size:100%;margin:0}
button,input{line-height:normal}
button,select{text-transform:none}
button,html input[type=button],input[type=reset],input[type=submit]{-webkit-appearance:button;cursor:pointer}
button[disabled],html input[disabled]{cursor:default}
input[type=checkbox],input[type=radio]{padding:0}
button::-moz-focus-inner,input::-moz-focus-inner{border:0;padding:0}
textarea{overflow:auto;vertical-align:top}
table{border-collapse:collapse;border-spacing:0}
*,::before,::after{box-sizing:border-box}
html,body{font-size:100%}
body{background:#fff;color:rgba(0,0,0,.8);padding:0;margin:0;font-family:"Noto Serif","DejaVu Serif",serif;line-height:1;position:relative;cursor:auto;-moz-tab-size:4;-o-tab-size:4;tab-size:4;word-wrap:anywhere;-moz-osx-font-smoothing:grayscale;-webkit-font-smoothing:antialiased}
a:hover{cursor:pointer}
img,object,embed{max-width:100%;height:auto}
object,embed{height:100%}
img{-ms-interpolation-mode:bicubic}
.left{float:left!important}
.right{float:right!important}
.text-left{text-align:left!important}
.text-right{text-align:right!important}
.text-center{text-align:center!important}
.text-justify{text-align:justify!important}
.hide{display:none}
img,object,svg{display:inline-block;vertical-align:middle}
textarea{height:auto;min-height:50px}
select{width:100%}
.subheader,.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{line-height:1.45;color:#7a2518;font-weight:400;margin-top:0;margin-bottom:.25em}
div,dl,dt,dd,ul,ol,li,h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6,pre,form,p,blockquote,th,td{margin:0;padding:0}
a{color:#2156a5;text-decoration:underline;line-height:inherit}
a:hover,a:focus{color:#1d4b8f}
a img{border:0}
p{line-height:1.6;margin-bottom:1.25em;text-rendering:optimizeLegibility}
p aside{font-size:.875em;line-height:1.35;font-style:italic}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{font-family:"Open Sans","DejaVu Sans",sans-serif;font-weight:300;font-style:normal;color:#ba3925;text-rendering:optimizeLegibility;margin-top:1em;margin-bottom:.5em;line-height:1.0125em}
h1 small,h2 small,h3 small,#toctitle small,.sidebarblock>.content>.title small,h4 small,h5 small,h6 small{font-size:60%;color:#e99b8f;line-height:0}
h1{font-size:2.125em}
h2{font-size:1.6875em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.375em}
h4,h5{font-size:1.125em}
h6{font-size:1em}
hr{border:solid #dddddf;border-width:1px 0 0;clear:both;margin:1.25em 0 1.1875em}
em,i{font-style:italic;line-height:inherit}
strong,b{font-weight:bold;line-height:inherit}
small{font-size:60%;line-height:inherit}
code{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:400;color:rgba(0,0,0,.9)}
ul,ol,dl{line-height:1.6;margin-bottom:1.25em;list-style-position:outside;font-family:inherit}
ul,ol{margin-left:1.5em}
ul li ul,ul li ol{margin-left:1.25em;margin-bottom:0}
ul.circle{list-style-type:circle}
ul.disc{list-style-type:disc}
ul.square{list-style-type:square}
ul.circle ul:not([class]),ul.disc ul:not([class]),ul.square ul:not([class]){list-style:inherit}
ol li ul,ol li ol{margin-left:1.25em;margin-bottom:0}
dl dt{margin-bottom:.3125em;font-weight:bold}
dl dd{margin-bottom:1.25em}
blockquote{margin:0 0 1.25em;padding:.5625em 1.25em 0 1.1875em;border-left:1px solid #ddd}
blockquote,blockquote p{line-height:1.6;color:rgba(0,0,0,.85)}
@media screen and (min-width:768px){h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2}
h1{font-size:2.75em}
h2{font-size:2.3125em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.6875em}
h4{font-size:1.4375em}}
table{background:#fff;margin-bottom:1.25em;border:1px solid #dedede;word-wrap:normal}
table thead,table tfoot{background:#f7f8f7}
table thead tr th,table thead tr td,table tfoot tr th,table tfoot tr td{padding:.5em .625em .625em;font-size:inherit;color:rgba(0,0,0,.8);text-align:left}
table tr th,table tr td{padding:.5625em .625em;font-size:inherit;color:rgba(0,0,0,.8)}
table tr.even,table tr.alt{background:#f8f8f7}
table thead tr th,table tfoot tr th,table tbody tr td,table tr td,table tfoot tr td{line-height:1.6}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2;word-spacing:-.05em}
h1 strong,h2 strong,h3 strong,#toctitle strong,.sidebarblock>.content>.title strong,h4 strong,h5 strong,h6 strong{font-weight:400}
.center{margin-left:auto;margin-right:auto}
.stretch{width:100%}
.clearfix::before,.clearfix::after,.float-group::before,.float-group::after{content:" ";display:table}
.clearfix::after,.float-group::after{clear:both}
:not(pre).nobreak{word-wrap:normal}
:not(pre).nowrap{white-space:nowrap}
:not(pre).pre-wrap{white-space:pre-wrap}
:not(pre):not([class^=L])>code{font-size:.9375em;font-style:normal!important;letter-spacing:0;padding:.1em .5ex;word-spacing:-.15em;background:#f7f7f8;border-radius:4px;line-height:1.45;text-rendering:optimizeSpeed}
pre{color:rgba(0,0,0,.9);font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;line-height:1.45;text-rendering:optimizeSpeed}
pre code,pre pre{color:inherit;font-size:inherit;line-height:inherit}
pre>code{display:block}
pre.nowrap,pre.nowrap pre{white-space:pre;word-wrap:normal}
em em{font-style:normal}
strong strong{font-weight:400}
.keyseq{color:rgba(51,51,51,.8)}
kbd{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;display:inline-block;color:rgba(0,0,0,.8);font-size:.65em;line-height:1.45;background:#f7f7f7;border:1px solid #ccc;border-radius:3px;box-shadow:0 1px 0 rgba(0,0,0,.2),inset 0 0 0 .1em #fff;margin:0 .15em;padding:.2em .5em;vertical-align:middle;position:relative;top:-.1em;white-space:nowrap}
.keyseq kbd:first-child{margin-left:0}
.keyseq kbd:last-child{margin-right:0}
.menuseq,.menuref{color:#000}
.menuseq b:not(.caret),.menuref{font-weight:inherit}
.menuseq{word-spacing:-.02em}
.menuseq b.caret{font-size:1.25em;line-height:.8}
.menuseq i.caret{font-weight:bold;text-align:center;width:.45em}
b.button::before,b.button::after{position:relative;top:-1px;font-weight:400}
b.button::before{content:"[";padding:0 3px 0 2px}
b.button::after{content:"]";padding:0 2px 0 3px}
p a>code:hover{color:rgba(0,0,0,.9)}
#header,#content,#footnotes,#footer{width:100%;margin:0 auto;max-width:62.5em;*zoom:1;position:relative;padding-left:.9375em;padding-right:.9375em}
#header::before,#header::after,#content::before,#content::after,#footnotes::before,#footnotes::after,#footer::before,#footer::after{content:" ";display:table}
#header::after,#content::after,#footnotes::after,#footer::after{clear:both}
#content{margin-top:1.25em}
#content::before{content:none}
#header>h1:first-child{color:rgba(0,0,0,.85);margin-top:2.25rem;margin-bottom:0}
#header>h1:first-child+#toc{margin-top:8px;border-top:1px solid #dddddf}
#header>h1:only-child{border-bottom:1px solid #dddddf;padding-bottom:8px}
#header .details{border-bottom:1px solid #dddddf;line-height:1.45;padding-top:.25em;padding-bottom:.25em;padding-left:.25em;color:rgba(0,0,0,.6);display:flex;flex-flow:row wrap}
#header .details span:first-child{margin-left:-.125em}
#header .details span.email a{color:rgba(0,0,0,.85)}
#header .details br{display:none}
#header .details br+span::before{content:"\00a0\2013\00a0"}
#header .details br+span.author::before{content:"\00a0\22c5\00a0";color:rgba(0,0,0,.85)}
#header .details br+span#revremark::before{content:"\00a0|\00a0"}
#header #revnumber{text-transform:capitalize}
#header #revnumber::after{content:"\00a0"}
#content>h1:first-child:not([class]){color:rgba(0,0,0,.85);border-bottom:1px solid #dddddf;padding-bottom:8px;margin-top:0;padding-top:1rem;margin-bottom:1.25rem}
#toc{border-bottom:1px solid #e7e7e9;padding-bottom:.5em}
#toc>ul{margin-left:.125em}
#toc ul.sectlevel0>li>a{font-style:italic}
#toc ul.sectlevel0 ul.sectlevel1{margin:.5em 0}
#toc ul{font-family:"Open Sans","DejaVu Sans",sans-serif;list-style-type:none}
#toc li{line-height:1.3334;margin-top:.3334em}
#toc a{text-decoration:none}
#toc a:active{text-decoration:underline}
#toctitle{color:#7a2518;font-size:1.2em}
@media screen and (min-width:768px){#toctitle{font-size:1.375em}
body.toc2{padding-left:15em;padding-right:0}
body.toc2 #header>h1:nth-last-child(2){border-bottom:1px solid #dddddf;padding-bottom:8px}
#toc.toc2{margin-top:0!important;background:#f8f8f7;position:fixed;width:15em;left:0;top:0;border-right:1px solid #e7e7e9;border-top-width:0!important;border-bottom-width:0!important;z-index:1000;padding:1.25em 1em;height:100%;overflow:auto}
#toc.toc2 #toctitle{margin-top:0;margin-bottom:.8rem;font-size:1.2em}
#toc.toc2>ul{font-size:.9em;margin-bottom:0}
#toc.toc2 ul ul{margin-left:0;padding-left:1em}
#toc.toc2 ul.sectlevel0 ul.sectlevel1{padding-left:0;margin-top:.5em;margin-bottom:.5em}
body.toc2.toc-right{padding-left:0;padding-right:15em}
body.toc2.toc-right #toc.toc2{border-right-width:0;border-left:1px solid #e7e7e9;left:auto;right:0}}
@media screen and (min-width:1280px){body.toc2{padding-left:20em;padding-right:0}
#toc.toc2{width:20em}
#toc.toc2 #toctitle{font-size:1.375em}
#toc.toc2>ul{font-size:.95em}
#toc.toc2 ul ul{padding-left:1.25em}
body.toc2.toc-right{padding-left:0;padding-right:20em}}
#content #toc{border:1px solid #e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;border-radius:4px}
#content #toc>:first-child{margin-top:0}
#content #toc>:last-child{margin-bottom:0}
#footer{max-width:none;background:rgba(0,0,0,.8);padding:1.25em}
#footer-text{color:hsla(0,0%,100%,.8);line-height:1.44}
#content{margin-bottom:.625em}
.sect1{padding-bottom:.625em}
@media screen and (min-width:768px){#content{margin-bottom:1.25em}
.sect1{padding-bottom:1.25em}}
.sect1:last-child{padding-bottom:0}
.sect1+.sect1{border-top:1px solid #e7e7e9}
#content h1>a.anchor,h2>a.anchor,h3>a.anchor,#toctitle>a.anchor,.sidebarblock>.content>.title>a.anchor,h4>a.anchor,h5>a.anchor,h6>a.anchor{position:absolute;z-index:1001;width:1.5ex;margin-left:-1.5ex;display:block;text-decoration:none!important;visibility:hidden;text-align:center;font-weight:400}
#content h1>a.anchor::before,h2>a.anchor::before,h3>a.anchor::before,#toctitle>a.anchor::before,.sidebarblock>.content>.title>a.anchor::before,h4>a.anchor::before,h5>a.anchor::before,h6>a.anchor::before{content:"\00A7";font-size:.85em;display:block;padding-top:.1em}
#content h1:hover>a.anchor,#content h1>a.anchor:hover,h2:hover>a.anchor,h2>a.anchor:hover,h3:hover>a.anchor,#toctitle:hover>a.anchor,.sidebarblock>.content>.title:hover>a.anchor,h3>a.anchor:hover,#toctitle>a.anchor:hover,.sidebarblock>.content>.title>a.anchor:hover,h4:hover>a.anchor,h4>a.anchor:hover,h5:hover>a.anchor,h5>a.anchor:hover,h6:hover>a.anchor,h6>a.anchor:hover{visibility:visible}
#content h1>a.link,h2>a.link,h3>a.link,#toctitle>a.link,.sidebarblock>.content>.title>a.link,h4>a.link,h5>a.link,h6>a.link{color:#ba3925;text-decoration:none}
#content h1>a.link:hover,h2>a.link:hover,h3>a.link:hover,#toctitle>a.link:hover,.sidebarblock>.content>.title>a.link:hover,h4>a.link:hover,h5>a.link:hover,h6>a.link:hover{color:#a53221}
details,.audioblock,.imageblock,.literalblock,.listingblock,.stemblock,.videoblock{margin-bottom:1.25em}
details{margin-left:1.25rem}
details>summary{cursor:pointer;display:block;position:relative;line-height:1.6;margin-bottom:.625rem;outline:none;-webkit-tap-highlight-color:transparent}
details>summary::-webkit-details-marker{display:none}
details>summary::before{content:"";border:solid transparent;border-left:solid;border-width:.3em 0 .3em .5em;position:absolute;top:.5em;left:-1.25rem;transform:translateX(15%)}
details[open]>summary::before{border:solid transparent;border-top:solid;border-width:.5em .3em 0;transform:translateY(15%)}
details>summary::after{content:"";width:1.25rem;height:1em;position:absolute;top:.3em;left:-1.25rem}
.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{text-rendering:optimizeLegibility;text-align:left;font-family:"Noto Serif","DejaVu Serif",serif;font-size:1rem;font-style:italic}
table.tableblock.fit-content>caption.title{white-space:nowrap;width:0}
.paragraph.lead>p,#preamble>.sectionbody>[class=paragraph]:first-of-type p{font-size:1.21875em;line-height:1.6;color:rgba(0,0,0,.85)}
.admonitionblock>table{border-collapse:separate;border:0;background:none;width:100%}
.admonitionblock>table td.icon{text-align:center;width:80px}
.admonitionblock>table td.icon img{max-width:none}
.admonitionblock>table td.icon .title{font-weight:bold;font-family:"Open Sans","DejaVu Sans",sans-serif;text-transform:uppercase}
.admonitionblock>table td.content{padding-left:1.125em;padding-right:1.25em;border-left:1px solid #dddddf;color:rgba(0,0,0,.6);word-wrap:anywhere}
.admonitionblock>table td.content>:last-child>:last-child{margin-bottom:0}
.exampleblock>.content{border:1px solid #e6e6e6;margin-bottom:1.25em;padding:1.25em;background:#fff;border-radius:4px}
.sidebarblock{border:1px solid #dbdbd6;margin-bottom:1.25em;padding:1.25em;background:#f3f3f2;border-radius:4px}
.sidebarblock>.content>.title{color:#7a2518;margin-top:0;text-align:center}
.exampleblock>.content>:first-child,.sidebarblock>.content>:first-child{margin-top:0}
.exampleblock>.content>:last-child,.exampleblock>.content>:last-child>:last-child,.exampleblock>.content .olist>ol>li:last-child>:last-child,.exampleblock>.content .ulist>ul>li:last-child>:last-child,.exampleblock>.content .qlist>ol>li:last-child>:last-child,.sidebarblock>.content>:last-child,.sidebarblock>.content>:last-child>:last-child,.sidebarblock>.content .olist>ol>li:last-child>:last-child,.sidebarblock>.content .ulist>ul>li:last-child>:last-child,.sidebarblock>.content .qlist>ol>li:last-child>:last-child{margin-bottom:0}
.literalblock pre,.listingblock>.content>pre{border-radius:4px;overflow-x:auto;padding:1em;font-size:.8125em}
@media screen and (min-width:768px){.literalblock pre,.listingblock>.content>pre{font-size:.90625em}}
@media screen and (min-width:1280px){.literalblock pre,.listingblock>.content>pre{font-size:1em}}
.literalblock pre,.listingblock>.content>pre:not(.highlight),.listingblock>.content>pre[class=highlight],.listingblock>.content>pre[class^="highlight "]{background:#f7f7f8}
.literalblock.output pre{color:#f7f7f8;background:rgba(0,0,0,.9)}
.listingblock>.content{position:relative}
.listingblock code[data-lang]::before{display:none;content:attr(data-lang);position:absolute;font-size:.75em;top:.425rem;right:.5rem;line-height:1;text-transform:uppercase;color:inherit;opacity:.5}
.listingblock:hover code[data-lang]::before{display:block}
.listingblock.terminal pre .command::before{content:attr(data-prompt);padding-right:.5em;color:inherit;opacity:.5}
.listingblock.terminal pre .command:not([data-prompt])::before{content:"$"}
.listingblock pre.highlightjs{padding:0}
.listingblock pre.highlightjs>code{padding:1em;border-radius:4px}
.listingblock pre.prettyprint{border-width:0}
.prettyprint{background:#f7f7f8}
pre.prettyprint .linenums{line-height:1.45;margin-left:2em}
pre.prettyprint li{background:none;list-style-type:inherit;padding-left:0}
pre.prettyprint li code[data-lang]::before{opacity:1}
pre.prettyprint li:not(:first-child) code[data-lang]::before{display:none}
table.linenotable{border-collapse:separate;border:0;margin-bottom:0;background:none}
table.linenotable td[class]{color:inherit;vertical-align:top;padding:0;line-height:inherit;white-space:normal}
table.linenotable td.code{padding-left:.75em}
table.linenotable td.linenos,pre.pygments .linenos{border-right:1px solid;opacity:.35;padding-right:.5em;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}
pre.pygments span.linenos{display:inline-block;margin-right:.75em}
.quoteblock{margin:0 1em 1.25em 1.5em;display:table}
.quoteblock:not(.excerpt)>.title{margin-left:-1.5em;margin-bottom:.75em}
.quoteblock blockquote,.quoteblock p{color:rgba(0,0,0,.85);font-size:1.15rem;line-height:1.75;word-spacing:.1em;letter-spacing:0;font-style:italic;text-align:justify}
.quoteblock blockquote{margin:0;padding:0;border:0}
.quoteblock blockquote::before{content:"\201c";float:left;font-size:2.75em;font-weight:bold;line-height:.6em;margin-left:-.6em;color:#7a2518;text-shadow:0 1px 2px rgba(0,0,0,.1)}
.quoteblock blockquote>.paragraph:last-child p{margin-bottom:0}
.quoteblock .attribution{margin-top:.75em;margin-right:.5ex;text-align:right}
.verseblock{margin:0 1em 1.25em}
.verseblock pre{font-family:"Open Sans","DejaVu Sans",sans-serif;font-size:1.15rem;color:rgba(0,0,0,.85);font-weight:300;text-rendering:optimizeLegibility}
.verseblock pre strong{font-weight:400}
.verseblock .attribution{margin-top:1.25rem;margin-left:.5ex}
.quoteblock .attribution,.verseblock .attribution{font-size:.9375em;line-height:1.45;font-style:italic}
.quoteblock .attribution br,.verseblock .attribution br{display:none}
.quoteblock .attribution cite,.verseblock .attribution cite{display:block;letter-spacing:-.025em;color:rgba(0,0,0,.6)}
.quoteblock.abstract blockquote::before,.quoteblock.excerpt blockquote::before,.quoteblock .quoteblock blockquote::before{display:none}
.quoteblock.abstract blockquote,.quoteblock.abstract p,.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{line-height:1.6;word-spacing:0}
.quoteblock.abstract{margin:0 1em 1.25em;display:block}
.quoteblock.abstract>.title{margin:0 0 .375em;font-size:1.15em;text-align:center}
.quoteblock.excerpt>blockquote,.quoteblock .quoteblock{padding:0 0 .25em 1em;border-left:.25em solid #dddddf}
.quoteblock.excerpt,.quoteblock .quoteblock{margin-left:0}
.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{color:inherit;font-size:1.0625rem}
.quoteblock.excerpt .attribution,.quoteblock .quoteblock .attribution{color:inherit;font-size:.85rem;text-align:left;margin-right:0}
p.tableblock:last-child{margin-bottom:0}
td.tableblock>.content{margin-bottom:1.25em;word-wrap:anywhere}
td.tableblock>.content>:last-child{margin-bottom:-1.25em}
table.tableblock,th.tableblock,td.tableblock{border:0 solid #dedede}
table.grid-all>*>tr>*{border-width:1px}
table.grid-cols>*>tr>*{border-width:0 1px}
table.grid-rows>*>tr>*{border-width:1px 0}
table.frame-all{border-width:1px}
table.frame-ends{border-width:1px 0}
table.frame-sides{border-width:0 1px}
table.frame-none>colgroup+*>:first-child>*,table.frame-sides>colgroup+*>:first-child>*{border-top-width:0}
table.frame-none>:last-child>:last-child>*,table.frame-sides>:last-child>:last-child>*{border-bottom-width:0}
table.frame-none>*>tr>:first-child,table.frame-ends>*>tr>:first-child{border-left-width:0}
table.frame-none>*>tr>:last-child,table.frame-ends>*>tr>:last-child{border-right-width:0}
table.stripes-all>*>tr,table.stripes-odd>*>tr:nth-of-type(odd),table.stripes-even>*>tr:nth-of-type(even),table.stripes-hover>*>tr:hover{background:#f8f8f7}
th.halign-left,td.halign-left{text-align:left}
th.halign-right,td.halign-right{text-align:right}
th.halign-center,td.halign-center{text-align:center}
th.valign-top,td.valign-top{vertical-align:top}
th.valign-bottom,td.valign-bottom{vertical-align:bottom}
th.valign-middle,td.valign-middle{vertical-align:middle}
table thead th,table tfoot th{font-weight:bold}
tbody tr th{background:#f7f8f7}
tbody tr th,tbody tr th p,tfoot tr th,tfoot tr th p{color:rgba(0,0,0,.8);font-weight:bold}
p.tableblock>code:only-child{background:none;padding:0}
p.tableblock{font-size:1em}
ol{margin-left:1.75em}
ul li ol{margin-left:1.5em}
dl dd{margin-left:1.125em}
dl dd:last-child,dl dd:last-child>:last-child{margin-bottom:0}
li p,ul dd,ol dd,.olist .olist,.ulist .ulist,.ulist .olist,.olist .ulist{margin-bottom:.625em}
ul.checklist,ul.none,ol.none,ul.no-bullet,ol.no-bullet,ol.unnumbered,ul.unstyled,ol.unstyled{list-style-type:none}
ul.no-bullet,ol.no-bullet,ol.unnumbered{margin-left:.625em}
ul.unstyled,ol.unstyled{margin-left:0}
li>p:empty:only-child::before{content:"";display:inline-block}
ul.checklist>li>p:first-child{margin-left:-1em}
ul.checklist>li>p:first-child>.fa-square-o:first-child,ul.checklist>li>p:first-child>.fa-check-square-o:first-child{width:1.25em;font-size:.8em;position:relative;bottom:.125em}
ul.checklist>li>p:first-child>input[type=checkbox]:first-child{margin-right:.25em}
ul.inline{display:flex;flex-flow:row wrap;list-style:none;margin:0 0 .625em -1.25em}
ul.inline>li{margin-left:1.25em}
.unstyled dl dt{font-weight:400;font-style:normal}
ol.arabic{list-style-type:decimal}
ol.decimal{list-style-type:decimal-leading-zero}
ol.loweralpha{list-style-type:lower-alpha}
ol.upperalpha{list-style-type:upper-alpha}
ol.lowerroman{list-style-type:lower-roman}
ol.upperroman{list-style-type:upper-roman}
ol.lowergreek{list-style-type:lower-greek}
.hdlist>table,.colist>table{border:0;background:none}
.hdlist>table>tbody>tr,.colist>table>tbody>tr{background:none}
td.hdlist1,td.hdlist2{vertical-align:top;padding:0 .625em}
td.hdlist1{font-weight:bold;padding-bottom:1.25em}
td.hdlist2{word-wrap:anywhere}
.literalblock+.colist,.listingblock+.colist{margin-top:-.5em}
.colist td:not([class]):first-child{padding:.4em .75em 0;line-height:1;vertical-align:top}
.colist td:not([class]):first-child img{max-width:none}
.colist td:not([class]):last-child{padding:.25em 0}
.thumb,.th{line-height:0;display:inline-block;border:4px solid #fff;box-shadow:0 0 0 1px #ddd}
.imageblock.left{margin:.25em .625em 1.25em 0}
.imageblock.right{margin:.25em 0 1.25em .625em}
.imageblock>.title{margin-bottom:0}
.imageblock.thumb,.imageblock.th{border-width:6px}
.imageblock.thumb>.title,.imageblock.th>.title{padding:0 .125em}
.image.left,.image.right{margin-top:.25em;margin-bottom:.25em;display:inline-block;line-height:0}
.image.left{margin-right:.625em}
.image.right{margin-left:.625em}
a.image{text-decoration:none;display:inline-block}
a.image object{pointer-events:none}
sup.footnote,sup.footnoteref{font-size:.875em;position:static;vertical-align:super}
sup.footnote a,sup.footnoteref a{text-decoration:none}
sup.footnote a:active,sup.footnoteref a:active,#footnotes .footnote a:first-of-type:active{text-decoration:underline}
#footnotes{padding-top:.75em;padding-bottom:.75em;margin-bottom:.625em}
#footnotes hr{width:20%;min-width:6.25em;margin:-.25em 0 .75em;border-width:1px 0 0}
#footnotes .footnote{padding:0 .375em 0 .225em;line-height:1.3334;font-size:.875em;margin-left:1.2em;margin-bottom:.2em}
#footnotes .footnote a:first-of-type{font-weight:bold;text-decoration:none;margin-left:-1.05em}
#footnotes .footnote:last-of-type{margin-bottom:0}
#content #footnotes{margin-top:-.625em;margin-bottom:0;padding:.75em 0}
div.unbreakable{page-break-inside:avoid}
.big{font-size:larger}
.small{font-size:smaller}
.underline{text-decoration:underline}
.overline{text-decoration:overline}
.line-through{text-decoration:line-through}
.aqua{color:#00bfbf}
.aqua-background{background:#00fafa}
.black{color:#000}
.black-background{background:#000}
.blue{color:#0000bf}
.blue-background{background:#0000fa}
.fuchsia{color:#bf00bf}
.fuchsia-background{background:#fa00fa}
.gray{color:#606060}
.gray-background{background:#7d7d7d}
.green{color:#006000}
.green-background{background:#007d00}
.lime{color:#00bf00}
.lime-background{background:#00fa00}
.maroon{color:#600000}
.maroon-background{background:#7d0000}
.navy{color:#000060}
.navy-background{background:#00007d}
.olive{color:#606000}
.olive-background{background:#7d7d00}
.purple{color:#600060}
.purple-background{background:#7d007d}
.red{color:#bf0000}
.red-background{background:#fa0000}
.silver{color:#909090}
.silver-background{background:#bcbcbc}
.teal{color:#006060}
.teal-background{background:#007d7d}
.white{color:#bfbfbf}
.white-background{background:#fafafa}
.yellow{color:#bfbf00}
.yellow-background{background:#fafa00}
span.icon>.fa{cursor:default}
a span.icon>.fa{cursor:inherit}
.admonitionblock td.icon [class^="fa icon-"]{font-size:2.5em;text-shadow:1px 1px 2px rgba(0,0,0,.5);cursor:default}
.admonitionblock td.icon .icon-note::before{content:"\f05a";color:#19407c}
.admonitionblock td.icon .icon-tip::before{content:"\f0eb";text-shadow:1px 1px 2px rgba(155,155,0,.8);color:#111}
.admonitionblock td.icon .icon-warning::before{content:"\f071";color:#bf6900}
.admonitionblock td.icon .icon-caution::before{content:"\f06d";color:#bf3400}
.admonitionblock td.icon .icon-important::before{content:"\f06a";color:#bf0000}
.conum[data-value]{display:inline-block;color:#fff!important;background:rgba(0,0,0,.8);border-radius:50%;text-align:center;font-size:.75em;width:1.67em;height:1.67em;line-height:1.67em;font-family:"Open Sans","DejaVu Sans",sans-serif;font-style:normal;font-weight:bold}
.conum[data-value] *{color:#fff!important}
.conum[data-value]+b{display:none}
.conum[data-value]::after{content:attr(data-value)}
pre .conum[data-value]{position:relative;top:-.125em}
b.conum *{color:inherit!important}
.conum:not([data-value]):empty{display:none}
dt,th.tableblock,td.content,div.footnote{text-rendering:optimizeLegibility}
h1,h2,p,td.content,span.alt,summary{letter-spacing:-.01em}
p strong,td.content strong,div.footnote strong{letter-spacing:-.005em}
p,blockquote,dt,td.content,td.hdlist1,span.alt,summary{font-size:1.0625rem}
p{margin-bottom:1.25rem}
.sidebarblock p,.sidebarblock dt,.sidebarblock td.content,p.tableblock{font-size:1em}
.exampleblock>.content{background:#fffef7;border-color:#e0e0dc;box-shadow:0 1px 4px #e0e0dc}
.print-only{display:none!important}
@page{margin:1.25cm .75cm}
@media print{*{box-shadow:none!important;text-shadow:none!important}
html{font-size:80%}
a{color:inherit!important;text-decoration:underline!important}
a.bare,a[href^="#"],a[href^="mailto:"]{text-decoration:none!important}
a[href^="http:"]:not(.bare)::after,a[href^="https:"]:not(.bare)::after{content:"(" attr(href) ")";display:inline-block;font-size:.875em;padding-left:.25em}
abbr[title]{border-bottom:1px dotted}
abbr[title]::after{content:" (" attr(title) ")"}
pre,blockquote,tr,img,object,svg{page-break-inside:avoid}
thead{display:table-header-group}
svg{max-width:100%}
p,blockquote,dt,td.content{font-size:1em;orphans:3;widows:3}
h2,h3,#toctitle,.sidebarblock>.content>.title{page-break-after:avoid}
#header,#content,#footnotes,#footer{max-width:none}
#toc,.sidebarblock,.exampleblock>.content{background:none!important}
#toc{border-bottom:1px solid #dddddf!important;padding-bottom:0!important}
body.book #header{text-align:center}
body.book #header>h1:first-child{border:0!important;margin:2.5em 0 1em}
body.book #header .details{border:0!important;display:block;padding:0!important}
body.book #header .details span:first-child{margin-left:0!important}
body.book #header .details br{display:block}
body.book #header .details br+span::before{content:none!important}
body.book #toc{border:0!important;text-align:left!important;padding:0!important;margin:0!important}
body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-break-before:always}
.listingblock code[data-lang]::before{display:block}
#footer{padding:0 .9375em}
.hide-on-print{display:none!important}
.print-only{display:block!important}
.hide-for-print{display:none!important}
.show-for-print{display:inherit!important}}
@media amzn-kf8,print{#header>h1:first-child{margin-top:1.25rem}
.sect1{padding:0!important}
.sect1+.sect1{border:0}
#footer{background:none}
#footer-text{color:rgba(0,0,0,.6);font-size:.9em}}
@media amzn-kf8{#header,#content,#footnotes,#footer{padding:0}}
</style>
</head>
<body class="article">
<div id="header">
<h1>parse-options API</h1>
</div>
<div id="content">
<div id="preamble">
<div class="sectionbody">
<div class="paragraph">
<p>The parse-options API is used to parse and massage options in Git
and to provide a usage help with consistent look.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_basics">Basics</h2>
<div class="sectionbody">
<div class="paragraph">
<p>The argument vector <code>argv</code>[] may usually contain mandatory or optional
<em>non-option arguments</em>, e.g. a filename or a branch, <em>options</em>, and
<em>subcommands</em>.
Options are optional arguments that start with a dash and
that allow to change the behavior of a command.</p>
</div>
<div class="ulist">
<ul>
<li>
<p>There are basically three types of options:
<em>boolean</em> options,
options with (mandatory) <em>arguments</em> and
options with <em>optional arguments</em>
(i.e. a boolean option that can be adjusted).</p>
</li>
<li>
<p>There are basically two forms of options:
<em>Short options</em> consist of one dash (<code>-</code>) and one alphanumeric
character.
<em>Long options</em> begin with two dashes (<code>--</code>) and some
alphanumeric characters.</p>
</li>
<li>
<p>Options are case-sensitive.
Please define <em>lower-case long options</em> only.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>The parse-options API allows:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><em>stuck</em> and <em>separate form</em> of options with arguments.
<code>-oArg</code> is stuck, <code>-o</code> <code>Arg</code> is separate form.
<code>--option=Arg</code> is stuck, <code>--option</code> <code>Arg</code> is separate form.</p>
</li>
<li>
<p>Long options may be <em>abbreviated</em>, as long as the abbreviation
is unambiguous.</p>
</li>
<li>
<p>Short options may be bundled, e.g. <code>-a</code> <code>-b</code> can be specified as <code>-ab</code>.</p>
</li>
<li>
<p>Boolean long options can be <em>negated</em> (or <em>unset</em>) by prepending
<code>no-</code>, e.g. <code>--no-abbrev</code> instead of <code>--abbrev</code>. Conversely,
options that begin with <code>no-</code> can be <em>negated</em> by removing it.
Other long options can be unset (e.g., set string to NULL, set
integer to 0) by prepending <code>no-</code>.</p>
</li>
<li>
<p>Options and non-option arguments can clearly be separated using the <code>--</code>
option, e.g. <code>-a</code> <code>-b</code> <code>--option</code> <code>--</code> <code>--this-is-a-file</code> indicates that
<code>--this-is-a-file</code> must not be processed as an option.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Subcommands are special in a couple of ways:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Subcommands only have long form, and they have no double dash prefix, no
negated form, and no description, and they don&#8217;t take any arguments, and
can&#8217;t be abbreviated.</p>
</li>
<li>
<p>There must be exactly one subcommand among the arguments, or zero if the
command has a default operation mode.</p>
</li>
<li>
<p>All arguments following the subcommand are considered to be arguments of
the subcommand, and, conversely, arguments meant for the subcommand may
not precede the subcommand.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Therefore, if the options array contains at least one subcommand and
<code>parse_options</code>() encounters the first dashless argument, it will either:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>stop and return, if that dashless argument is a known subcommand, setting
<code>value</code> to the function pointer associated with that subcommand, storing
the name of the subcommand in argv[0], and leaving the rest of the
arguments unprocessed, or</p>
</li>
<li>
<p>stop and return, if it was invoked with the <code>PARSE_OPT_SUBCOMMAND_OPTIONAL</code>
flag and that dashless argument doesn&#8217;t match any subcommands, leaving
<code>value</code> unchanged and the rest of the arguments unprocessed, or</p>
</li>
<li>
<p>show error and usage, and abort.</p>
</li>
</ul>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_steps_to_parse_options">Steps to parse options</h2>
<div class="sectionbody">
<div class="olist arabic">
<ol class="arabic">
<li>
<p>#include "parse-options.<code>h</code>"</p>
</li>
<li>
<p>define a NULL-terminated
<code>static</code> <code>const</code> <code>char</code> * <code>const</code> <code>builtin_foo_usage</code>[] array
containing alternative usage strings</p>
</li>
<li>
<p>define <code>builtin_foo_options</code> array as described below
in section <em>Data Structure</em>.</p>
</li>
<li>
<p>in <code>cmd_foo</code>(<code>int</code> <code>argc,</code> <code>const</code> <code>char</code> **argv, <code>const</code> <code>char</code> *prefix)
call</p>
<div class="literalblock">
<div class="content">
<pre>argc = parse_options(argc, argv, prefix, builtin_foo_options, builtin_foo_usage, flags);</pre>
</div>
</div>
<div class="paragraph">
<p><code>parse_options</code>() will filter out the processed options of <code>argv</code>[] and leave the
non-option arguments in <code>argv</code>[].
<code>argc</code> is updated appropriately because of the assignment.</p>
</div>
<div class="paragraph">
<p>You can also pass NULL instead of a usage array as the fifth parameter of
parse_options(), to avoid displaying a help screen with usage info and
option list. This should only be done if necessary, e.g. to implement
a limited parser for only a subset of the options that needs to be run
before the full parser, which in turn shows the full help message.</p>
</div>
<div class="paragraph">
<p>Flags are the bitwise-or of:</p>
</div>
<div class="dlist">
<dl>
<dt class="hdlist1"><code>PARSE_OPT_KEEP_DASHDASH</code></dt>
<dd>
<p>Keep the <code>--</code> that usually separates options from
non-option arguments.</p>
</dd>
<dt class="hdlist1"><code>PARSE_OPT_STOP_AT_NON_OPTION</code></dt>
<dd>
<p>Usually the whole argument vector is massaged and reordered.
Using this flag, processing is stopped at the first non-option
argument.</p>
</dd>
<dt class="hdlist1"><code>PARSE_OPT_KEEP_ARGV0</code></dt>
<dd>
<p>Keep the first argument, which contains the program name. It&#8217;s
removed from argv[] by default.</p>
</dd>
<dt class="hdlist1"><code>PARSE_OPT_KEEP_UNKNOWN_OPT</code></dt>
<dd>
<p>Keep unknown options instead of erroring out. This doesn&#8217;t
work for all combinations of arguments as users might expect
it to do. E.g. if the first argument in <code>--unknown</code> <code>--known</code>
takes a value (which we can&#8217;t know), the second one is
mistakenly interpreted as a known option. Similarly, if
<code>PARSE_OPT_STOP_AT_NON_OPTION</code> is set, the second argument in
<code>--unknown</code> <code>value</code> will be mistakenly interpreted as a
non-option, not as a value belonging to the unknown option,
the parser early. That&#8217;s why parse_options() errors out if
both options are set.
Note that non-option arguments are always kept, even without
this flag.</p>
</dd>
<dt class="hdlist1"><code>PARSE_OPT_NO_INTERNAL_HELP</code></dt>
<dd>
<p>By default, parse_options() handles <code>-h</code>, <code>--help</code> and
<code>--help-all</code> internally, by showing a help screen. This option
turns it off and allows one to add custom handlers for these
options, or to just leave them unknown.</p>
</dd>
<dt class="hdlist1"><code>PARSE_OPT_SUBCOMMAND_OPTIONAL</code></dt>
<dd>
<p>Don&#8217;t error out when no subcommand is specified.</p>
</dd>
</dl>
</div>
</li>
</ol>
</div>
<div class="paragraph">
<p>Note that <code>PARSE_OPT_STOP_AT_NON_OPTION</code> is incompatible with subcommands;
while <code>PARSE_OPT_KEEP_DASHDASH</code> and <code>PARSE_OPT_KEEP_UNKNOWN_OPT</code> can only be
used with subcommands when combined with <code>PARSE_OPT_SUBCOMMAND_OPTIONAL</code>.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_data_structure">Data Structure</h2>
<div class="sectionbody">
<div class="paragraph">
<p>The main data structure is an array of the <code>option</code> struct,
say <code>static</code> <code>struct</code> <code>option</code> <code>builtin_add_options</code>[].
There are some macros to easily define options:</p>
</div>
<div class="dlist">
<dl>
<dt class="hdlist1"><code>OPT__ABBREV</code>(&amp;int_var)</dt>
<dd>
<p>Add <code>--abbrev</code>[<code>=</code><em>&lt;n&gt;</em>].</p>
</dd>
<dt class="hdlist1"><code>OPT__COLOR</code>(&amp;int_var, <code>description</code>)</dt>
<dd>
<p>Add <code>--color</code>[<code>=</code><em>&lt;when&gt;</em>] and <code>--no-color</code>.</p>
</dd>
<dt class="hdlist1"><code>OPT__DRY_RUN</code>(&amp;int_var, <code>description</code>)</dt>
<dd>
<p>Add <code>-n,</code> <code>--dry-run</code>.</p>
</dd>
<dt class="hdlist1"><code>OPT__FORCE</code>(&amp;int_var, <code>description</code>)</dt>
<dd>
<p>Add <code>-f,</code> <code>--force</code>.</p>
</dd>
<dt class="hdlist1"><code>OPT__QUIET</code>(&amp;int_var, <code>description</code>)</dt>
<dd>
<p>Add <code>-q,</code> <code>--quiet</code>.</p>
</dd>
<dt class="hdlist1"><code>OPT__VERBOSE</code>(&amp;int_var, <code>description</code>)</dt>
<dd>
<p>Add <code>-v,</code> <code>--verbose</code>.</p>
</dd>
<dt class="hdlist1"><code>OPT_GROUP</code>(<code>description</code>)</dt>
<dd>
<p>Start an option group. <code>description</code> is a short string that
describes the group or an empty string.
Start the description with an upper-case letter.</p>
</dd>
<dt class="hdlist1"><code>OPT_BOOL</code>(<code>short,</code> <code>long,</code> &amp;int_var, <code>description</code>)</dt>
<dd>
<p>Introduce a boolean option. <code>int_var</code> is set to one with
<code>--option</code> and set to zero with <code>--no-option</code>.</p>
</dd>
<dt class="hdlist1"><code>OPT_COUNTUP</code>(<code>short,</code> <code>long,</code> &amp;int_var, <code>description</code>)</dt>
<dd>
<p>Introduce a count-up option.
Each use of <code>--option</code> increments <code>int_var</code>, starting from zero
(even if initially negative), and <code>--no-option</code> resets it to
zero. To determine if <code>--option</code> or <code>--no-option</code> was encountered at
all, initialize <code>int_var</code> to a negative value, and if it is still
negative after parse_options(), then neither <code>--option</code> nor
<code>--no-option</code> was seen.</p>
</dd>
<dt class="hdlist1"><code>OPT_BIT</code>(<code>short,</code> <code>long,</code> &amp;int_var, <code>description,</code> <code>mask</code>)</dt>
<dd>
<p>Introduce a boolean option.
If used, <code>int_var</code> is bitwise-ored with <code>mask</code>.</p>
</dd>
<dt class="hdlist1"><code>OPT_NEGBIT</code>(<code>short,</code> <code>long,</code> &amp;int_var, <code>description,</code> <code>mask</code>)</dt>
<dd>
<p>Introduce a boolean option.
If used, <code>int_var</code> is bitwise-anded with the inverted <code>mask</code>.</p>
</dd>
<dt class="hdlist1"><code>OPT_SET_INT</code>(<code>short,</code> <code>long,</code> &amp;int_var, <code>description,</code> <code>integer</code>)</dt>
<dd>
<p>Introduce an integer option.
<code>int_var</code> is set to <code>integer</code> with <code>--option</code>, and
reset to zero with <code>--no-option</code>.</p>
</dd>
<dt class="hdlist1"><code>OPT_STRING</code>(<code>short,</code> <code>long,</code> &amp;str_var, <code>arg_str,</code> <code>description</code>)</dt>
<dd>
<p>Introduce an option with string argument.
The string argument is put into <code>str_var</code>.</p>
</dd>
<dt class="hdlist1"><code>OPT_STRING_LIST</code>(<code>short,</code> <code>long,</code> &amp;struct <code>string_list,</code> <code>arg_str,</code> <code>description</code>)</dt>
<dd>
<p>Introduce an option with string argument.
The string argument is stored as an element in <code>string_list</code>.
Use of <code>--no-option</code> will clear the list of preceding values.</p>
</dd>
<dt class="hdlist1"><code>OPT_INTEGER</code>(<code>short,</code> <code>long,</code> &amp;int_var, <code>description</code>)</dt>
<dd>
<p>Introduce an option with integer argument.
The integer is put into <code>int_var</code>.</p>
</dd>
<dt class="hdlist1"><code>OPT_MAGNITUDE</code>(<code>short,</code> <code>long,</code> &amp;unsigned_long_var, <code>description</code>)</dt>
<dd>
<p>Introduce an option with a size argument. The argument must be a
non-negative integer and may include a suffix of <em>k</em>, <em>m</em> or <em>g</em> to
scale the provided value by 1024, 1024^2 or 1024^3 respectively.
The scaled value is put into <code>unsigned_long_var</code>.</p>
</dd>
<dt class="hdlist1"><code>OPT_EXPIRY_DATE</code>(<code>short,</code> <code>long,</code> &amp;timestamp_t_var, <code>description</code>)</dt>
<dd>
<p>Introduce an option with expiry date argument, see <code>parse_expiry_date</code>().
The timestamp is put into <code>timestamp_t_var</code>.</p>
</dd>
<dt class="hdlist1"><code>OPT_CALLBACK</code>(<code>short,</code> <code>long,</code> &amp;var, <code>arg_str,</code> <code>description,</code> <code>func_ptr</code>)</dt>
<dd>
<p>Introduce an option with argument.
The argument will be fed into the function given by <code>func_ptr</code>
and the result will be put into <code>var</code>.
See <em>Option Callbacks</em> below for a more elaborate description.</p>
</dd>
<dt class="hdlist1"><code>OPT_FILENAME</code>(<code>short,</code> <code>long,</code> &amp;var, <code>description</code>)</dt>
<dd>
<p>Introduce an option with a filename argument.
The filename will be prefixed by passing the filename along with
the prefix argument of <code>parse_options</code>() to <code>prefix_filename</code>().</p>
</dd>
<dt class="hdlist1"><code>OPT_NUMBER_CALLBACK</code>(&amp;var, <code>description,</code> <code>func_ptr</code>)</dt>
<dd>
<p>Recognize numerical options like -123 and feed the integer as
if it was an argument to the function given by <code>func_ptr</code>.
The result will be put into <code>var</code>. There can be only one such
option definition. It cannot be negated and it takes no
arguments. Short options that happen to be digits take
precedence over it.</p>
</dd>
<dt class="hdlist1"><code>OPT_COLOR_FLAG</code>(<code>short,</code> <code>long,</code> &amp;int_var, <code>description</code>)</dt>
<dd>
<p>Introduce an option that takes an optional argument that can
have one of three values: "always", "never", or "auto". If the
argument is not given, it defaults to "always". The <code>--no-</code> form
works like <code>--long=never</code>; it cannot take an argument. If
"always", set <code>int_var</code> to 1; if "never", set <code>int_var</code> to 0; if
"auto", set <code>int_var</code> to 1 if stdout is a tty or a pager,
0 otherwise.</p>
</dd>
<dt class="hdlist1"><code>OPT_NOOP_NOARG</code>(<code>short,</code> <code>long</code>)</dt>
<dd>
<p>Introduce an option that has no effect and takes no arguments.
Use it to hide deprecated options that are still to be recognized
and ignored silently.</p>
</dd>
<dt class="hdlist1"><code>OPT_PASSTHRU</code>(<code>short,</code> <code>long,</code> &amp;char_var, <code>arg_str,</code> <code>description,</code> <code>flags</code>)</dt>
<dd>
<p>Introduce an option that will be reconstructed into a char* string,
which must be initialized to NULL. This is useful when you need to
pass the command-line option to another command. Any previous value
will be overwritten, so this should only be used for options where
the last one specified on the command line wins.</p>
</dd>
<dt class="hdlist1"><code>OPT_PASSTHRU_ARGV</code>(<code>short,</code> <code>long,</code> &amp;strvec_var, <code>arg_str,</code> <code>description,</code> <code>flags</code>)</dt>
<dd>
<p>Introduce an option where all instances of it on the command-line will
be reconstructed into a strvec. This is useful when you need to
pass the command-line option, which can be specified multiple times,
to another command.</p>
</dd>
<dt class="hdlist1"><code>OPT_CMDMODE</code>(<code>short,</code> <code>long,</code> &amp;int_var, <code>description,</code> <code>enum_val</code>)</dt>
<dd>
<p>Define an "operation mode" option, only one of which in the same
group of "operating mode" options that share the same <code>int_var</code>
can be given by the user. <code>int_var</code> is set to <code>enum_val</code> when the
option is used, but an error is reported if other "operating mode"
option has already set its value to the same <code>int_var</code>.
In new commands consider using subcommands instead.</p>
</dd>
<dt class="hdlist1"><code>OPT_SUBCOMMAND</code>(<code>long,</code> &amp;fn_ptr, <code>subcommand_fn</code>)</dt>
<dd>
<p>Define a subcommand. <code>subcommand_fn</code> is put into <code>fn_ptr</code> when
this subcommand is used.</p>
</dd>
</dl>
</div>
<div class="paragraph">
<p>The last element of the array must be <code>OPT_END</code>().</p>
</div>
<div class="paragraph">
<p>If not stated otherwise, interpret the arguments as follows:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>short</code> is a character for the short option
(e.g. 'e' for <code>-e</code>, use <code>0</code> to omit),</p>
</li>
<li>
<p><code>long</code> is a string for the long option
(e.g. "example" for <code>--example</code>, use <code>NULL</code> to omit),</p>
</li>
<li>
<p><code>int_var</code> is an integer variable,</p>
</li>
<li>
<p><code>str_var</code> is a string variable (<code>char</code> *),</p>
</li>
<li>
<p><code>arg_str</code> is the string that is shown as argument
(e.g. "branch" will result in <em>&lt;branch&gt;</em>).
If set to <code>NULL</code>, three dots (...) will be displayed.</p>
</li>
<li>
<p><code>description</code> is a short string to describe the effect of the option.
It shall begin with a lower-case letter and a full stop (.) shall be
omitted at the end.</p>
</li>
</ul>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_option_callbacks">Option Callbacks</h2>
<div class="sectionbody">
<div class="paragraph">
<p>The function must be defined in this form:</p>
</div>
<div class="literalblock">
<div class="content">
<pre>int func(const struct option *opt, const char *arg, int unset)</pre>
</div>
</div>
<div class="paragraph">
<p>The callback mechanism is as follows:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Inside <code>func</code>, the only interesting member of the structure
given by <code>opt</code> is the void pointer <code>opt-</code>&gt;<code>value</code>.
*opt-&gt;<code>value</code> will be the value that is saved into <code>var</code>, if you
use <code>OPT_CALLBACK</code>().
For example, do *(<code>unsigned</code> <code>long</code> *)<code>opt-</code>&gt;<code>value</code> <code>=</code> <code>42</code>; to get 42
into an <code>unsigned</code> <code>long</code> variable.</p>
</li>
<li>
<p>Return value <code>0</code> indicates success and non-zero return
value will invoke <code>usage_with_options</code>() and, thus, die.</p>
</li>
<li>
<p>If the user negates the option, <code>arg</code> is <code>NULL</code> and <code>unset</code> is 1.</p>
</li>
</ul>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_sophisticated_option_parsing">Sophisticated option parsing</h2>
<div class="sectionbody">
<div class="paragraph">
<p>If you need, for example, option callbacks with optional arguments
or without arguments at all, or if you need other special cases,
that are not handled by the macros above, you need to specify the
members of the <code>option</code> structure manually.</p>
</div>
<div class="paragraph">
<p>This is not covered in this document, but well documented
in <code>parse-options.h</code> itself.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_examples">Examples</h2>
<div class="sectionbody">
<div class="paragraph">
<p>See <code>test-parse-options.c</code> and
<code>builtin/add.c</code>,
<code>builtin/clone.c</code>,
<code>builtin/commit.c</code>,
<code>builtin/fetch.c</code>,
<code>builtin/fsck.c</code>,
<code>builtin/rm.c</code>
for real-world examples.</p>
</div>
</div>
</div>
</div>
<div id="footer">
<div id="footer-text">
Last updated 2025-03-17 07:09:38 UTC
</div>
</div>
</body>
</html>

View File

@@ -0,0 +1,73 @@
Path-Walk API
=============
The path-walk API is used to walk reachable objects, but to visit objects
in batches based on a common path they appear in, or by type.
For example, all reachable commits are visited in a group. All tags are
visited in a group. Then, all root trees are visited. At some point, all
blobs reachable via a path `my/dir/to/A` are visited. When there are
multiple paths possible to reach the same object, then only one of those
paths is used to visit the object.
Basics
------
To use the path-walk API, include `path-walk.h` and call
`walk_objects_by_path()` with a customized `path_walk_info` struct. The
struct is used to set all of the options for how the walk should proceed.
Let's dig into the different options and their use.
`path_fn` and `path_fn_data`::
The most important option is the `path_fn` option, which is a
function pointer to the callback that can execute logic on the
object IDs for objects grouped by type and path. This function
also receives a `data` value that corresponds to the
`path_fn_data` member, for providing custom data structures to
this callback function.
`revs`::
To configure the exact details of the reachable set of objects,
use the `revs` member and initialize it using the revision
machinery in `revision.h`. Initialize `revs` using calls such as
`setup_revisions()` or `parse_revision_opt()`. Do not call
`prepare_revision_walk()`, as that will be called within
`walk_objects_by_path()`.
+
It is also important that you do not specify the `--objects` flag for the
`revs` struct. The revision walk should only be used to walk commits, and
the objects will be walked in a separate way based on those starting
commits.
`commits`, `blobs`, `trees`, `tags`::
By default, these members are enabled and signal that the path-walk
API should call the `path_fn` on objects of these types. Specialized
applications could disable some options to make it simpler to walk
the objects or to have fewer calls to `path_fn`.
+
While it is possible to walk only commits in this way, consumers would be
better off using the revision walk API instead.
`prune_all_uninteresting`::
By default, all reachable paths are emitted by the path-walk API.
This option allows consumers to declare that they are not
interested in paths where all included objects are marked with the
`UNINTERESTING` flag. This requires using the `boundary` option in
the revision walk so that the walk emits commits marked with the
`UNINTERESTING` flag.
`pl`::
This pattern list pointer allows focusing the path-walk search to
a set of patterns, only emitting paths that match the given
patterns. See linkgit:gitignore[5] or
linkgit:git-sparse-checkout[1] for details about pattern lists.
When the pattern list uses cone-mode patterns, then the path-walk
API can prune the set of paths it walks to improve performance.
Examples
--------
See example usages in:
`t/helper/test-path-walk.c`,
`builtin/backfill.c`
`builtin/pack-objects.c`

View File

@@ -0,0 +1,543 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="en">
<head>
<meta charset="UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=edge"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<meta name="generator" content="Asciidoctor 2.0.23"/>
<title>Path-Walk API</title>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"/>
<style>
/*! Asciidoctor default stylesheet | MIT License | https://asciidoctor.org */
/* Uncomment the following line when using as a custom stylesheet */
/* @import "https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"; */
html{font-family:sans-serif;-webkit-text-size-adjust:100%}
a{background:none}
a:focus{outline:thin dotted}
a:active,a:hover{outline:0}
h1{font-size:2em;margin:.67em 0}
b,strong{font-weight:bold}
abbr{font-size:.9em}
abbr[title]{cursor:help;border-bottom:1px dotted #dddddf;text-decoration:none}
dfn{font-style:italic}
hr{height:0}
mark{background:#ff0;color:#000}
code,kbd,pre,samp{font-family:monospace;font-size:1em}
pre{white-space:pre-wrap}
q{quotes:"\201C" "\201D" "\2018" "\2019"}
small{font-size:80%}
sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline}
sup{top:-.5em}
sub{bottom:-.25em}
img{border:0}
svg:not(:root){overflow:hidden}
figure{margin:0}
audio,video{display:inline-block}
audio:not([controls]){display:none;height:0}
fieldset{border:1px solid silver;margin:0 2px;padding:.35em .625em .75em}
legend{border:0;padding:0}
button,input,select,textarea{font-family:inherit;font-size:100%;margin:0}
button,input{line-height:normal}
button,select{text-transform:none}
button,html input[type=button],input[type=reset],input[type=submit]{-webkit-appearance:button;cursor:pointer}
button[disabled],html input[disabled]{cursor:default}
input[type=checkbox],input[type=radio]{padding:0}
button::-moz-focus-inner,input::-moz-focus-inner{border:0;padding:0}
textarea{overflow:auto;vertical-align:top}
table{border-collapse:collapse;border-spacing:0}
*,::before,::after{box-sizing:border-box}
html,body{font-size:100%}
body{background:#fff;color:rgba(0,0,0,.8);padding:0;margin:0;font-family:"Noto Serif","DejaVu Serif",serif;line-height:1;position:relative;cursor:auto;-moz-tab-size:4;-o-tab-size:4;tab-size:4;word-wrap:anywhere;-moz-osx-font-smoothing:grayscale;-webkit-font-smoothing:antialiased}
a:hover{cursor:pointer}
img,object,embed{max-width:100%;height:auto}
object,embed{height:100%}
img{-ms-interpolation-mode:bicubic}
.left{float:left!important}
.right{float:right!important}
.text-left{text-align:left!important}
.text-right{text-align:right!important}
.text-center{text-align:center!important}
.text-justify{text-align:justify!important}
.hide{display:none}
img,object,svg{display:inline-block;vertical-align:middle}
textarea{height:auto;min-height:50px}
select{width:100%}
.subheader,.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{line-height:1.45;color:#7a2518;font-weight:400;margin-top:0;margin-bottom:.25em}
div,dl,dt,dd,ul,ol,li,h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6,pre,form,p,blockquote,th,td{margin:0;padding:0}
a{color:#2156a5;text-decoration:underline;line-height:inherit}
a:hover,a:focus{color:#1d4b8f}
a img{border:0}
p{line-height:1.6;margin-bottom:1.25em;text-rendering:optimizeLegibility}
p aside{font-size:.875em;line-height:1.35;font-style:italic}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{font-family:"Open Sans","DejaVu Sans",sans-serif;font-weight:300;font-style:normal;color:#ba3925;text-rendering:optimizeLegibility;margin-top:1em;margin-bottom:.5em;line-height:1.0125em}
h1 small,h2 small,h3 small,#toctitle small,.sidebarblock>.content>.title small,h4 small,h5 small,h6 small{font-size:60%;color:#e99b8f;line-height:0}
h1{font-size:2.125em}
h2{font-size:1.6875em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.375em}
h4,h5{font-size:1.125em}
h6{font-size:1em}
hr{border:solid #dddddf;border-width:1px 0 0;clear:both;margin:1.25em 0 1.1875em}
em,i{font-style:italic;line-height:inherit}
strong,b{font-weight:bold;line-height:inherit}
small{font-size:60%;line-height:inherit}
code{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:400;color:rgba(0,0,0,.9)}
ul,ol,dl{line-height:1.6;margin-bottom:1.25em;list-style-position:outside;font-family:inherit}
ul,ol{margin-left:1.5em}
ul li ul,ul li ol{margin-left:1.25em;margin-bottom:0}
ul.circle{list-style-type:circle}
ul.disc{list-style-type:disc}
ul.square{list-style-type:square}
ul.circle ul:not([class]),ul.disc ul:not([class]),ul.square ul:not([class]){list-style:inherit}
ol li ul,ol li ol{margin-left:1.25em;margin-bottom:0}
dl dt{margin-bottom:.3125em;font-weight:bold}
dl dd{margin-bottom:1.25em}
blockquote{margin:0 0 1.25em;padding:.5625em 1.25em 0 1.1875em;border-left:1px solid #ddd}
blockquote,blockquote p{line-height:1.6;color:rgba(0,0,0,.85)}
@media screen and (min-width:768px){h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2}
h1{font-size:2.75em}
h2{font-size:2.3125em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.6875em}
h4{font-size:1.4375em}}
table{background:#fff;margin-bottom:1.25em;border:1px solid #dedede;word-wrap:normal}
table thead,table tfoot{background:#f7f8f7}
table thead tr th,table thead tr td,table tfoot tr th,table tfoot tr td{padding:.5em .625em .625em;font-size:inherit;color:rgba(0,0,0,.8);text-align:left}
table tr th,table tr td{padding:.5625em .625em;font-size:inherit;color:rgba(0,0,0,.8)}
table tr.even,table tr.alt{background:#f8f8f7}
table thead tr th,table tfoot tr th,table tbody tr td,table tr td,table tfoot tr td{line-height:1.6}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2;word-spacing:-.05em}
h1 strong,h2 strong,h3 strong,#toctitle strong,.sidebarblock>.content>.title strong,h4 strong,h5 strong,h6 strong{font-weight:400}
.center{margin-left:auto;margin-right:auto}
.stretch{width:100%}
.clearfix::before,.clearfix::after,.float-group::before,.float-group::after{content:" ";display:table}
.clearfix::after,.float-group::after{clear:both}
:not(pre).nobreak{word-wrap:normal}
:not(pre).nowrap{white-space:nowrap}
:not(pre).pre-wrap{white-space:pre-wrap}
:not(pre):not([class^=L])>code{font-size:.9375em;font-style:normal!important;letter-spacing:0;padding:.1em .5ex;word-spacing:-.15em;background:#f7f7f8;border-radius:4px;line-height:1.45;text-rendering:optimizeSpeed}
pre{color:rgba(0,0,0,.9);font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;line-height:1.45;text-rendering:optimizeSpeed}
pre code,pre pre{color:inherit;font-size:inherit;line-height:inherit}
pre>code{display:block}
pre.nowrap,pre.nowrap pre{white-space:pre;word-wrap:normal}
em em{font-style:normal}
strong strong{font-weight:400}
.keyseq{color:rgba(51,51,51,.8)}
kbd{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;display:inline-block;color:rgba(0,0,0,.8);font-size:.65em;line-height:1.45;background:#f7f7f7;border:1px solid #ccc;border-radius:3px;box-shadow:0 1px 0 rgba(0,0,0,.2),inset 0 0 0 .1em #fff;margin:0 .15em;padding:.2em .5em;vertical-align:middle;position:relative;top:-.1em;white-space:nowrap}
.keyseq kbd:first-child{margin-left:0}
.keyseq kbd:last-child{margin-right:0}
.menuseq,.menuref{color:#000}
.menuseq b:not(.caret),.menuref{font-weight:inherit}
.menuseq{word-spacing:-.02em}
.menuseq b.caret{font-size:1.25em;line-height:.8}
.menuseq i.caret{font-weight:bold;text-align:center;width:.45em}
b.button::before,b.button::after{position:relative;top:-1px;font-weight:400}
b.button::before{content:"[";padding:0 3px 0 2px}
b.button::after{content:"]";padding:0 2px 0 3px}
p a>code:hover{color:rgba(0,0,0,.9)}
#header,#content,#footnotes,#footer{width:100%;margin:0 auto;max-width:62.5em;*zoom:1;position:relative;padding-left:.9375em;padding-right:.9375em}
#header::before,#header::after,#content::before,#content::after,#footnotes::before,#footnotes::after,#footer::before,#footer::after{content:" ";display:table}
#header::after,#content::after,#footnotes::after,#footer::after{clear:both}
#content{margin-top:1.25em}
#content::before{content:none}
#header>h1:first-child{color:rgba(0,0,0,.85);margin-top:2.25rem;margin-bottom:0}
#header>h1:first-child+#toc{margin-top:8px;border-top:1px solid #dddddf}
#header>h1:only-child{border-bottom:1px solid #dddddf;padding-bottom:8px}
#header .details{border-bottom:1px solid #dddddf;line-height:1.45;padding-top:.25em;padding-bottom:.25em;padding-left:.25em;color:rgba(0,0,0,.6);display:flex;flex-flow:row wrap}
#header .details span:first-child{margin-left:-.125em}
#header .details span.email a{color:rgba(0,0,0,.85)}
#header .details br{display:none}
#header .details br+span::before{content:"\00a0\2013\00a0"}
#header .details br+span.author::before{content:"\00a0\22c5\00a0";color:rgba(0,0,0,.85)}
#header .details br+span#revremark::before{content:"\00a0|\00a0"}
#header #revnumber{text-transform:capitalize}
#header #revnumber::after{content:"\00a0"}
#content>h1:first-child:not([class]){color:rgba(0,0,0,.85);border-bottom:1px solid #dddddf;padding-bottom:8px;margin-top:0;padding-top:1rem;margin-bottom:1.25rem}
#toc{border-bottom:1px solid #e7e7e9;padding-bottom:.5em}
#toc>ul{margin-left:.125em}
#toc ul.sectlevel0>li>a{font-style:italic}
#toc ul.sectlevel0 ul.sectlevel1{margin:.5em 0}
#toc ul{font-family:"Open Sans","DejaVu Sans",sans-serif;list-style-type:none}
#toc li{line-height:1.3334;margin-top:.3334em}
#toc a{text-decoration:none}
#toc a:active{text-decoration:underline}
#toctitle{color:#7a2518;font-size:1.2em}
@media screen and (min-width:768px){#toctitle{font-size:1.375em}
body.toc2{padding-left:15em;padding-right:0}
body.toc2 #header>h1:nth-last-child(2){border-bottom:1px solid #dddddf;padding-bottom:8px}
#toc.toc2{margin-top:0!important;background:#f8f8f7;position:fixed;width:15em;left:0;top:0;border-right:1px solid #e7e7e9;border-top-width:0!important;border-bottom-width:0!important;z-index:1000;padding:1.25em 1em;height:100%;overflow:auto}
#toc.toc2 #toctitle{margin-top:0;margin-bottom:.8rem;font-size:1.2em}
#toc.toc2>ul{font-size:.9em;margin-bottom:0}
#toc.toc2 ul ul{margin-left:0;padding-left:1em}
#toc.toc2 ul.sectlevel0 ul.sectlevel1{padding-left:0;margin-top:.5em;margin-bottom:.5em}
body.toc2.toc-right{padding-left:0;padding-right:15em}
body.toc2.toc-right #toc.toc2{border-right-width:0;border-left:1px solid #e7e7e9;left:auto;right:0}}
@media screen and (min-width:1280px){body.toc2{padding-left:20em;padding-right:0}
#toc.toc2{width:20em}
#toc.toc2 #toctitle{font-size:1.375em}
#toc.toc2>ul{font-size:.95em}
#toc.toc2 ul ul{padding-left:1.25em}
body.toc2.toc-right{padding-left:0;padding-right:20em}}
#content #toc{border:1px solid #e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;border-radius:4px}
#content #toc>:first-child{margin-top:0}
#content #toc>:last-child{margin-bottom:0}
#footer{max-width:none;background:rgba(0,0,0,.8);padding:1.25em}
#footer-text{color:hsla(0,0%,100%,.8);line-height:1.44}
#content{margin-bottom:.625em}
.sect1{padding-bottom:.625em}
@media screen and (min-width:768px){#content{margin-bottom:1.25em}
.sect1{padding-bottom:1.25em}}
.sect1:last-child{padding-bottom:0}
.sect1+.sect1{border-top:1px solid #e7e7e9}
#content h1>a.anchor,h2>a.anchor,h3>a.anchor,#toctitle>a.anchor,.sidebarblock>.content>.title>a.anchor,h4>a.anchor,h5>a.anchor,h6>a.anchor{position:absolute;z-index:1001;width:1.5ex;margin-left:-1.5ex;display:block;text-decoration:none!important;visibility:hidden;text-align:center;font-weight:400}
#content h1>a.anchor::before,h2>a.anchor::before,h3>a.anchor::before,#toctitle>a.anchor::before,.sidebarblock>.content>.title>a.anchor::before,h4>a.anchor::before,h5>a.anchor::before,h6>a.anchor::before{content:"\00A7";font-size:.85em;display:block;padding-top:.1em}
#content h1:hover>a.anchor,#content h1>a.anchor:hover,h2:hover>a.anchor,h2>a.anchor:hover,h3:hover>a.anchor,#toctitle:hover>a.anchor,.sidebarblock>.content>.title:hover>a.anchor,h3>a.anchor:hover,#toctitle>a.anchor:hover,.sidebarblock>.content>.title>a.anchor:hover,h4:hover>a.anchor,h4>a.anchor:hover,h5:hover>a.anchor,h5>a.anchor:hover,h6:hover>a.anchor,h6>a.anchor:hover{visibility:visible}
#content h1>a.link,h2>a.link,h3>a.link,#toctitle>a.link,.sidebarblock>.content>.title>a.link,h4>a.link,h5>a.link,h6>a.link{color:#ba3925;text-decoration:none}
#content h1>a.link:hover,h2>a.link:hover,h3>a.link:hover,#toctitle>a.link:hover,.sidebarblock>.content>.title>a.link:hover,h4>a.link:hover,h5>a.link:hover,h6>a.link:hover{color:#a53221}
details,.audioblock,.imageblock,.literalblock,.listingblock,.stemblock,.videoblock{margin-bottom:1.25em}
details{margin-left:1.25rem}
details>summary{cursor:pointer;display:block;position:relative;line-height:1.6;margin-bottom:.625rem;outline:none;-webkit-tap-highlight-color:transparent}
details>summary::-webkit-details-marker{display:none}
details>summary::before{content:"";border:solid transparent;border-left:solid;border-width:.3em 0 .3em .5em;position:absolute;top:.5em;left:-1.25rem;transform:translateX(15%)}
details[open]>summary::before{border:solid transparent;border-top:solid;border-width:.5em .3em 0;transform:translateY(15%)}
details>summary::after{content:"";width:1.25rem;height:1em;position:absolute;top:.3em;left:-1.25rem}
.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{text-rendering:optimizeLegibility;text-align:left;font-family:"Noto Serif","DejaVu Serif",serif;font-size:1rem;font-style:italic}
table.tableblock.fit-content>caption.title{white-space:nowrap;width:0}
.paragraph.lead>p,#preamble>.sectionbody>[class=paragraph]:first-of-type p{font-size:1.21875em;line-height:1.6;color:rgba(0,0,0,.85)}
.admonitionblock>table{border-collapse:separate;border:0;background:none;width:100%}
.admonitionblock>table td.icon{text-align:center;width:80px}
.admonitionblock>table td.icon img{max-width:none}
.admonitionblock>table td.icon .title{font-weight:bold;font-family:"Open Sans","DejaVu Sans",sans-serif;text-transform:uppercase}
.admonitionblock>table td.content{padding-left:1.125em;padding-right:1.25em;border-left:1px solid #dddddf;color:rgba(0,0,0,.6);word-wrap:anywhere}
.admonitionblock>table td.content>:last-child>:last-child{margin-bottom:0}
.exampleblock>.content{border:1px solid #e6e6e6;margin-bottom:1.25em;padding:1.25em;background:#fff;border-radius:4px}
.sidebarblock{border:1px solid #dbdbd6;margin-bottom:1.25em;padding:1.25em;background:#f3f3f2;border-radius:4px}
.sidebarblock>.content>.title{color:#7a2518;margin-top:0;text-align:center}
.exampleblock>.content>:first-child,.sidebarblock>.content>:first-child{margin-top:0}
.exampleblock>.content>:last-child,.exampleblock>.content>:last-child>:last-child,.exampleblock>.content .olist>ol>li:last-child>:last-child,.exampleblock>.content .ulist>ul>li:last-child>:last-child,.exampleblock>.content .qlist>ol>li:last-child>:last-child,.sidebarblock>.content>:last-child,.sidebarblock>.content>:last-child>:last-child,.sidebarblock>.content .olist>ol>li:last-child>:last-child,.sidebarblock>.content .ulist>ul>li:last-child>:last-child,.sidebarblock>.content .qlist>ol>li:last-child>:last-child{margin-bottom:0}
.literalblock pre,.listingblock>.content>pre{border-radius:4px;overflow-x:auto;padding:1em;font-size:.8125em}
@media screen and (min-width:768px){.literalblock pre,.listingblock>.content>pre{font-size:.90625em}}
@media screen and (min-width:1280px){.literalblock pre,.listingblock>.content>pre{font-size:1em}}
.literalblock pre,.listingblock>.content>pre:not(.highlight),.listingblock>.content>pre[class=highlight],.listingblock>.content>pre[class^="highlight "]{background:#f7f7f8}
.literalblock.output pre{color:#f7f7f8;background:rgba(0,0,0,.9)}
.listingblock>.content{position:relative}
.listingblock code[data-lang]::before{display:none;content:attr(data-lang);position:absolute;font-size:.75em;top:.425rem;right:.5rem;line-height:1;text-transform:uppercase;color:inherit;opacity:.5}
.listingblock:hover code[data-lang]::before{display:block}
.listingblock.terminal pre .command::before{content:attr(data-prompt);padding-right:.5em;color:inherit;opacity:.5}
.listingblock.terminal pre .command:not([data-prompt])::before{content:"$"}
.listingblock pre.highlightjs{padding:0}
.listingblock pre.highlightjs>code{padding:1em;border-radius:4px}
.listingblock pre.prettyprint{border-width:0}
.prettyprint{background:#f7f7f8}
pre.prettyprint .linenums{line-height:1.45;margin-left:2em}
pre.prettyprint li{background:none;list-style-type:inherit;padding-left:0}
pre.prettyprint li code[data-lang]::before{opacity:1}
pre.prettyprint li:not(:first-child) code[data-lang]::before{display:none}
table.linenotable{border-collapse:separate;border:0;margin-bottom:0;background:none}
table.linenotable td[class]{color:inherit;vertical-align:top;padding:0;line-height:inherit;white-space:normal}
table.linenotable td.code{padding-left:.75em}
table.linenotable td.linenos,pre.pygments .linenos{border-right:1px solid;opacity:.35;padding-right:.5em;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}
pre.pygments span.linenos{display:inline-block;margin-right:.75em}
.quoteblock{margin:0 1em 1.25em 1.5em;display:table}
.quoteblock:not(.excerpt)>.title{margin-left:-1.5em;margin-bottom:.75em}
.quoteblock blockquote,.quoteblock p{color:rgba(0,0,0,.85);font-size:1.15rem;line-height:1.75;word-spacing:.1em;letter-spacing:0;font-style:italic;text-align:justify}
.quoteblock blockquote{margin:0;padding:0;border:0}
.quoteblock blockquote::before{content:"\201c";float:left;font-size:2.75em;font-weight:bold;line-height:.6em;margin-left:-.6em;color:#7a2518;text-shadow:0 1px 2px rgba(0,0,0,.1)}
.quoteblock blockquote>.paragraph:last-child p{margin-bottom:0}
.quoteblock .attribution{margin-top:.75em;margin-right:.5ex;text-align:right}
.verseblock{margin:0 1em 1.25em}
.verseblock pre{font-family:"Open Sans","DejaVu Sans",sans-serif;font-size:1.15rem;color:rgba(0,0,0,.85);font-weight:300;text-rendering:optimizeLegibility}
.verseblock pre strong{font-weight:400}
.verseblock .attribution{margin-top:1.25rem;margin-left:.5ex}
.quoteblock .attribution,.verseblock .attribution{font-size:.9375em;line-height:1.45;font-style:italic}
.quoteblock .attribution br,.verseblock .attribution br{display:none}
.quoteblock .attribution cite,.verseblock .attribution cite{display:block;letter-spacing:-.025em;color:rgba(0,0,0,.6)}
.quoteblock.abstract blockquote::before,.quoteblock.excerpt blockquote::before,.quoteblock .quoteblock blockquote::before{display:none}
.quoteblock.abstract blockquote,.quoteblock.abstract p,.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{line-height:1.6;word-spacing:0}
.quoteblock.abstract{margin:0 1em 1.25em;display:block}
.quoteblock.abstract>.title{margin:0 0 .375em;font-size:1.15em;text-align:center}
.quoteblock.excerpt>blockquote,.quoteblock .quoteblock{padding:0 0 .25em 1em;border-left:.25em solid #dddddf}
.quoteblock.excerpt,.quoteblock .quoteblock{margin-left:0}
.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{color:inherit;font-size:1.0625rem}
.quoteblock.excerpt .attribution,.quoteblock .quoteblock .attribution{color:inherit;font-size:.85rem;text-align:left;margin-right:0}
p.tableblock:last-child{margin-bottom:0}
td.tableblock>.content{margin-bottom:1.25em;word-wrap:anywhere}
td.tableblock>.content>:last-child{margin-bottom:-1.25em}
table.tableblock,th.tableblock,td.tableblock{border:0 solid #dedede}
table.grid-all>*>tr>*{border-width:1px}
table.grid-cols>*>tr>*{border-width:0 1px}
table.grid-rows>*>tr>*{border-width:1px 0}
table.frame-all{border-width:1px}
table.frame-ends{border-width:1px 0}
table.frame-sides{border-width:0 1px}
table.frame-none>colgroup+*>:first-child>*,table.frame-sides>colgroup+*>:first-child>*{border-top-width:0}
table.frame-none>:last-child>:last-child>*,table.frame-sides>:last-child>:last-child>*{border-bottom-width:0}
table.frame-none>*>tr>:first-child,table.frame-ends>*>tr>:first-child{border-left-width:0}
table.frame-none>*>tr>:last-child,table.frame-ends>*>tr>:last-child{border-right-width:0}
table.stripes-all>*>tr,table.stripes-odd>*>tr:nth-of-type(odd),table.stripes-even>*>tr:nth-of-type(even),table.stripes-hover>*>tr:hover{background:#f8f8f7}
th.halign-left,td.halign-left{text-align:left}
th.halign-right,td.halign-right{text-align:right}
th.halign-center,td.halign-center{text-align:center}
th.valign-top,td.valign-top{vertical-align:top}
th.valign-bottom,td.valign-bottom{vertical-align:bottom}
th.valign-middle,td.valign-middle{vertical-align:middle}
table thead th,table tfoot th{font-weight:bold}
tbody tr th{background:#f7f8f7}
tbody tr th,tbody tr th p,tfoot tr th,tfoot tr th p{color:rgba(0,0,0,.8);font-weight:bold}
p.tableblock>code:only-child{background:none;padding:0}
p.tableblock{font-size:1em}
ol{margin-left:1.75em}
ul li ol{margin-left:1.5em}
dl dd{margin-left:1.125em}
dl dd:last-child,dl dd:last-child>:last-child{margin-bottom:0}
li p,ul dd,ol dd,.olist .olist,.ulist .ulist,.ulist .olist,.olist .ulist{margin-bottom:.625em}
ul.checklist,ul.none,ol.none,ul.no-bullet,ol.no-bullet,ol.unnumbered,ul.unstyled,ol.unstyled{list-style-type:none}
ul.no-bullet,ol.no-bullet,ol.unnumbered{margin-left:.625em}
ul.unstyled,ol.unstyled{margin-left:0}
li>p:empty:only-child::before{content:"";display:inline-block}
ul.checklist>li>p:first-child{margin-left:-1em}
ul.checklist>li>p:first-child>.fa-square-o:first-child,ul.checklist>li>p:first-child>.fa-check-square-o:first-child{width:1.25em;font-size:.8em;position:relative;bottom:.125em}
ul.checklist>li>p:first-child>input[type=checkbox]:first-child{margin-right:.25em}
ul.inline{display:flex;flex-flow:row wrap;list-style:none;margin:0 0 .625em -1.25em}
ul.inline>li{margin-left:1.25em}
.unstyled dl dt{font-weight:400;font-style:normal}
ol.arabic{list-style-type:decimal}
ol.decimal{list-style-type:decimal-leading-zero}
ol.loweralpha{list-style-type:lower-alpha}
ol.upperalpha{list-style-type:upper-alpha}
ol.lowerroman{list-style-type:lower-roman}
ol.upperroman{list-style-type:upper-roman}
ol.lowergreek{list-style-type:lower-greek}
.hdlist>table,.colist>table{border:0;background:none}
.hdlist>table>tbody>tr,.colist>table>tbody>tr{background:none}
td.hdlist1,td.hdlist2{vertical-align:top;padding:0 .625em}
td.hdlist1{font-weight:bold;padding-bottom:1.25em}
td.hdlist2{word-wrap:anywhere}
.literalblock+.colist,.listingblock+.colist{margin-top:-.5em}
.colist td:not([class]):first-child{padding:.4em .75em 0;line-height:1;vertical-align:top}
.colist td:not([class]):first-child img{max-width:none}
.colist td:not([class]):last-child{padding:.25em 0}
.thumb,.th{line-height:0;display:inline-block;border:4px solid #fff;box-shadow:0 0 0 1px #ddd}
.imageblock.left{margin:.25em .625em 1.25em 0}
.imageblock.right{margin:.25em 0 1.25em .625em}
.imageblock>.title{margin-bottom:0}
.imageblock.thumb,.imageblock.th{border-width:6px}
.imageblock.thumb>.title,.imageblock.th>.title{padding:0 .125em}
.image.left,.image.right{margin-top:.25em;margin-bottom:.25em;display:inline-block;line-height:0}
.image.left{margin-right:.625em}
.image.right{margin-left:.625em}
a.image{text-decoration:none;display:inline-block}
a.image object{pointer-events:none}
sup.footnote,sup.footnoteref{font-size:.875em;position:static;vertical-align:super}
sup.footnote a,sup.footnoteref a{text-decoration:none}
sup.footnote a:active,sup.footnoteref a:active,#footnotes .footnote a:first-of-type:active{text-decoration:underline}
#footnotes{padding-top:.75em;padding-bottom:.75em;margin-bottom:.625em}
#footnotes hr{width:20%;min-width:6.25em;margin:-.25em 0 .75em;border-width:1px 0 0}
#footnotes .footnote{padding:0 .375em 0 .225em;line-height:1.3334;font-size:.875em;margin-left:1.2em;margin-bottom:.2em}
#footnotes .footnote a:first-of-type{font-weight:bold;text-decoration:none;margin-left:-1.05em}
#footnotes .footnote:last-of-type{margin-bottom:0}
#content #footnotes{margin-top:-.625em;margin-bottom:0;padding:.75em 0}
div.unbreakable{page-break-inside:avoid}
.big{font-size:larger}
.small{font-size:smaller}
.underline{text-decoration:underline}
.overline{text-decoration:overline}
.line-through{text-decoration:line-through}
.aqua{color:#00bfbf}
.aqua-background{background:#00fafa}
.black{color:#000}
.black-background{background:#000}
.blue{color:#0000bf}
.blue-background{background:#0000fa}
.fuchsia{color:#bf00bf}
.fuchsia-background{background:#fa00fa}
.gray{color:#606060}
.gray-background{background:#7d7d7d}
.green{color:#006000}
.green-background{background:#007d00}
.lime{color:#00bf00}
.lime-background{background:#00fa00}
.maroon{color:#600000}
.maroon-background{background:#7d0000}
.navy{color:#000060}
.navy-background{background:#00007d}
.olive{color:#606000}
.olive-background{background:#7d7d00}
.purple{color:#600060}
.purple-background{background:#7d007d}
.red{color:#bf0000}
.red-background{background:#fa0000}
.silver{color:#909090}
.silver-background{background:#bcbcbc}
.teal{color:#006060}
.teal-background{background:#007d7d}
.white{color:#bfbfbf}
.white-background{background:#fafafa}
.yellow{color:#bfbf00}
.yellow-background{background:#fafa00}
span.icon>.fa{cursor:default}
a span.icon>.fa{cursor:inherit}
.admonitionblock td.icon [class^="fa icon-"]{font-size:2.5em;text-shadow:1px 1px 2px rgba(0,0,0,.5);cursor:default}
.admonitionblock td.icon .icon-note::before{content:"\f05a";color:#19407c}
.admonitionblock td.icon .icon-tip::before{content:"\f0eb";text-shadow:1px 1px 2px rgba(155,155,0,.8);color:#111}
.admonitionblock td.icon .icon-warning::before{content:"\f071";color:#bf6900}
.admonitionblock td.icon .icon-caution::before{content:"\f06d";color:#bf3400}
.admonitionblock td.icon .icon-important::before{content:"\f06a";color:#bf0000}
.conum[data-value]{display:inline-block;color:#fff!important;background:rgba(0,0,0,.8);border-radius:50%;text-align:center;font-size:.75em;width:1.67em;height:1.67em;line-height:1.67em;font-family:"Open Sans","DejaVu Sans",sans-serif;font-style:normal;font-weight:bold}
.conum[data-value] *{color:#fff!important}
.conum[data-value]+b{display:none}
.conum[data-value]::after{content:attr(data-value)}
pre .conum[data-value]{position:relative;top:-.125em}
b.conum *{color:inherit!important}
.conum:not([data-value]):empty{display:none}
dt,th.tableblock,td.content,div.footnote{text-rendering:optimizeLegibility}
h1,h2,p,td.content,span.alt,summary{letter-spacing:-.01em}
p strong,td.content strong,div.footnote strong{letter-spacing:-.005em}
p,blockquote,dt,td.content,td.hdlist1,span.alt,summary{font-size:1.0625rem}
p{margin-bottom:1.25rem}
.sidebarblock p,.sidebarblock dt,.sidebarblock td.content,p.tableblock{font-size:1em}
.exampleblock>.content{background:#fffef7;border-color:#e0e0dc;box-shadow:0 1px 4px #e0e0dc}
.print-only{display:none!important}
@page{margin:1.25cm .75cm}
@media print{*{box-shadow:none!important;text-shadow:none!important}
html{font-size:80%}
a{color:inherit!important;text-decoration:underline!important}
a.bare,a[href^="#"],a[href^="mailto:"]{text-decoration:none!important}
a[href^="http:"]:not(.bare)::after,a[href^="https:"]:not(.bare)::after{content:"(" attr(href) ")";display:inline-block;font-size:.875em;padding-left:.25em}
abbr[title]{border-bottom:1px dotted}
abbr[title]::after{content:" (" attr(title) ")"}
pre,blockquote,tr,img,object,svg{page-break-inside:avoid}
thead{display:table-header-group}
svg{max-width:100%}
p,blockquote,dt,td.content{font-size:1em;orphans:3;widows:3}
h2,h3,#toctitle,.sidebarblock>.content>.title{page-break-after:avoid}
#header,#content,#footnotes,#footer{max-width:none}
#toc,.sidebarblock,.exampleblock>.content{background:none!important}
#toc{border-bottom:1px solid #dddddf!important;padding-bottom:0!important}
body.book #header{text-align:center}
body.book #header>h1:first-child{border:0!important;margin:2.5em 0 1em}
body.book #header .details{border:0!important;display:block;padding:0!important}
body.book #header .details span:first-child{margin-left:0!important}
body.book #header .details br{display:block}
body.book #header .details br+span::before{content:none!important}
body.book #toc{border:0!important;text-align:left!important;padding:0!important;margin:0!important}
body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-break-before:always}
.listingblock code[data-lang]::before{display:block}
#footer{padding:0 .9375em}
.hide-on-print{display:none!important}
.print-only{display:block!important}
.hide-for-print{display:none!important}
.show-for-print{display:inherit!important}}
@media amzn-kf8,print{#header>h1:first-child{margin-top:1.25rem}
.sect1{padding:0!important}
.sect1+.sect1{border:0}
#footer{background:none}
#footer-text{color:rgba(0,0,0,.6);font-size:.9em}}
@media amzn-kf8{#header,#content,#footnotes,#footer{padding:0}}
</style>
</head>
<body class="article">
<div id="header">
<h1>Path-Walk API</h1>
</div>
<div id="content">
<div id="preamble">
<div class="sectionbody">
<div class="paragraph">
<p>The path-walk API is used to walk reachable objects, but to visit objects
in batches based on a common path they appear in, or by type.</p>
</div>
<div class="paragraph">
<p>For example, all reachable commits are visited in a group. All tags are
visited in a group. Then, all root trees are visited. At some point, all
blobs reachable via a path <code>my/dir/to/A</code> are visited. When there are
multiple paths possible to reach the same object, then only one of those
paths is used to visit the object.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_basics">Basics</h2>
<div class="sectionbody">
<div class="paragraph">
<p>To use the path-walk API, include <code>path-walk.h</code> and call
<code>walk_objects_by_path</code>() with a customized <code>path_walk_info</code> struct. The
struct is used to set all of the options for how the walk should proceed.
Let&#8217;s dig into the different options and their use.</p>
</div>
<div class="dlist">
<dl>
<dt class="hdlist1"><code>path_fn</code> and <code>path_fn_data</code></dt>
<dd>
<p>The most important option is the <code>path_fn</code> option, which is a
function pointer to the callback that can execute logic on the
object IDs for objects grouped by type and path. This function
also receives a <code>data</code> value that corresponds to the
<code>path_fn_data</code> member, for providing custom data structures to
this callback function.</p>
</dd>
<dt class="hdlist1"><code>revs</code></dt>
<dd>
<p>To configure the exact details of the reachable set of objects,
use the <code>revs</code> member and initialize it using the revision
machinery in <code>revision.h</code>. Initialize <code>revs</code> using calls such as
<code>setup_revisions</code>() or <code>parse_revision_opt</code>(). Do not call
<code>prepare_revision_walk</code>(), as that will be called within
<code>walk_objects_by_path</code>().</p>
<div class="paragraph">
<p>It is also important that you do not specify the <code>--objects</code> flag for the
<code>revs</code> struct. The revision walk should only be used to walk commits, and
the objects will be walked in a separate way based on those starting
commits.</p>
</div>
</dd>
<dt class="hdlist1"><code>commits</code>, <code>blobs</code>, <code>trees</code>, <code>tags</code></dt>
<dd>
<p>By default, these members are enabled and signal that the path-walk
API should call the <code>path_fn</code> on objects of these types. Specialized
applications could disable some options to make it simpler to walk
the objects or to have fewer calls to <code>path_fn</code>.</p>
<div class="paragraph">
<p>While it is possible to walk only commits in this way, consumers would be
better off using the revision walk API instead.</p>
</div>
</dd>
<dt class="hdlist1"><code>prune_all_uninteresting</code></dt>
<dd>
<p>By default, all reachable paths are emitted by the path-walk API.
This option allows consumers to declare that they are not
interested in paths where all included objects are marked with the
<code>UNINTERESTING</code> flag. This requires using the <code>boundary</code> option in
the revision walk so that the walk emits commits marked with the
<code>UNINTERESTING</code> flag.</p>
</dd>
<dt class="hdlist1"><code>pl</code></dt>
<dd>
<p>This pattern list pointer allows focusing the path-walk search to
a set of patterns, only emitting paths that match the given
patterns. See <a href="../gitignore.html">gitignore(5)</a> or
<a href="../git-sparse-checkout.html">git-sparse-checkout(1)</a> for details about pattern lists.
When the pattern list uses cone-mode patterns, then the path-walk
API can prune the set of paths it walks to improve performance.</p>
</dd>
</dl>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_examples">Examples</h2>
<div class="sectionbody">
<div class="paragraph">
<p>See example usages in:
<code>t/helper/test-path-walk.c</code>,
<code>builtin/backfill.c</code>
<code>builtin/pack-objects.c</code></p>
</div>
</div>
</div>
</div>
<div id="footer">
<div id="footer-text">
Last updated 2025-03-17 07:09:38 UTC
</div>
</div>
</body>
</html>

View File

@@ -0,0 +1,105 @@
Simple-IPC API
==============
The Simple-IPC API is a collection of `ipc_` prefixed library routines
and a basic communication protocol that allows an IPC-client process to
send an application-specific IPC-request message to an IPC-server
process and receive an application-specific IPC-response message.
Communication occurs over a named pipe on Windows and a Unix domain
socket on other platforms. IPC-clients and IPC-servers rendezvous at
a previously agreed-to application-specific pathname (which is outside
the scope of this design) that is local to the computer system.
The IPC-server routines within the server application process create a
thread pool to listen for connections and receive request messages
from multiple concurrent IPC-clients. When received, these messages
are dispatched up to the server application callbacks for handling.
IPC-server routines then incrementally relay responses back to the
IPC-client.
The IPC-client routines within a client application process connect
to the IPC-server and send a request message and wait for a response.
When received, the response is returned back to the caller.
For example, the `fsmonitor--daemon` feature will be built as a server
application on top of the IPC-server library routines. It will have
threads watching for file system events and a thread pool waiting for
client connections. Clients, such as `git status`, will request a list
of file system events since a point in time and the server will
respond with a list of changed files and directories. The formats of
the request and response are application-specific; the IPC-client and
IPC-server routines treat them as opaque byte streams.
Comparison with sub-process model
---------------------------------
The Simple-IPC mechanism differs from the existing `sub-process.c`
model (Documentation/technical/long-running-process-protocol.adoc) and
used by applications like Git-LFS. In the LFS-style sub-process model,
the helper is started by the foreground process, communication happens
via a pair of file descriptors bound to the stdin/stdout of the
sub-process, the sub-process only serves the current foreground
process, and the sub-process exits when the foreground process
terminates.
In the Simple-IPC model the server is a very long-running service. It
can service many clients at the same time and has a private socket or
named pipe connection to each active client. It might be started
(on-demand) by the current client process or it might have been
started by a previous client or by the OS at boot time. The server
process is not associated with a terminal and it persists after
clients terminate. Clients do not have access to the stdin/stdout of
the server process and therefore must communicate over sockets or
named pipes.
Server startup and shutdown
---------------------------
How an application server based upon IPC-server is started is also
outside the scope of the Simple-IPC design and is a property of the
application using it. For example, the server might be started or
restarted during routine maintenance operations, or it might be
started as a system service during the system boot-up sequence, or it
might be started on-demand by a foreground Git command when needed.
Similarly, server shutdown is a property of the application using
the simple-ipc routines. For example, the server might decide to
shutdown when idle or only upon explicit request.
Simple-IPC protocol
-------------------
The Simple-IPC protocol consists of a single request message from the
client and an optional response message from the server. Both the
client and server messages are unlimited in length and are terminated
with a flush packet.
The pkt-line routines (linkgit:gitprotocol-common[5])
are used to simplify buffer management during message generation,
transmission, and reception. A flush packet is used to mark the end
of the message. This allows the sender to incrementally generate and
transmit the message. It allows the receiver to incrementally receive
the message in chunks and to know when they have received the entire
message.
The actual byte format of the client request and server response
messages are application specific. The IPC layer transmits and
receives them as opaque byte buffers without any concern for the
content within. It is the job of the calling application layer to
understand the contents of the request and response messages.
Summary
-------
Conceptually, the Simple-IPC protocol is similar to an HTTP REST
request. Clients connect, make an application-specific and
stateless request, receive an application-specific
response, and disconnect. It is a one round trip facility for
querying the server. The Simple-IPC routines hide the socket,
named pipe, and thread pool details and allow the application
layer to focus on the task at hand.

View File

@@ -0,0 +1,573 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="en">
<head>
<meta charset="UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=edge"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<meta name="generator" content="Asciidoctor 2.0.23"/>
<title>Simple-IPC API</title>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"/>
<style>
/*! Asciidoctor default stylesheet | MIT License | https://asciidoctor.org */
/* Uncomment the following line when using as a custom stylesheet */
/* @import "https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"; */
html{font-family:sans-serif;-webkit-text-size-adjust:100%}
a{background:none}
a:focus{outline:thin dotted}
a:active,a:hover{outline:0}
h1{font-size:2em;margin:.67em 0}
b,strong{font-weight:bold}
abbr{font-size:.9em}
abbr[title]{cursor:help;border-bottom:1px dotted #dddddf;text-decoration:none}
dfn{font-style:italic}
hr{height:0}
mark{background:#ff0;color:#000}
code,kbd,pre,samp{font-family:monospace;font-size:1em}
pre{white-space:pre-wrap}
q{quotes:"\201C" "\201D" "\2018" "\2019"}
small{font-size:80%}
sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline}
sup{top:-.5em}
sub{bottom:-.25em}
img{border:0}
svg:not(:root){overflow:hidden}
figure{margin:0}
audio,video{display:inline-block}
audio:not([controls]){display:none;height:0}
fieldset{border:1px solid silver;margin:0 2px;padding:.35em .625em .75em}
legend{border:0;padding:0}
button,input,select,textarea{font-family:inherit;font-size:100%;margin:0}
button,input{line-height:normal}
button,select{text-transform:none}
button,html input[type=button],input[type=reset],input[type=submit]{-webkit-appearance:button;cursor:pointer}
button[disabled],html input[disabled]{cursor:default}
input[type=checkbox],input[type=radio]{padding:0}
button::-moz-focus-inner,input::-moz-focus-inner{border:0;padding:0}
textarea{overflow:auto;vertical-align:top}
table{border-collapse:collapse;border-spacing:0}
*,::before,::after{box-sizing:border-box}
html,body{font-size:100%}
body{background:#fff;color:rgba(0,0,0,.8);padding:0;margin:0;font-family:"Noto Serif","DejaVu Serif",serif;line-height:1;position:relative;cursor:auto;-moz-tab-size:4;-o-tab-size:4;tab-size:4;word-wrap:anywhere;-moz-osx-font-smoothing:grayscale;-webkit-font-smoothing:antialiased}
a:hover{cursor:pointer}
img,object,embed{max-width:100%;height:auto}
object,embed{height:100%}
img{-ms-interpolation-mode:bicubic}
.left{float:left!important}
.right{float:right!important}
.text-left{text-align:left!important}
.text-right{text-align:right!important}
.text-center{text-align:center!important}
.text-justify{text-align:justify!important}
.hide{display:none}
img,object,svg{display:inline-block;vertical-align:middle}
textarea{height:auto;min-height:50px}
select{width:100%}
.subheader,.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{line-height:1.45;color:#7a2518;font-weight:400;margin-top:0;margin-bottom:.25em}
div,dl,dt,dd,ul,ol,li,h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6,pre,form,p,blockquote,th,td{margin:0;padding:0}
a{color:#2156a5;text-decoration:underline;line-height:inherit}
a:hover,a:focus{color:#1d4b8f}
a img{border:0}
p{line-height:1.6;margin-bottom:1.25em;text-rendering:optimizeLegibility}
p aside{font-size:.875em;line-height:1.35;font-style:italic}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{font-family:"Open Sans","DejaVu Sans",sans-serif;font-weight:300;font-style:normal;color:#ba3925;text-rendering:optimizeLegibility;margin-top:1em;margin-bottom:.5em;line-height:1.0125em}
h1 small,h2 small,h3 small,#toctitle small,.sidebarblock>.content>.title small,h4 small,h5 small,h6 small{font-size:60%;color:#e99b8f;line-height:0}
h1{font-size:2.125em}
h2{font-size:1.6875em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.375em}
h4,h5{font-size:1.125em}
h6{font-size:1em}
hr{border:solid #dddddf;border-width:1px 0 0;clear:both;margin:1.25em 0 1.1875em}
em,i{font-style:italic;line-height:inherit}
strong,b{font-weight:bold;line-height:inherit}
small{font-size:60%;line-height:inherit}
code{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:400;color:rgba(0,0,0,.9)}
ul,ol,dl{line-height:1.6;margin-bottom:1.25em;list-style-position:outside;font-family:inherit}
ul,ol{margin-left:1.5em}
ul li ul,ul li ol{margin-left:1.25em;margin-bottom:0}
ul.circle{list-style-type:circle}
ul.disc{list-style-type:disc}
ul.square{list-style-type:square}
ul.circle ul:not([class]),ul.disc ul:not([class]),ul.square ul:not([class]){list-style:inherit}
ol li ul,ol li ol{margin-left:1.25em;margin-bottom:0}
dl dt{margin-bottom:.3125em;font-weight:bold}
dl dd{margin-bottom:1.25em}
blockquote{margin:0 0 1.25em;padding:.5625em 1.25em 0 1.1875em;border-left:1px solid #ddd}
blockquote,blockquote p{line-height:1.6;color:rgba(0,0,0,.85)}
@media screen and (min-width:768px){h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2}
h1{font-size:2.75em}
h2{font-size:2.3125em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.6875em}
h4{font-size:1.4375em}}
table{background:#fff;margin-bottom:1.25em;border:1px solid #dedede;word-wrap:normal}
table thead,table tfoot{background:#f7f8f7}
table thead tr th,table thead tr td,table tfoot tr th,table tfoot tr td{padding:.5em .625em .625em;font-size:inherit;color:rgba(0,0,0,.8);text-align:left}
table tr th,table tr td{padding:.5625em .625em;font-size:inherit;color:rgba(0,0,0,.8)}
table tr.even,table tr.alt{background:#f8f8f7}
table thead tr th,table tfoot tr th,table tbody tr td,table tr td,table tfoot tr td{line-height:1.6}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2;word-spacing:-.05em}
h1 strong,h2 strong,h3 strong,#toctitle strong,.sidebarblock>.content>.title strong,h4 strong,h5 strong,h6 strong{font-weight:400}
.center{margin-left:auto;margin-right:auto}
.stretch{width:100%}
.clearfix::before,.clearfix::after,.float-group::before,.float-group::after{content:" ";display:table}
.clearfix::after,.float-group::after{clear:both}
:not(pre).nobreak{word-wrap:normal}
:not(pre).nowrap{white-space:nowrap}
:not(pre).pre-wrap{white-space:pre-wrap}
:not(pre):not([class^=L])>code{font-size:.9375em;font-style:normal!important;letter-spacing:0;padding:.1em .5ex;word-spacing:-.15em;background:#f7f7f8;border-radius:4px;line-height:1.45;text-rendering:optimizeSpeed}
pre{color:rgba(0,0,0,.9);font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;line-height:1.45;text-rendering:optimizeSpeed}
pre code,pre pre{color:inherit;font-size:inherit;line-height:inherit}
pre>code{display:block}
pre.nowrap,pre.nowrap pre{white-space:pre;word-wrap:normal}
em em{font-style:normal}
strong strong{font-weight:400}
.keyseq{color:rgba(51,51,51,.8)}
kbd{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;display:inline-block;color:rgba(0,0,0,.8);font-size:.65em;line-height:1.45;background:#f7f7f7;border:1px solid #ccc;border-radius:3px;box-shadow:0 1px 0 rgba(0,0,0,.2),inset 0 0 0 .1em #fff;margin:0 .15em;padding:.2em .5em;vertical-align:middle;position:relative;top:-.1em;white-space:nowrap}
.keyseq kbd:first-child{margin-left:0}
.keyseq kbd:last-child{margin-right:0}
.menuseq,.menuref{color:#000}
.menuseq b:not(.caret),.menuref{font-weight:inherit}
.menuseq{word-spacing:-.02em}
.menuseq b.caret{font-size:1.25em;line-height:.8}
.menuseq i.caret{font-weight:bold;text-align:center;width:.45em}
b.button::before,b.button::after{position:relative;top:-1px;font-weight:400}
b.button::before{content:"[";padding:0 3px 0 2px}
b.button::after{content:"]";padding:0 2px 0 3px}
p a>code:hover{color:rgba(0,0,0,.9)}
#header,#content,#footnotes,#footer{width:100%;margin:0 auto;max-width:62.5em;*zoom:1;position:relative;padding-left:.9375em;padding-right:.9375em}
#header::before,#header::after,#content::before,#content::after,#footnotes::before,#footnotes::after,#footer::before,#footer::after{content:" ";display:table}
#header::after,#content::after,#footnotes::after,#footer::after{clear:both}
#content{margin-top:1.25em}
#content::before{content:none}
#header>h1:first-child{color:rgba(0,0,0,.85);margin-top:2.25rem;margin-bottom:0}
#header>h1:first-child+#toc{margin-top:8px;border-top:1px solid #dddddf}
#header>h1:only-child{border-bottom:1px solid #dddddf;padding-bottom:8px}
#header .details{border-bottom:1px solid #dddddf;line-height:1.45;padding-top:.25em;padding-bottom:.25em;padding-left:.25em;color:rgba(0,0,0,.6);display:flex;flex-flow:row wrap}
#header .details span:first-child{margin-left:-.125em}
#header .details span.email a{color:rgba(0,0,0,.85)}
#header .details br{display:none}
#header .details br+span::before{content:"\00a0\2013\00a0"}
#header .details br+span.author::before{content:"\00a0\22c5\00a0";color:rgba(0,0,0,.85)}
#header .details br+span#revremark::before{content:"\00a0|\00a0"}
#header #revnumber{text-transform:capitalize}
#header #revnumber::after{content:"\00a0"}
#content>h1:first-child:not([class]){color:rgba(0,0,0,.85);border-bottom:1px solid #dddddf;padding-bottom:8px;margin-top:0;padding-top:1rem;margin-bottom:1.25rem}
#toc{border-bottom:1px solid #e7e7e9;padding-bottom:.5em}
#toc>ul{margin-left:.125em}
#toc ul.sectlevel0>li>a{font-style:italic}
#toc ul.sectlevel0 ul.sectlevel1{margin:.5em 0}
#toc ul{font-family:"Open Sans","DejaVu Sans",sans-serif;list-style-type:none}
#toc li{line-height:1.3334;margin-top:.3334em}
#toc a{text-decoration:none}
#toc a:active{text-decoration:underline}
#toctitle{color:#7a2518;font-size:1.2em}
@media screen and (min-width:768px){#toctitle{font-size:1.375em}
body.toc2{padding-left:15em;padding-right:0}
body.toc2 #header>h1:nth-last-child(2){border-bottom:1px solid #dddddf;padding-bottom:8px}
#toc.toc2{margin-top:0!important;background:#f8f8f7;position:fixed;width:15em;left:0;top:0;border-right:1px solid #e7e7e9;border-top-width:0!important;border-bottom-width:0!important;z-index:1000;padding:1.25em 1em;height:100%;overflow:auto}
#toc.toc2 #toctitle{margin-top:0;margin-bottom:.8rem;font-size:1.2em}
#toc.toc2>ul{font-size:.9em;margin-bottom:0}
#toc.toc2 ul ul{margin-left:0;padding-left:1em}
#toc.toc2 ul.sectlevel0 ul.sectlevel1{padding-left:0;margin-top:.5em;margin-bottom:.5em}
body.toc2.toc-right{padding-left:0;padding-right:15em}
body.toc2.toc-right #toc.toc2{border-right-width:0;border-left:1px solid #e7e7e9;left:auto;right:0}}
@media screen and (min-width:1280px){body.toc2{padding-left:20em;padding-right:0}
#toc.toc2{width:20em}
#toc.toc2 #toctitle{font-size:1.375em}
#toc.toc2>ul{font-size:.95em}
#toc.toc2 ul ul{padding-left:1.25em}
body.toc2.toc-right{padding-left:0;padding-right:20em}}
#content #toc{border:1px solid #e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;border-radius:4px}
#content #toc>:first-child{margin-top:0}
#content #toc>:last-child{margin-bottom:0}
#footer{max-width:none;background:rgba(0,0,0,.8);padding:1.25em}
#footer-text{color:hsla(0,0%,100%,.8);line-height:1.44}
#content{margin-bottom:.625em}
.sect1{padding-bottom:.625em}
@media screen and (min-width:768px){#content{margin-bottom:1.25em}
.sect1{padding-bottom:1.25em}}
.sect1:last-child{padding-bottom:0}
.sect1+.sect1{border-top:1px solid #e7e7e9}
#content h1>a.anchor,h2>a.anchor,h3>a.anchor,#toctitle>a.anchor,.sidebarblock>.content>.title>a.anchor,h4>a.anchor,h5>a.anchor,h6>a.anchor{position:absolute;z-index:1001;width:1.5ex;margin-left:-1.5ex;display:block;text-decoration:none!important;visibility:hidden;text-align:center;font-weight:400}
#content h1>a.anchor::before,h2>a.anchor::before,h3>a.anchor::before,#toctitle>a.anchor::before,.sidebarblock>.content>.title>a.anchor::before,h4>a.anchor::before,h5>a.anchor::before,h6>a.anchor::before{content:"\00A7";font-size:.85em;display:block;padding-top:.1em}
#content h1:hover>a.anchor,#content h1>a.anchor:hover,h2:hover>a.anchor,h2>a.anchor:hover,h3:hover>a.anchor,#toctitle:hover>a.anchor,.sidebarblock>.content>.title:hover>a.anchor,h3>a.anchor:hover,#toctitle>a.anchor:hover,.sidebarblock>.content>.title>a.anchor:hover,h4:hover>a.anchor,h4>a.anchor:hover,h5:hover>a.anchor,h5>a.anchor:hover,h6:hover>a.anchor,h6>a.anchor:hover{visibility:visible}
#content h1>a.link,h2>a.link,h3>a.link,#toctitle>a.link,.sidebarblock>.content>.title>a.link,h4>a.link,h5>a.link,h6>a.link{color:#ba3925;text-decoration:none}
#content h1>a.link:hover,h2>a.link:hover,h3>a.link:hover,#toctitle>a.link:hover,.sidebarblock>.content>.title>a.link:hover,h4>a.link:hover,h5>a.link:hover,h6>a.link:hover{color:#a53221}
details,.audioblock,.imageblock,.literalblock,.listingblock,.stemblock,.videoblock{margin-bottom:1.25em}
details{margin-left:1.25rem}
details>summary{cursor:pointer;display:block;position:relative;line-height:1.6;margin-bottom:.625rem;outline:none;-webkit-tap-highlight-color:transparent}
details>summary::-webkit-details-marker{display:none}
details>summary::before{content:"";border:solid transparent;border-left:solid;border-width:.3em 0 .3em .5em;position:absolute;top:.5em;left:-1.25rem;transform:translateX(15%)}
details[open]>summary::before{border:solid transparent;border-top:solid;border-width:.5em .3em 0;transform:translateY(15%)}
details>summary::after{content:"";width:1.25rem;height:1em;position:absolute;top:.3em;left:-1.25rem}
.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{text-rendering:optimizeLegibility;text-align:left;font-family:"Noto Serif","DejaVu Serif",serif;font-size:1rem;font-style:italic}
table.tableblock.fit-content>caption.title{white-space:nowrap;width:0}
.paragraph.lead>p,#preamble>.sectionbody>[class=paragraph]:first-of-type p{font-size:1.21875em;line-height:1.6;color:rgba(0,0,0,.85)}
.admonitionblock>table{border-collapse:separate;border:0;background:none;width:100%}
.admonitionblock>table td.icon{text-align:center;width:80px}
.admonitionblock>table td.icon img{max-width:none}
.admonitionblock>table td.icon .title{font-weight:bold;font-family:"Open Sans","DejaVu Sans",sans-serif;text-transform:uppercase}
.admonitionblock>table td.content{padding-left:1.125em;padding-right:1.25em;border-left:1px solid #dddddf;color:rgba(0,0,0,.6);word-wrap:anywhere}
.admonitionblock>table td.content>:last-child>:last-child{margin-bottom:0}
.exampleblock>.content{border:1px solid #e6e6e6;margin-bottom:1.25em;padding:1.25em;background:#fff;border-radius:4px}
.sidebarblock{border:1px solid #dbdbd6;margin-bottom:1.25em;padding:1.25em;background:#f3f3f2;border-radius:4px}
.sidebarblock>.content>.title{color:#7a2518;margin-top:0;text-align:center}
.exampleblock>.content>:first-child,.sidebarblock>.content>:first-child{margin-top:0}
.exampleblock>.content>:last-child,.exampleblock>.content>:last-child>:last-child,.exampleblock>.content .olist>ol>li:last-child>:last-child,.exampleblock>.content .ulist>ul>li:last-child>:last-child,.exampleblock>.content .qlist>ol>li:last-child>:last-child,.sidebarblock>.content>:last-child,.sidebarblock>.content>:last-child>:last-child,.sidebarblock>.content .olist>ol>li:last-child>:last-child,.sidebarblock>.content .ulist>ul>li:last-child>:last-child,.sidebarblock>.content .qlist>ol>li:last-child>:last-child{margin-bottom:0}
.literalblock pre,.listingblock>.content>pre{border-radius:4px;overflow-x:auto;padding:1em;font-size:.8125em}
@media screen and (min-width:768px){.literalblock pre,.listingblock>.content>pre{font-size:.90625em}}
@media screen and (min-width:1280px){.literalblock pre,.listingblock>.content>pre{font-size:1em}}
.literalblock pre,.listingblock>.content>pre:not(.highlight),.listingblock>.content>pre[class=highlight],.listingblock>.content>pre[class^="highlight "]{background:#f7f7f8}
.literalblock.output pre{color:#f7f7f8;background:rgba(0,0,0,.9)}
.listingblock>.content{position:relative}
.listingblock code[data-lang]::before{display:none;content:attr(data-lang);position:absolute;font-size:.75em;top:.425rem;right:.5rem;line-height:1;text-transform:uppercase;color:inherit;opacity:.5}
.listingblock:hover code[data-lang]::before{display:block}
.listingblock.terminal pre .command::before{content:attr(data-prompt);padding-right:.5em;color:inherit;opacity:.5}
.listingblock.terminal pre .command:not([data-prompt])::before{content:"$"}
.listingblock pre.highlightjs{padding:0}
.listingblock pre.highlightjs>code{padding:1em;border-radius:4px}
.listingblock pre.prettyprint{border-width:0}
.prettyprint{background:#f7f7f8}
pre.prettyprint .linenums{line-height:1.45;margin-left:2em}
pre.prettyprint li{background:none;list-style-type:inherit;padding-left:0}
pre.prettyprint li code[data-lang]::before{opacity:1}
pre.prettyprint li:not(:first-child) code[data-lang]::before{display:none}
table.linenotable{border-collapse:separate;border:0;margin-bottom:0;background:none}
table.linenotable td[class]{color:inherit;vertical-align:top;padding:0;line-height:inherit;white-space:normal}
table.linenotable td.code{padding-left:.75em}
table.linenotable td.linenos,pre.pygments .linenos{border-right:1px solid;opacity:.35;padding-right:.5em;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}
pre.pygments span.linenos{display:inline-block;margin-right:.75em}
.quoteblock{margin:0 1em 1.25em 1.5em;display:table}
.quoteblock:not(.excerpt)>.title{margin-left:-1.5em;margin-bottom:.75em}
.quoteblock blockquote,.quoteblock p{color:rgba(0,0,0,.85);font-size:1.15rem;line-height:1.75;word-spacing:.1em;letter-spacing:0;font-style:italic;text-align:justify}
.quoteblock blockquote{margin:0;padding:0;border:0}
.quoteblock blockquote::before{content:"\201c";float:left;font-size:2.75em;font-weight:bold;line-height:.6em;margin-left:-.6em;color:#7a2518;text-shadow:0 1px 2px rgba(0,0,0,.1)}
.quoteblock blockquote>.paragraph:last-child p{margin-bottom:0}
.quoteblock .attribution{margin-top:.75em;margin-right:.5ex;text-align:right}
.verseblock{margin:0 1em 1.25em}
.verseblock pre{font-family:"Open Sans","DejaVu Sans",sans-serif;font-size:1.15rem;color:rgba(0,0,0,.85);font-weight:300;text-rendering:optimizeLegibility}
.verseblock pre strong{font-weight:400}
.verseblock .attribution{margin-top:1.25rem;margin-left:.5ex}
.quoteblock .attribution,.verseblock .attribution{font-size:.9375em;line-height:1.45;font-style:italic}
.quoteblock .attribution br,.verseblock .attribution br{display:none}
.quoteblock .attribution cite,.verseblock .attribution cite{display:block;letter-spacing:-.025em;color:rgba(0,0,0,.6)}
.quoteblock.abstract blockquote::before,.quoteblock.excerpt blockquote::before,.quoteblock .quoteblock blockquote::before{display:none}
.quoteblock.abstract blockquote,.quoteblock.abstract p,.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{line-height:1.6;word-spacing:0}
.quoteblock.abstract{margin:0 1em 1.25em;display:block}
.quoteblock.abstract>.title{margin:0 0 .375em;font-size:1.15em;text-align:center}
.quoteblock.excerpt>blockquote,.quoteblock .quoteblock{padding:0 0 .25em 1em;border-left:.25em solid #dddddf}
.quoteblock.excerpt,.quoteblock .quoteblock{margin-left:0}
.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{color:inherit;font-size:1.0625rem}
.quoteblock.excerpt .attribution,.quoteblock .quoteblock .attribution{color:inherit;font-size:.85rem;text-align:left;margin-right:0}
p.tableblock:last-child{margin-bottom:0}
td.tableblock>.content{margin-bottom:1.25em;word-wrap:anywhere}
td.tableblock>.content>:last-child{margin-bottom:-1.25em}
table.tableblock,th.tableblock,td.tableblock{border:0 solid #dedede}
table.grid-all>*>tr>*{border-width:1px}
table.grid-cols>*>tr>*{border-width:0 1px}
table.grid-rows>*>tr>*{border-width:1px 0}
table.frame-all{border-width:1px}
table.frame-ends{border-width:1px 0}
table.frame-sides{border-width:0 1px}
table.frame-none>colgroup+*>:first-child>*,table.frame-sides>colgroup+*>:first-child>*{border-top-width:0}
table.frame-none>:last-child>:last-child>*,table.frame-sides>:last-child>:last-child>*{border-bottom-width:0}
table.frame-none>*>tr>:first-child,table.frame-ends>*>tr>:first-child{border-left-width:0}
table.frame-none>*>tr>:last-child,table.frame-ends>*>tr>:last-child{border-right-width:0}
table.stripes-all>*>tr,table.stripes-odd>*>tr:nth-of-type(odd),table.stripes-even>*>tr:nth-of-type(even),table.stripes-hover>*>tr:hover{background:#f8f8f7}
th.halign-left,td.halign-left{text-align:left}
th.halign-right,td.halign-right{text-align:right}
th.halign-center,td.halign-center{text-align:center}
th.valign-top,td.valign-top{vertical-align:top}
th.valign-bottom,td.valign-bottom{vertical-align:bottom}
th.valign-middle,td.valign-middle{vertical-align:middle}
table thead th,table tfoot th{font-weight:bold}
tbody tr th{background:#f7f8f7}
tbody tr th,tbody tr th p,tfoot tr th,tfoot tr th p{color:rgba(0,0,0,.8);font-weight:bold}
p.tableblock>code:only-child{background:none;padding:0}
p.tableblock{font-size:1em}
ol{margin-left:1.75em}
ul li ol{margin-left:1.5em}
dl dd{margin-left:1.125em}
dl dd:last-child,dl dd:last-child>:last-child{margin-bottom:0}
li p,ul dd,ol dd,.olist .olist,.ulist .ulist,.ulist .olist,.olist .ulist{margin-bottom:.625em}
ul.checklist,ul.none,ol.none,ul.no-bullet,ol.no-bullet,ol.unnumbered,ul.unstyled,ol.unstyled{list-style-type:none}
ul.no-bullet,ol.no-bullet,ol.unnumbered{margin-left:.625em}
ul.unstyled,ol.unstyled{margin-left:0}
li>p:empty:only-child::before{content:"";display:inline-block}
ul.checklist>li>p:first-child{margin-left:-1em}
ul.checklist>li>p:first-child>.fa-square-o:first-child,ul.checklist>li>p:first-child>.fa-check-square-o:first-child{width:1.25em;font-size:.8em;position:relative;bottom:.125em}
ul.checklist>li>p:first-child>input[type=checkbox]:first-child{margin-right:.25em}
ul.inline{display:flex;flex-flow:row wrap;list-style:none;margin:0 0 .625em -1.25em}
ul.inline>li{margin-left:1.25em}
.unstyled dl dt{font-weight:400;font-style:normal}
ol.arabic{list-style-type:decimal}
ol.decimal{list-style-type:decimal-leading-zero}
ol.loweralpha{list-style-type:lower-alpha}
ol.upperalpha{list-style-type:upper-alpha}
ol.lowerroman{list-style-type:lower-roman}
ol.upperroman{list-style-type:upper-roman}
ol.lowergreek{list-style-type:lower-greek}
.hdlist>table,.colist>table{border:0;background:none}
.hdlist>table>tbody>tr,.colist>table>tbody>tr{background:none}
td.hdlist1,td.hdlist2{vertical-align:top;padding:0 .625em}
td.hdlist1{font-weight:bold;padding-bottom:1.25em}
td.hdlist2{word-wrap:anywhere}
.literalblock+.colist,.listingblock+.colist{margin-top:-.5em}
.colist td:not([class]):first-child{padding:.4em .75em 0;line-height:1;vertical-align:top}
.colist td:not([class]):first-child img{max-width:none}
.colist td:not([class]):last-child{padding:.25em 0}
.thumb,.th{line-height:0;display:inline-block;border:4px solid #fff;box-shadow:0 0 0 1px #ddd}
.imageblock.left{margin:.25em .625em 1.25em 0}
.imageblock.right{margin:.25em 0 1.25em .625em}
.imageblock>.title{margin-bottom:0}
.imageblock.thumb,.imageblock.th{border-width:6px}
.imageblock.thumb>.title,.imageblock.th>.title{padding:0 .125em}
.image.left,.image.right{margin-top:.25em;margin-bottom:.25em;display:inline-block;line-height:0}
.image.left{margin-right:.625em}
.image.right{margin-left:.625em}
a.image{text-decoration:none;display:inline-block}
a.image object{pointer-events:none}
sup.footnote,sup.footnoteref{font-size:.875em;position:static;vertical-align:super}
sup.footnote a,sup.footnoteref a{text-decoration:none}
sup.footnote a:active,sup.footnoteref a:active,#footnotes .footnote a:first-of-type:active{text-decoration:underline}
#footnotes{padding-top:.75em;padding-bottom:.75em;margin-bottom:.625em}
#footnotes hr{width:20%;min-width:6.25em;margin:-.25em 0 .75em;border-width:1px 0 0}
#footnotes .footnote{padding:0 .375em 0 .225em;line-height:1.3334;font-size:.875em;margin-left:1.2em;margin-bottom:.2em}
#footnotes .footnote a:first-of-type{font-weight:bold;text-decoration:none;margin-left:-1.05em}
#footnotes .footnote:last-of-type{margin-bottom:0}
#content #footnotes{margin-top:-.625em;margin-bottom:0;padding:.75em 0}
div.unbreakable{page-break-inside:avoid}
.big{font-size:larger}
.small{font-size:smaller}
.underline{text-decoration:underline}
.overline{text-decoration:overline}
.line-through{text-decoration:line-through}
.aqua{color:#00bfbf}
.aqua-background{background:#00fafa}
.black{color:#000}
.black-background{background:#000}
.blue{color:#0000bf}
.blue-background{background:#0000fa}
.fuchsia{color:#bf00bf}
.fuchsia-background{background:#fa00fa}
.gray{color:#606060}
.gray-background{background:#7d7d7d}
.green{color:#006000}
.green-background{background:#007d00}
.lime{color:#00bf00}
.lime-background{background:#00fa00}
.maroon{color:#600000}
.maroon-background{background:#7d0000}
.navy{color:#000060}
.navy-background{background:#00007d}
.olive{color:#606000}
.olive-background{background:#7d7d00}
.purple{color:#600060}
.purple-background{background:#7d007d}
.red{color:#bf0000}
.red-background{background:#fa0000}
.silver{color:#909090}
.silver-background{background:#bcbcbc}
.teal{color:#006060}
.teal-background{background:#007d7d}
.white{color:#bfbfbf}
.white-background{background:#fafafa}
.yellow{color:#bfbf00}
.yellow-background{background:#fafa00}
span.icon>.fa{cursor:default}
a span.icon>.fa{cursor:inherit}
.admonitionblock td.icon [class^="fa icon-"]{font-size:2.5em;text-shadow:1px 1px 2px rgba(0,0,0,.5);cursor:default}
.admonitionblock td.icon .icon-note::before{content:"\f05a";color:#19407c}
.admonitionblock td.icon .icon-tip::before{content:"\f0eb";text-shadow:1px 1px 2px rgba(155,155,0,.8);color:#111}
.admonitionblock td.icon .icon-warning::before{content:"\f071";color:#bf6900}
.admonitionblock td.icon .icon-caution::before{content:"\f06d";color:#bf3400}
.admonitionblock td.icon .icon-important::before{content:"\f06a";color:#bf0000}
.conum[data-value]{display:inline-block;color:#fff!important;background:rgba(0,0,0,.8);border-radius:50%;text-align:center;font-size:.75em;width:1.67em;height:1.67em;line-height:1.67em;font-family:"Open Sans","DejaVu Sans",sans-serif;font-style:normal;font-weight:bold}
.conum[data-value] *{color:#fff!important}
.conum[data-value]+b{display:none}
.conum[data-value]::after{content:attr(data-value)}
pre .conum[data-value]{position:relative;top:-.125em}
b.conum *{color:inherit!important}
.conum:not([data-value]):empty{display:none}
dt,th.tableblock,td.content,div.footnote{text-rendering:optimizeLegibility}
h1,h2,p,td.content,span.alt,summary{letter-spacing:-.01em}
p strong,td.content strong,div.footnote strong{letter-spacing:-.005em}
p,blockquote,dt,td.content,td.hdlist1,span.alt,summary{font-size:1.0625rem}
p{margin-bottom:1.25rem}
.sidebarblock p,.sidebarblock dt,.sidebarblock td.content,p.tableblock{font-size:1em}
.exampleblock>.content{background:#fffef7;border-color:#e0e0dc;box-shadow:0 1px 4px #e0e0dc}
.print-only{display:none!important}
@page{margin:1.25cm .75cm}
@media print{*{box-shadow:none!important;text-shadow:none!important}
html{font-size:80%}
a{color:inherit!important;text-decoration:underline!important}
a.bare,a[href^="#"],a[href^="mailto:"]{text-decoration:none!important}
a[href^="http:"]:not(.bare)::after,a[href^="https:"]:not(.bare)::after{content:"(" attr(href) ")";display:inline-block;font-size:.875em;padding-left:.25em}
abbr[title]{border-bottom:1px dotted}
abbr[title]::after{content:" (" attr(title) ")"}
pre,blockquote,tr,img,object,svg{page-break-inside:avoid}
thead{display:table-header-group}
svg{max-width:100%}
p,blockquote,dt,td.content{font-size:1em;orphans:3;widows:3}
h2,h3,#toctitle,.sidebarblock>.content>.title{page-break-after:avoid}
#header,#content,#footnotes,#footer{max-width:none}
#toc,.sidebarblock,.exampleblock>.content{background:none!important}
#toc{border-bottom:1px solid #dddddf!important;padding-bottom:0!important}
body.book #header{text-align:center}
body.book #header>h1:first-child{border:0!important;margin:2.5em 0 1em}
body.book #header .details{border:0!important;display:block;padding:0!important}
body.book #header .details span:first-child{margin-left:0!important}
body.book #header .details br{display:block}
body.book #header .details br+span::before{content:none!important}
body.book #toc{border:0!important;text-align:left!important;padding:0!important;margin:0!important}
body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-break-before:always}
.listingblock code[data-lang]::before{display:block}
#footer{padding:0 .9375em}
.hide-on-print{display:none!important}
.print-only{display:block!important}
.hide-for-print{display:none!important}
.show-for-print{display:inherit!important}}
@media amzn-kf8,print{#header>h1:first-child{margin-top:1.25rem}
.sect1{padding:0!important}
.sect1+.sect1{border:0}
#footer{background:none}
#footer-text{color:rgba(0,0,0,.6);font-size:.9em}}
@media amzn-kf8{#header,#content,#footnotes,#footer{padding:0}}
</style>
</head>
<body class="article">
<div id="header">
<h1>Simple-IPC API</h1>
</div>
<div id="content">
<div id="preamble">
<div class="sectionbody">
<div class="paragraph">
<p>The Simple-IPC API is a collection of <code>ipc_</code> prefixed library routines
and a basic communication protocol that allows an IPC-client process to
send an application-specific IPC-request message to an IPC-server
process and receive an application-specific IPC-response message.</p>
</div>
<div class="paragraph">
<p>Communication occurs over a named pipe on Windows and a Unix domain
socket on other platforms. IPC-clients and IPC-servers rendezvous at
a previously agreed-to application-specific pathname (which is outside
the scope of this design) that is local to the computer system.</p>
</div>
<div class="paragraph">
<p>The IPC-server routines within the server application process create a
thread pool to listen for connections and receive request messages
from multiple concurrent IPC-clients. When received, these messages
are dispatched up to the server application callbacks for handling.
IPC-server routines then incrementally relay responses back to the
IPC-client.</p>
</div>
<div class="paragraph">
<p>The IPC-client routines within a client application process connect
to the IPC-server and send a request message and wait for a response.
When received, the response is returned back to the caller.</p>
</div>
<div class="paragraph">
<p>For example, the <code>fsmonitor--daemon</code> feature will be built as a server
application on top of the IPC-server library routines. It will have
threads watching for file system events and a thread pool waiting for
client connections. Clients, such as <code>git</code> <code>status</code>, will request a list
of file system events since a point in time and the server will
respond with a list of changed files and directories. The formats of
the request and response are application-specific; the IPC-client and
IPC-server routines treat them as opaque byte streams.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_comparison_with_sub_process_model">Comparison with sub-process model</h2>
<div class="sectionbody">
<div class="paragraph">
<p>The Simple-IPC mechanism differs from the existing <code>sub-process.c</code>
model (Documentation/technical/long-running-process-protocol.adoc) and
used by applications like Git-LFS. In the LFS-style sub-process model,
the helper is started by the foreground process, communication happens
via a pair of file descriptors bound to the stdin/stdout of the
sub-process, the sub-process only serves the current foreground
process, and the sub-process exits when the foreground process
terminates.</p>
</div>
<div class="paragraph">
<p>In the Simple-IPC model the server is a very long-running service. It
can service many clients at the same time and has a private socket or
named pipe connection to each active client. It might be started
(on-demand) by the current client process or it might have been
started by a previous client or by the OS at boot time. The server
process is not associated with a terminal and it persists after
clients terminate. Clients do not have access to the stdin/stdout of
the server process and therefore must communicate over sockets or
named pipes.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_server_startup_and_shutdown">Server startup and shutdown</h2>
<div class="sectionbody">
<div class="paragraph">
<p>How an application server based upon IPC-server is started is also
outside the scope of the Simple-IPC design and is a property of the
application using it. For example, the server might be started or
restarted during routine maintenance operations, or it might be
started as a system service during the system boot-up sequence, or it
might be started on-demand by a foreground Git command when needed.</p>
</div>
<div class="paragraph">
<p>Similarly, server shutdown is a property of the application using
the simple-ipc routines. For example, the server might decide to
shutdown when idle or only upon explicit request.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_simple_ipc_protocol">Simple-IPC protocol</h2>
<div class="sectionbody">
<div class="paragraph">
<p>The Simple-IPC protocol consists of a single request message from the
client and an optional response message from the server. Both the
client and server messages are unlimited in length and are terminated
with a flush packet.</p>
</div>
<div class="paragraph">
<p>The pkt-line routines (<a href="../gitprotocol-common.html">gitprotocol-common(5)</a>)
are used to simplify buffer management during message generation,
transmission, and reception. A flush packet is used to mark the end
of the message. This allows the sender to incrementally generate and
transmit the message. It allows the receiver to incrementally receive
the message in chunks and to know when they have received the entire
message.</p>
</div>
<div class="paragraph">
<p>The actual byte format of the client request and server response
messages are application specific. The IPC layer transmits and
receives them as opaque byte buffers without any concern for the
content within. It is the job of the calling application layer to
understand the contents of the request and response messages.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_summary">Summary</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Conceptually, the Simple-IPC protocol is similar to an HTTP REST
request. Clients connect, make an application-specific and
stateless request, receive an application-specific
response, and disconnect. It is a one round trip facility for
querying the server. The Simple-IPC routines hide the socket,
named pipe, and thread pool details and allow the application
layer to focus on the task at hand.</p>
</div>
</div>
</div>
</div>
<div id="footer">
<div id="footer-text">
Last updated 2025-03-17 07:09:38 UTC
</div>
</div>
</body>
</html>

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,398 @@
GIT bitmap v1 format
====================
== Pack and multi-pack bitmaps
Bitmaps store reachability information about the set of objects in a packfile,
or a multi-pack index (MIDX). The former is defined obviously, and the latter is
defined as the union of objects in packs contained in the MIDX.
A bitmap may belong to either one pack, or the repository's multi-pack index (if
it exists). A repository may have at most one bitmap.
An object is uniquely described by its bit position within a bitmap:
- If the bitmap belongs to a packfile, the __n__th bit corresponds to
the __n__th object in pack order. For a function `offset` which maps
objects to their byte offset within a pack, pack order is defined as
follows:
o1 <= o2 <==> offset(o1) <= offset(o2)
- If the bitmap belongs to a MIDX, the __n__th bit corresponds to the
__n__th object in MIDX order. With an additional function `pack` which
maps objects to the pack they were selected from by the MIDX, MIDX order
is defined as follows:
o1 <= o2 <==> pack(o1) <= pack(o2) /\ offset(o1) <= offset(o2)
+
The ordering between packs is done according to the MIDX's .rev file.
Notably, the preferred pack sorts ahead of all other packs.
The on-disk representation (described below) of a bitmap is the same regardless
of whether or not that bitmap belongs to a packfile or a MIDX. The only
difference is the interpretation of the bits, which is described above.
Certain bitmap extensions are supported (see: Appendix B). No extensions are
required for bitmaps corresponding to packfiles. For bitmaps that correspond to
MIDXs, both the bit-cache and rev-cache extensions are required.
== On-disk format
* A header appears at the beginning:
4-byte signature: :: {'B', 'I', 'T', 'M'}
2-byte version number (network byte order): ::
The current implementation only supports version 1
of the bitmap index (the same one as JGit).
2-byte flags (network byte order): ::
The following flags are supported:
** {empty}
BITMAP_OPT_FULL_DAG (0x1) REQUIRED: :::
This flag must always be present. It implies that the
bitmap index has been generated for a packfile or
multi-pack index (MIDX) with full closure (i.e. where
every single object in the packfile/MIDX can find its
parent links inside the same packfile/MIDX). This is a
requirement for the bitmap index format, also present in
JGit, that greatly reduces the complexity of the
implementation.
** {empty}
BITMAP_OPT_HASH_CACHE (0x4): :::
If present, the end of the bitmap file contains
`N` 32-bit name-hash values, one per object in the
pack/MIDX. The format and meaning of the name-hash is
described below.
** {empty}
BITMAP_OPT_LOOKUP_TABLE (0x10): :::
If present, the end of the bitmap file contains a table
containing a list of `N` <commit_pos, offset, xor_row>
triplets. The format and meaning of the table is described
below.
+
NOTE: Unlike the xor_offset used to compress an individual bitmap,
`xor_row` stores an *absolute* index into the lookup table, not a location
relative to the current entry.
4-byte entry count (network byte order): ::
The total count of entries (bitmapped commits) in this bitmap index.
20-byte checksum: ::
The SHA1 checksum of the pack/MIDX this bitmap index
belongs to.
* 4 EWAH bitmaps that act as type indexes
+
Type indexes are serialized after the hash cache in the shape
of four EWAH bitmaps stored consecutively (see Appendix A for
the serialization format of an EWAH bitmap).
+
There is a bitmap for each Git object type, stored in the following
order:
+
- Commits
- Trees
- Blobs
- Tags
+
In each bitmap, the `n`th bit is set to true if the `n`th object
in the packfile or multi-pack index is of that type.
+
The obvious consequence is that the OR of all 4 bitmaps will result
in a full set (all bits set), and the AND of all 4 bitmaps will
result in an empty bitmap (no bits set).
* N entries with compressed bitmaps, one for each indexed commit
+
Where `N` is the total number of entries in this bitmap index.
Each entry contains the following:
** {empty}
4-byte object position (network byte order): ::
The position **in the index for the packfile or
multi-pack index** where the bitmap for this commit is
found.
** {empty}
1-byte XOR-offset: ::
The xor offset used to compress this bitmap. For an entry
in position `x`, an XOR offset of `y` means that the actual
bitmap representing this commit is composed by XORing the
bitmap for this entry with the bitmap in entry `x-y` (i.e.
the bitmap `y` entries before this one).
+
NOTE: This compression can be recursive. In order to
XOR this entry with a previous one, the previous entry needs
to be decompressed first, and so on.
+
The hard-limit for this offset is 160 (an entry can only be
xor'ed against one of the 160 entries preceding it). This
number is always positive, and hence entries are always xor'ed
with **previous** bitmaps, not bitmaps that will come afterwards
in the index.
** {empty}
1-byte flags for this bitmap: ::
At the moment the only available flag is `0x1`, which hints
that this bitmap can be re-used when rebuilding bitmap indexes
for the repository.
** The compressed bitmap itself, see Appendix A.
* {empty}
TRAILER: ::
Trailing checksum of the preceding contents.
== Appendix A: Serialization format for an EWAH bitmap
Ewah bitmaps are serialized in the same protocol as the JAVAEWAH
library, making them backwards compatible with the JGit
implementation:
- 4-byte number of bits of the resulting UNCOMPRESSED bitmap
- 4-byte number of words of the COMPRESSED bitmap, when stored
- N x 8-byte words, as specified by the previous field
+
This is the actual content of the compressed bitmap.
- 4-byte position of the current RLW for the compressed
bitmap
All words are stored in network byte order for their corresponding
sizes.
The compressed bitmap is stored in a form of run-length encoding, as
follows. It consists of a concatenation of an arbitrary number of
chunks. Each chunk consists of one or more 64-bit words
H L_1 L_2 L_3 .... L_M
H is called RLW (run length word). It consists of (from lower to higher
order bits):
- 1 bit: the repeated bit B
- 32 bits: repetition count K (unsigned)
- 31 bits: literal word count M (unsigned)
The bitstream represented by the above chunk is then:
- K repetitions of B
- The bits stored in `L_1` through `L_M`. Within a word, bits at
lower order come earlier in the stream than those at higher
order.
The next word after `L_M` (if any) must again be a RLW, for the next
chunk. For efficient appending to the bitstream, the EWAH stores a
pointer to the last RLW in the stream.
== Appendix B: Optional Bitmap Sections
These sections may or may not be present in the `.bitmap` file; their
presence is indicated by the header flags section described above.
Name-hash cache
---------------
If the BITMAP_OPT_HASH_CACHE flag is set, the end of the bitmap contains
a cache of 32-bit values, one per object in the pack/MIDX. The value at
position `i` is the hash of the pathname at which the `i`th object
(counting in index or multi-pack index order) in the pack/MIDX can be found.
This can be fed into the delta heuristics to compare objects with similar
pathnames.
The hash algorithm used is:
hash = 0;
while ((c = *name++))
if (!isspace(c))
hash = (hash >> 2) + (c << 24);
Note that this hashing scheme is tied to the BITMAP_OPT_HASH_CACHE flag.
If implementations want to choose a different hashing scheme, they are
free to do so, but MUST allocate a new header flag (because comparing
hashes made under two different schemes would be pointless).
Commit lookup table
-------------------
If the BITMAP_OPT_LOOKUP_TABLE flag is set, the last `N * (4 + 8 + 4)`
bytes (preceding the name-hash cache and trailing hash) of the `.bitmap`
file contains a lookup table specifying the information needed to get
the desired bitmap from the entries without parsing previous unnecessary
bitmaps.
For a `.bitmap` containing `nr_entries` reachability bitmaps, the table
contains a list of `nr_entries` <commit_pos, offset, xor_row> triplets
(sorted in the ascending order of `commit_pos`). The content of the i'th
triplet is -
* {empty}
commit_pos (4 byte integer, network byte order): ::
It stores the object position of a commit (in the midx or pack
index).
* {empty}
offset (8 byte integer, network byte order): ::
The offset from which that commit's bitmap can be read.
* {empty}
xor_row (4 byte integer, network byte order): ::
The position of the triplet whose bitmap is used to compress
this one, or `0xffffffff` if no such bitmap exists.
Pseudo-merge bitmaps
--------------------
If the `BITMAP_OPT_PSEUDO_MERGES` flag is set, a variable number of
bytes (preceding the name-hash cache, commit lookup table, and trailing
checksum) of the `.bitmap` file is used to store pseudo-merge bitmaps.
For more information on what pseudo-merges are, why they are useful, and
how to configure them, see the information in linkgit:gitpacking[7].
=== File format
If enabled, pseudo-merge bitmaps are stored in an optional section at
the end of a `.bitmap` file. The format is as follows:
....
+-------------------------------------------+
| .bitmap File |
+-------------------------------------------+
| |
| Pseudo-merge bitmaps (Variable Length) |
| +---------------------------+ |
| | commits_bitmap (EWAH) | |
| +---------------------------+ |
| | merge_bitmap (EWAH) | |
| +---------------------------+ |
| |
+-------------------------------------------+
| |
| Lookup Table |
| +---------------------------+ |
| | commit_pos (4 bytes) | |
| +---------------------------+ |
| | offset (8 bytes) | |
| +------------+--------------+ |
| |
| Offset Cases: |
| ------------- |
| |
| 1. MSB Unset: single pseudo-merge bitmap |
| + offset to pseudo-merge bitmap |
| |
| 2. MSB Set: multiple pseudo-merges |
| + offset to extended lookup table |
| |
+-------------------------------------------+
| |
| Extended Lookup Table (Optional) |
| +----+----------+----------+----------+ |
| | N | Offset 1 | .... | Offset N | |
| +----+----------+----------+----------+ |
| | | 8 bytes | .... | 8 bytes | |
| +----+----------+----------+----------+ |
| |
+-------------------------------------------+
| |
| Pseudo-merge position table |
| +----+----------+----------+----------+ |
| | N | Offset 1 | .... | Offset N | |
| +----+----------+----------+----------+ |
| | | 8 bytes | .... | 8 bytes | |
| +----+----------+----------+----------+ |
| |
+-------------------------------------------+
| |
| Pseudo-merge Metadata |
| +-----------------------------------+ |
| | # pseudo-merges (4 bytes) | |
| +-----------------------------------+ |
| | # commits (4 bytes) | |
| +-----------------------------------+ |
| | Lookup offset (8 bytes) | |
| +-----------------------------------+ |
| | Extension size (8 bytes) | |
| +-----------------------------------+ |
| |
+-------------------------------------------+
....
* One or more pseudo-merge bitmaps, each containing:
** `commits_bitmap`, an EWAH-compressed bitmap describing the set of
commits included in the this psuedo-merge.
** `merge_bitmap`, an EWAH-compressed bitmap describing the union of
the set of objects reachable from all commits listed in the
`commits_bitmap`.
* A lookup table, mapping pseudo-merged commits to the pseudo-merges
they belong to. Entries appear in increasing order of each commit's
bit position. Each entry is 12 bytes wide, and is comprised of the
following:
** `commit_pos`, a 4-byte unsigned value (in network byte-order)
containing the bit position for this commit.
** `offset`, an 8-byte unsigned value (also in network byte-order)
containing either one of two possible offsets, depending on whether or
not the most-significant bit is set.
*** If unset (i.e. `offset & ((uint64_t)1<<63) == 0`), the offset
(relative to the beginning of the `.bitmap` file) at which the
pseudo-merge bitmap for this commit can be read. This indicates
only a single pseudo-merge bitmap contains this commit.
*** If set (i.e. `offset & ((uint64_t)1<<63) != 0`), the offset
(again relative to the beginning of the `.bitmap` file) at which
the extended offset table can be located describing the set of
pseudo-merge bitmaps which contain this commit. This indicates
that multiple pseudo-merge bitmaps contain this commit.
* An (optional) extended lookup table (written if and only if there is
at least one commit which appears in more than one pseudo-merge).
There are as many entries as commits which appear in multiple
pseudo-merges. Each entry contains the following:
** `N`, a 4-byte unsigned value equal to the number of pseudo-merges
which contain a given commit.
** An array of `N` 8-byte unsigned values, each of which is
interpreted as an offset (relative to the beginning of the
`.bitmap` file) at which a pseudo-merge bitmap for this commit can
be read. These values occur in no particular order.
* Positions for all pseudo-merges, each stored as an 8-byte unsigned
value (in network byte-order) containing the offset (relative to the
beginning of the `.bitmap` file) of each consecutive pseudo-merge.
* A 4-byte unsigned value (in network byte-order) equal to the number of
pseudo-merges.
* A 4-byte unsigned value (in network byte-order) equal to the number of
unique commits which appear in any pseudo-merge.
* An 8-byte unsigned value (in network byte-order) equal to the number
of bytes between the start of the pseudo-merge section and the
beginning of the lookup table.
* An 8-byte unsigned value (in network byte-order) equal to the number
of bytes in the pseudo-merge section (including this field).

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,224 @@
= Build Systems
The build system is the primary way for both developers and system integrators
to interact with the Git project. As such, being easy to use and extend for
those who are not directly developing Git itself is just as important as other
requirements we have on any potential build system.
This document outlines the different requirements that we have for the build
system and then compares available build systems using these criteria.
== Requirements
The following subsections present a list of requirements that we have for any
potential build system. Sections are sorted by decreasing priority.
=== Platform support
The build system must have support for all of our platforms that we continually
test against as outlined by our platform support policy. These platforms are:
- Linux
- Windows
- macOS
Furthermore, the build system should have support for the following platforms
that generally have somebody running test pipelines against regularly:
- AIX
- FreeBSD
- NetBSD
- NonStop
- OpenBSD
The platforms which must be supported by the tool should be aligned with our
[platform support policy](platform-support.txt).
=== Auto-detection of supported features
The build system must support auto-detection of features which are or aren't
available on the current platform. Platform maintainers should not be required
to manually configure the complete build.
Auto-detection of the following items is considered to be important:
- Check for the existence of headers.
- Check for the existence of libraries.
- Check for the existence of exectuables.
- Check for the runtime behavior of specific functions.
- Check for specific link order requirements when multiple libraries are
involved.
=== Ease of use
The build system should be both easy to use and easy to extend. While this is
naturally a subjective metric it is likely not controversial to say that some
build systems are considerably harder to use than others.
=== IDE support
The build system should integrate with well-known IDEs. Well-known IDEs include:
- Microsoft Visual Studio
- Visual Studio Code
- Xcode
There are four levels of support:
- Native integration into the IDE.
- Integration into the IDE via a plugin.
- Integration into the IDE via generating a project description with the build
system.
- No integration.
Native integration is preferable, but integration via either a plugin or by
generating a project description via the build system are considered feasible
alternatives.
Another important distinction is the level of integration. There are two
features that one generally wants to have:
- Integration of build targets.
- Automatic setup of features like code completion with detected build
dependencies.
The first bullet point is the bare minimum, but is not sufficient to be
considered proper integration.
=== Out-of-tree builds
The build system should support out-of-tree builds. Out-of-tree builds allow a
developer to configure multiple different build directories with different
configuration, e.g. one "debug" build and one "release" build.
=== Cross-platform builds
The build system should support cross-platform builds, e.g. building for arm on
an x86-64 host.
=== Language support
The following languages and toolchains are of relevance and should be supported
by the build system:
- C: the primary compiled language used by Git, must be supported. Relevant
toolchains are GCC, Clang and MSVC.
- Rust: candidate as a second compiled lanugage, should be supported. Relevant
toolchains is the LLVM-based rustc.
Built-in support for the respective languages is preferred over support that
needs to be wired up manually to avoid unnecessary complexity. Native support
includes the following features:
- Compiling objects.
- Dependency tracking.
- Detection of available features.
- Discovery of relevant toolchains.
- Linking libraries and executables.
- Templating placeholders in scripts.
=== Test integration
It should be possible to integrate tests into the build system such that it is
possible to build and test Git within the build system. Features which are nice
to have:
- Track build-time dependencies for respective tests. Unit tests have
different requirements than integration tests.
- Allow filtering of which tests to run.
- Allow running tests such that utilities like `test_pause` or `debug` work.
== Comparison
The following list of build systems are considered:
- GNU Make
- autoconf
- CMake
- Meson
=== GNU Make
- Platform support: ubitquitous on all platforms, but not well-integrated into Windows.
- Auto-detection: no built-in support for auto-detection of features.
- Ease of use: easy to use, but discovering available options is hard. Makefile
rules can quickly get out of hand once reaching a certain scope.
- IDE support: execution of Makefile targets is supported by many IDEs
- Out-of-tree builds: supported in theory, not wired up in practice.
- Cross-platform builds: supported in theory, not wired up in practice.
- Language support:
- C: Limited built-in support, many parts need to be wired up manually.
- Rust: No built-in support, needs to be wired up manually.
- Test integration: partially supported, many parts need to be wired up
manually.
=== autoconf
- Platform support: ubiquitous on all platforms, but not well-integrated into Windows.
- Auto-detection: supported.
- Ease of use: easy to use, discovering available options is comparatively
easy. The autoconf syntax is prohibitively hard to extend though due to its
complex set of interacting files and the hard-to-understand M4 language.
- IDE support: no integration into IDEs at generation time. The generated
Makefiles have the same level of support as GNU Make.
- Out-of-tree builds: supported in theory, not wired up in practice.
- Cross-platform builds: supported.
- Language support:
- C: Limited built-in support, many parts need to be wired up manually.
- Rust: No built-in support, needs to be wired up manually.
- Test integration: partially supported, many parts need to be wired up
manually.
=== CMake
- Platform support: not as extensive as GNU Make or autoconf, but all major
platforms are supported.
- AIX
- Cygwin
- FreeBSD
- Linux
- OpenBSD
- Solaris
- Windows
- macOS
- Ease of use: easy to use, discovering available options is not always
trivial. The scripting language used by CMake is somewhat cumbersome to use,
but extending CMake build instructions is doable.
- IDE support: natively integrated into Microsoft Visual Studio. Can generate
project descriptions for Xcode. An extension is available for Visual Studio
Code. Many other IDEs have plugins for CMake.
- Out-of-tree builds: supported.
- Cross-platform builds: supported.
- Language support:
- C: Supported for GCC, Clang, MSVC and other toolchains.
- Rust: No built-in support, needs to be wired up manually.
- Test integration: supported, even though test dependencies are a bit
cumbersome to use via "test fixtures". Interactive test runs are not
supported.
=== Meson
- Platform: not as extensive as GNU Make or autoconf, but all major platforms
and some smaller ones are supported.
- AIX
- Cygwin
- DragonflyBSD
- FreeBSD
- Haiku
- Linux
- NetBSD
- OpenBSD
- Solaris
- Windows
- macOS
- Ease of use: easy to use, discovering available options is easy. The
scripting language is straight-forward to use.
- IDE support: Supports generating build instructions for Xcode and Microsoft
Visual Studio, a plugin exists for Visual Studio Code.
- Out-of-tree builds: supported.
- Cross-platform builds: supported.
- Language support:
- C: Supported for GCC, Clang, MSVC and other toolchains.
- Rust: Supported for rustc.
- Test integration: supported. Interactive tests are supported starting with
Meson 1.5.0 via the `--interactive` flag.

View File

@@ -0,0 +1,944 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="en">
<head>
<meta charset="UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=edge"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<meta name="generator" content="Asciidoctor 2.0.23"/>
<title>Build Systems</title>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"/>
<style>
/*! Asciidoctor default stylesheet | MIT License | https://asciidoctor.org */
/* Uncomment the following line when using as a custom stylesheet */
/* @import "https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"; */
html{font-family:sans-serif;-webkit-text-size-adjust:100%}
a{background:none}
a:focus{outline:thin dotted}
a:active,a:hover{outline:0}
h1{font-size:2em;margin:.67em 0}
b,strong{font-weight:bold}
abbr{font-size:.9em}
abbr[title]{cursor:help;border-bottom:1px dotted #dddddf;text-decoration:none}
dfn{font-style:italic}
hr{height:0}
mark{background:#ff0;color:#000}
code,kbd,pre,samp{font-family:monospace;font-size:1em}
pre{white-space:pre-wrap}
q{quotes:"\201C" "\201D" "\2018" "\2019"}
small{font-size:80%}
sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline}
sup{top:-.5em}
sub{bottom:-.25em}
img{border:0}
svg:not(:root){overflow:hidden}
figure{margin:0}
audio,video{display:inline-block}
audio:not([controls]){display:none;height:0}
fieldset{border:1px solid silver;margin:0 2px;padding:.35em .625em .75em}
legend{border:0;padding:0}
button,input,select,textarea{font-family:inherit;font-size:100%;margin:0}
button,input{line-height:normal}
button,select{text-transform:none}
button,html input[type=button],input[type=reset],input[type=submit]{-webkit-appearance:button;cursor:pointer}
button[disabled],html input[disabled]{cursor:default}
input[type=checkbox],input[type=radio]{padding:0}
button::-moz-focus-inner,input::-moz-focus-inner{border:0;padding:0}
textarea{overflow:auto;vertical-align:top}
table{border-collapse:collapse;border-spacing:0}
*,::before,::after{box-sizing:border-box}
html,body{font-size:100%}
body{background:#fff;color:rgba(0,0,0,.8);padding:0;margin:0;font-family:"Noto Serif","DejaVu Serif",serif;line-height:1;position:relative;cursor:auto;-moz-tab-size:4;-o-tab-size:4;tab-size:4;word-wrap:anywhere;-moz-osx-font-smoothing:grayscale;-webkit-font-smoothing:antialiased}
a:hover{cursor:pointer}
img,object,embed{max-width:100%;height:auto}
object,embed{height:100%}
img{-ms-interpolation-mode:bicubic}
.left{float:left!important}
.right{float:right!important}
.text-left{text-align:left!important}
.text-right{text-align:right!important}
.text-center{text-align:center!important}
.text-justify{text-align:justify!important}
.hide{display:none}
img,object,svg{display:inline-block;vertical-align:middle}
textarea{height:auto;min-height:50px}
select{width:100%}
.subheader,.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{line-height:1.45;color:#7a2518;font-weight:400;margin-top:0;margin-bottom:.25em}
div,dl,dt,dd,ul,ol,li,h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6,pre,form,p,blockquote,th,td{margin:0;padding:0}
a{color:#2156a5;text-decoration:underline;line-height:inherit}
a:hover,a:focus{color:#1d4b8f}
a img{border:0}
p{line-height:1.6;margin-bottom:1.25em;text-rendering:optimizeLegibility}
p aside{font-size:.875em;line-height:1.35;font-style:italic}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{font-family:"Open Sans","DejaVu Sans",sans-serif;font-weight:300;font-style:normal;color:#ba3925;text-rendering:optimizeLegibility;margin-top:1em;margin-bottom:.5em;line-height:1.0125em}
h1 small,h2 small,h3 small,#toctitle small,.sidebarblock>.content>.title small,h4 small,h5 small,h6 small{font-size:60%;color:#e99b8f;line-height:0}
h1{font-size:2.125em}
h2{font-size:1.6875em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.375em}
h4,h5{font-size:1.125em}
h6{font-size:1em}
hr{border:solid #dddddf;border-width:1px 0 0;clear:both;margin:1.25em 0 1.1875em}
em,i{font-style:italic;line-height:inherit}
strong,b{font-weight:bold;line-height:inherit}
small{font-size:60%;line-height:inherit}
code{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:400;color:rgba(0,0,0,.9)}
ul,ol,dl{line-height:1.6;margin-bottom:1.25em;list-style-position:outside;font-family:inherit}
ul,ol{margin-left:1.5em}
ul li ul,ul li ol{margin-left:1.25em;margin-bottom:0}
ul.circle{list-style-type:circle}
ul.disc{list-style-type:disc}
ul.square{list-style-type:square}
ul.circle ul:not([class]),ul.disc ul:not([class]),ul.square ul:not([class]){list-style:inherit}
ol li ul,ol li ol{margin-left:1.25em;margin-bottom:0}
dl dt{margin-bottom:.3125em;font-weight:bold}
dl dd{margin-bottom:1.25em}
blockquote{margin:0 0 1.25em;padding:.5625em 1.25em 0 1.1875em;border-left:1px solid #ddd}
blockquote,blockquote p{line-height:1.6;color:rgba(0,0,0,.85)}
@media screen and (min-width:768px){h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2}
h1{font-size:2.75em}
h2{font-size:2.3125em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.6875em}
h4{font-size:1.4375em}}
table{background:#fff;margin-bottom:1.25em;border:1px solid #dedede;word-wrap:normal}
table thead,table tfoot{background:#f7f8f7}
table thead tr th,table thead tr td,table tfoot tr th,table tfoot tr td{padding:.5em .625em .625em;font-size:inherit;color:rgba(0,0,0,.8);text-align:left}
table tr th,table tr td{padding:.5625em .625em;font-size:inherit;color:rgba(0,0,0,.8)}
table tr.even,table tr.alt{background:#f8f8f7}
table thead tr th,table tfoot tr th,table tbody tr td,table tr td,table tfoot tr td{line-height:1.6}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2;word-spacing:-.05em}
h1 strong,h2 strong,h3 strong,#toctitle strong,.sidebarblock>.content>.title strong,h4 strong,h5 strong,h6 strong{font-weight:400}
.center{margin-left:auto;margin-right:auto}
.stretch{width:100%}
.clearfix::before,.clearfix::after,.float-group::before,.float-group::after{content:" ";display:table}
.clearfix::after,.float-group::after{clear:both}
:not(pre).nobreak{word-wrap:normal}
:not(pre).nowrap{white-space:nowrap}
:not(pre).pre-wrap{white-space:pre-wrap}
:not(pre):not([class^=L])>code{font-size:.9375em;font-style:normal!important;letter-spacing:0;padding:.1em .5ex;word-spacing:-.15em;background:#f7f7f8;border-radius:4px;line-height:1.45;text-rendering:optimizeSpeed}
pre{color:rgba(0,0,0,.9);font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;line-height:1.45;text-rendering:optimizeSpeed}
pre code,pre pre{color:inherit;font-size:inherit;line-height:inherit}
pre>code{display:block}
pre.nowrap,pre.nowrap pre{white-space:pre;word-wrap:normal}
em em{font-style:normal}
strong strong{font-weight:400}
.keyseq{color:rgba(51,51,51,.8)}
kbd{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;display:inline-block;color:rgba(0,0,0,.8);font-size:.65em;line-height:1.45;background:#f7f7f7;border:1px solid #ccc;border-radius:3px;box-shadow:0 1px 0 rgba(0,0,0,.2),inset 0 0 0 .1em #fff;margin:0 .15em;padding:.2em .5em;vertical-align:middle;position:relative;top:-.1em;white-space:nowrap}
.keyseq kbd:first-child{margin-left:0}
.keyseq kbd:last-child{margin-right:0}
.menuseq,.menuref{color:#000}
.menuseq b:not(.caret),.menuref{font-weight:inherit}
.menuseq{word-spacing:-.02em}
.menuseq b.caret{font-size:1.25em;line-height:.8}
.menuseq i.caret{font-weight:bold;text-align:center;width:.45em}
b.button::before,b.button::after{position:relative;top:-1px;font-weight:400}
b.button::before{content:"[";padding:0 3px 0 2px}
b.button::after{content:"]";padding:0 2px 0 3px}
p a>code:hover{color:rgba(0,0,0,.9)}
#header,#content,#footnotes,#footer{width:100%;margin:0 auto;max-width:62.5em;*zoom:1;position:relative;padding-left:.9375em;padding-right:.9375em}
#header::before,#header::after,#content::before,#content::after,#footnotes::before,#footnotes::after,#footer::before,#footer::after{content:" ";display:table}
#header::after,#content::after,#footnotes::after,#footer::after{clear:both}
#content{margin-top:1.25em}
#content::before{content:none}
#header>h1:first-child{color:rgba(0,0,0,.85);margin-top:2.25rem;margin-bottom:0}
#header>h1:first-child+#toc{margin-top:8px;border-top:1px solid #dddddf}
#header>h1:only-child{border-bottom:1px solid #dddddf;padding-bottom:8px}
#header .details{border-bottom:1px solid #dddddf;line-height:1.45;padding-top:.25em;padding-bottom:.25em;padding-left:.25em;color:rgba(0,0,0,.6);display:flex;flex-flow:row wrap}
#header .details span:first-child{margin-left:-.125em}
#header .details span.email a{color:rgba(0,0,0,.85)}
#header .details br{display:none}
#header .details br+span::before{content:"\00a0\2013\00a0"}
#header .details br+span.author::before{content:"\00a0\22c5\00a0";color:rgba(0,0,0,.85)}
#header .details br+span#revremark::before{content:"\00a0|\00a0"}
#header #revnumber{text-transform:capitalize}
#header #revnumber::after{content:"\00a0"}
#content>h1:first-child:not([class]){color:rgba(0,0,0,.85);border-bottom:1px solid #dddddf;padding-bottom:8px;margin-top:0;padding-top:1rem;margin-bottom:1.25rem}
#toc{border-bottom:1px solid #e7e7e9;padding-bottom:.5em}
#toc>ul{margin-left:.125em}
#toc ul.sectlevel0>li>a{font-style:italic}
#toc ul.sectlevel0 ul.sectlevel1{margin:.5em 0}
#toc ul{font-family:"Open Sans","DejaVu Sans",sans-serif;list-style-type:none}
#toc li{line-height:1.3334;margin-top:.3334em}
#toc a{text-decoration:none}
#toc a:active{text-decoration:underline}
#toctitle{color:#7a2518;font-size:1.2em}
@media screen and (min-width:768px){#toctitle{font-size:1.375em}
body.toc2{padding-left:15em;padding-right:0}
body.toc2 #header>h1:nth-last-child(2){border-bottom:1px solid #dddddf;padding-bottom:8px}
#toc.toc2{margin-top:0!important;background:#f8f8f7;position:fixed;width:15em;left:0;top:0;border-right:1px solid #e7e7e9;border-top-width:0!important;border-bottom-width:0!important;z-index:1000;padding:1.25em 1em;height:100%;overflow:auto}
#toc.toc2 #toctitle{margin-top:0;margin-bottom:.8rem;font-size:1.2em}
#toc.toc2>ul{font-size:.9em;margin-bottom:0}
#toc.toc2 ul ul{margin-left:0;padding-left:1em}
#toc.toc2 ul.sectlevel0 ul.sectlevel1{padding-left:0;margin-top:.5em;margin-bottom:.5em}
body.toc2.toc-right{padding-left:0;padding-right:15em}
body.toc2.toc-right #toc.toc2{border-right-width:0;border-left:1px solid #e7e7e9;left:auto;right:0}}
@media screen and (min-width:1280px){body.toc2{padding-left:20em;padding-right:0}
#toc.toc2{width:20em}
#toc.toc2 #toctitle{font-size:1.375em}
#toc.toc2>ul{font-size:.95em}
#toc.toc2 ul ul{padding-left:1.25em}
body.toc2.toc-right{padding-left:0;padding-right:20em}}
#content #toc{border:1px solid #e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;border-radius:4px}
#content #toc>:first-child{margin-top:0}
#content #toc>:last-child{margin-bottom:0}
#footer{max-width:none;background:rgba(0,0,0,.8);padding:1.25em}
#footer-text{color:hsla(0,0%,100%,.8);line-height:1.44}
#content{margin-bottom:.625em}
.sect1{padding-bottom:.625em}
@media screen and (min-width:768px){#content{margin-bottom:1.25em}
.sect1{padding-bottom:1.25em}}
.sect1:last-child{padding-bottom:0}
.sect1+.sect1{border-top:1px solid #e7e7e9}
#content h1>a.anchor,h2>a.anchor,h3>a.anchor,#toctitle>a.anchor,.sidebarblock>.content>.title>a.anchor,h4>a.anchor,h5>a.anchor,h6>a.anchor{position:absolute;z-index:1001;width:1.5ex;margin-left:-1.5ex;display:block;text-decoration:none!important;visibility:hidden;text-align:center;font-weight:400}
#content h1>a.anchor::before,h2>a.anchor::before,h3>a.anchor::before,#toctitle>a.anchor::before,.sidebarblock>.content>.title>a.anchor::before,h4>a.anchor::before,h5>a.anchor::before,h6>a.anchor::before{content:"\00A7";font-size:.85em;display:block;padding-top:.1em}
#content h1:hover>a.anchor,#content h1>a.anchor:hover,h2:hover>a.anchor,h2>a.anchor:hover,h3:hover>a.anchor,#toctitle:hover>a.anchor,.sidebarblock>.content>.title:hover>a.anchor,h3>a.anchor:hover,#toctitle>a.anchor:hover,.sidebarblock>.content>.title>a.anchor:hover,h4:hover>a.anchor,h4>a.anchor:hover,h5:hover>a.anchor,h5>a.anchor:hover,h6:hover>a.anchor,h6>a.anchor:hover{visibility:visible}
#content h1>a.link,h2>a.link,h3>a.link,#toctitle>a.link,.sidebarblock>.content>.title>a.link,h4>a.link,h5>a.link,h6>a.link{color:#ba3925;text-decoration:none}
#content h1>a.link:hover,h2>a.link:hover,h3>a.link:hover,#toctitle>a.link:hover,.sidebarblock>.content>.title>a.link:hover,h4>a.link:hover,h5>a.link:hover,h6>a.link:hover{color:#a53221}
details,.audioblock,.imageblock,.literalblock,.listingblock,.stemblock,.videoblock{margin-bottom:1.25em}
details{margin-left:1.25rem}
details>summary{cursor:pointer;display:block;position:relative;line-height:1.6;margin-bottom:.625rem;outline:none;-webkit-tap-highlight-color:transparent}
details>summary::-webkit-details-marker{display:none}
details>summary::before{content:"";border:solid transparent;border-left:solid;border-width:.3em 0 .3em .5em;position:absolute;top:.5em;left:-1.25rem;transform:translateX(15%)}
details[open]>summary::before{border:solid transparent;border-top:solid;border-width:.5em .3em 0;transform:translateY(15%)}
details>summary::after{content:"";width:1.25rem;height:1em;position:absolute;top:.3em;left:-1.25rem}
.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{text-rendering:optimizeLegibility;text-align:left;font-family:"Noto Serif","DejaVu Serif",serif;font-size:1rem;font-style:italic}
table.tableblock.fit-content>caption.title{white-space:nowrap;width:0}
.paragraph.lead>p,#preamble>.sectionbody>[class=paragraph]:first-of-type p{font-size:1.21875em;line-height:1.6;color:rgba(0,0,0,.85)}
.admonitionblock>table{border-collapse:separate;border:0;background:none;width:100%}
.admonitionblock>table td.icon{text-align:center;width:80px}
.admonitionblock>table td.icon img{max-width:none}
.admonitionblock>table td.icon .title{font-weight:bold;font-family:"Open Sans","DejaVu Sans",sans-serif;text-transform:uppercase}
.admonitionblock>table td.content{padding-left:1.125em;padding-right:1.25em;border-left:1px solid #dddddf;color:rgba(0,0,0,.6);word-wrap:anywhere}
.admonitionblock>table td.content>:last-child>:last-child{margin-bottom:0}
.exampleblock>.content{border:1px solid #e6e6e6;margin-bottom:1.25em;padding:1.25em;background:#fff;border-radius:4px}
.sidebarblock{border:1px solid #dbdbd6;margin-bottom:1.25em;padding:1.25em;background:#f3f3f2;border-radius:4px}
.sidebarblock>.content>.title{color:#7a2518;margin-top:0;text-align:center}
.exampleblock>.content>:first-child,.sidebarblock>.content>:first-child{margin-top:0}
.exampleblock>.content>:last-child,.exampleblock>.content>:last-child>:last-child,.exampleblock>.content .olist>ol>li:last-child>:last-child,.exampleblock>.content .ulist>ul>li:last-child>:last-child,.exampleblock>.content .qlist>ol>li:last-child>:last-child,.sidebarblock>.content>:last-child,.sidebarblock>.content>:last-child>:last-child,.sidebarblock>.content .olist>ol>li:last-child>:last-child,.sidebarblock>.content .ulist>ul>li:last-child>:last-child,.sidebarblock>.content .qlist>ol>li:last-child>:last-child{margin-bottom:0}
.literalblock pre,.listingblock>.content>pre{border-radius:4px;overflow-x:auto;padding:1em;font-size:.8125em}
@media screen and (min-width:768px){.literalblock pre,.listingblock>.content>pre{font-size:.90625em}}
@media screen and (min-width:1280px){.literalblock pre,.listingblock>.content>pre{font-size:1em}}
.literalblock pre,.listingblock>.content>pre:not(.highlight),.listingblock>.content>pre[class=highlight],.listingblock>.content>pre[class^="highlight "]{background:#f7f7f8}
.literalblock.output pre{color:#f7f7f8;background:rgba(0,0,0,.9)}
.listingblock>.content{position:relative}
.listingblock code[data-lang]::before{display:none;content:attr(data-lang);position:absolute;font-size:.75em;top:.425rem;right:.5rem;line-height:1;text-transform:uppercase;color:inherit;opacity:.5}
.listingblock:hover code[data-lang]::before{display:block}
.listingblock.terminal pre .command::before{content:attr(data-prompt);padding-right:.5em;color:inherit;opacity:.5}
.listingblock.terminal pre .command:not([data-prompt])::before{content:"$"}
.listingblock pre.highlightjs{padding:0}
.listingblock pre.highlightjs>code{padding:1em;border-radius:4px}
.listingblock pre.prettyprint{border-width:0}
.prettyprint{background:#f7f7f8}
pre.prettyprint .linenums{line-height:1.45;margin-left:2em}
pre.prettyprint li{background:none;list-style-type:inherit;padding-left:0}
pre.prettyprint li code[data-lang]::before{opacity:1}
pre.prettyprint li:not(:first-child) code[data-lang]::before{display:none}
table.linenotable{border-collapse:separate;border:0;margin-bottom:0;background:none}
table.linenotable td[class]{color:inherit;vertical-align:top;padding:0;line-height:inherit;white-space:normal}
table.linenotable td.code{padding-left:.75em}
table.linenotable td.linenos,pre.pygments .linenos{border-right:1px solid;opacity:.35;padding-right:.5em;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}
pre.pygments span.linenos{display:inline-block;margin-right:.75em}
.quoteblock{margin:0 1em 1.25em 1.5em;display:table}
.quoteblock:not(.excerpt)>.title{margin-left:-1.5em;margin-bottom:.75em}
.quoteblock blockquote,.quoteblock p{color:rgba(0,0,0,.85);font-size:1.15rem;line-height:1.75;word-spacing:.1em;letter-spacing:0;font-style:italic;text-align:justify}
.quoteblock blockquote{margin:0;padding:0;border:0}
.quoteblock blockquote::before{content:"\201c";float:left;font-size:2.75em;font-weight:bold;line-height:.6em;margin-left:-.6em;color:#7a2518;text-shadow:0 1px 2px rgba(0,0,0,.1)}
.quoteblock blockquote>.paragraph:last-child p{margin-bottom:0}
.quoteblock .attribution{margin-top:.75em;margin-right:.5ex;text-align:right}
.verseblock{margin:0 1em 1.25em}
.verseblock pre{font-family:"Open Sans","DejaVu Sans",sans-serif;font-size:1.15rem;color:rgba(0,0,0,.85);font-weight:300;text-rendering:optimizeLegibility}
.verseblock pre strong{font-weight:400}
.verseblock .attribution{margin-top:1.25rem;margin-left:.5ex}
.quoteblock .attribution,.verseblock .attribution{font-size:.9375em;line-height:1.45;font-style:italic}
.quoteblock .attribution br,.verseblock .attribution br{display:none}
.quoteblock .attribution cite,.verseblock .attribution cite{display:block;letter-spacing:-.025em;color:rgba(0,0,0,.6)}
.quoteblock.abstract blockquote::before,.quoteblock.excerpt blockquote::before,.quoteblock .quoteblock blockquote::before{display:none}
.quoteblock.abstract blockquote,.quoteblock.abstract p,.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{line-height:1.6;word-spacing:0}
.quoteblock.abstract{margin:0 1em 1.25em;display:block}
.quoteblock.abstract>.title{margin:0 0 .375em;font-size:1.15em;text-align:center}
.quoteblock.excerpt>blockquote,.quoteblock .quoteblock{padding:0 0 .25em 1em;border-left:.25em solid #dddddf}
.quoteblock.excerpt,.quoteblock .quoteblock{margin-left:0}
.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{color:inherit;font-size:1.0625rem}
.quoteblock.excerpt .attribution,.quoteblock .quoteblock .attribution{color:inherit;font-size:.85rem;text-align:left;margin-right:0}
p.tableblock:last-child{margin-bottom:0}
td.tableblock>.content{margin-bottom:1.25em;word-wrap:anywhere}
td.tableblock>.content>:last-child{margin-bottom:-1.25em}
table.tableblock,th.tableblock,td.tableblock{border:0 solid #dedede}
table.grid-all>*>tr>*{border-width:1px}
table.grid-cols>*>tr>*{border-width:0 1px}
table.grid-rows>*>tr>*{border-width:1px 0}
table.frame-all{border-width:1px}
table.frame-ends{border-width:1px 0}
table.frame-sides{border-width:0 1px}
table.frame-none>colgroup+*>:first-child>*,table.frame-sides>colgroup+*>:first-child>*{border-top-width:0}
table.frame-none>:last-child>:last-child>*,table.frame-sides>:last-child>:last-child>*{border-bottom-width:0}
table.frame-none>*>tr>:first-child,table.frame-ends>*>tr>:first-child{border-left-width:0}
table.frame-none>*>tr>:last-child,table.frame-ends>*>tr>:last-child{border-right-width:0}
table.stripes-all>*>tr,table.stripes-odd>*>tr:nth-of-type(odd),table.stripes-even>*>tr:nth-of-type(even),table.stripes-hover>*>tr:hover{background:#f8f8f7}
th.halign-left,td.halign-left{text-align:left}
th.halign-right,td.halign-right{text-align:right}
th.halign-center,td.halign-center{text-align:center}
th.valign-top,td.valign-top{vertical-align:top}
th.valign-bottom,td.valign-bottom{vertical-align:bottom}
th.valign-middle,td.valign-middle{vertical-align:middle}
table thead th,table tfoot th{font-weight:bold}
tbody tr th{background:#f7f8f7}
tbody tr th,tbody tr th p,tfoot tr th,tfoot tr th p{color:rgba(0,0,0,.8);font-weight:bold}
p.tableblock>code:only-child{background:none;padding:0}
p.tableblock{font-size:1em}
ol{margin-left:1.75em}
ul li ol{margin-left:1.5em}
dl dd{margin-left:1.125em}
dl dd:last-child,dl dd:last-child>:last-child{margin-bottom:0}
li p,ul dd,ol dd,.olist .olist,.ulist .ulist,.ulist .olist,.olist .ulist{margin-bottom:.625em}
ul.checklist,ul.none,ol.none,ul.no-bullet,ol.no-bullet,ol.unnumbered,ul.unstyled,ol.unstyled{list-style-type:none}
ul.no-bullet,ol.no-bullet,ol.unnumbered{margin-left:.625em}
ul.unstyled,ol.unstyled{margin-left:0}
li>p:empty:only-child::before{content:"";display:inline-block}
ul.checklist>li>p:first-child{margin-left:-1em}
ul.checklist>li>p:first-child>.fa-square-o:first-child,ul.checklist>li>p:first-child>.fa-check-square-o:first-child{width:1.25em;font-size:.8em;position:relative;bottom:.125em}
ul.checklist>li>p:first-child>input[type=checkbox]:first-child{margin-right:.25em}
ul.inline{display:flex;flex-flow:row wrap;list-style:none;margin:0 0 .625em -1.25em}
ul.inline>li{margin-left:1.25em}
.unstyled dl dt{font-weight:400;font-style:normal}
ol.arabic{list-style-type:decimal}
ol.decimal{list-style-type:decimal-leading-zero}
ol.loweralpha{list-style-type:lower-alpha}
ol.upperalpha{list-style-type:upper-alpha}
ol.lowerroman{list-style-type:lower-roman}
ol.upperroman{list-style-type:upper-roman}
ol.lowergreek{list-style-type:lower-greek}
.hdlist>table,.colist>table{border:0;background:none}
.hdlist>table>tbody>tr,.colist>table>tbody>tr{background:none}
td.hdlist1,td.hdlist2{vertical-align:top;padding:0 .625em}
td.hdlist1{font-weight:bold;padding-bottom:1.25em}
td.hdlist2{word-wrap:anywhere}
.literalblock+.colist,.listingblock+.colist{margin-top:-.5em}
.colist td:not([class]):first-child{padding:.4em .75em 0;line-height:1;vertical-align:top}
.colist td:not([class]):first-child img{max-width:none}
.colist td:not([class]):last-child{padding:.25em 0}
.thumb,.th{line-height:0;display:inline-block;border:4px solid #fff;box-shadow:0 0 0 1px #ddd}
.imageblock.left{margin:.25em .625em 1.25em 0}
.imageblock.right{margin:.25em 0 1.25em .625em}
.imageblock>.title{margin-bottom:0}
.imageblock.thumb,.imageblock.th{border-width:6px}
.imageblock.thumb>.title,.imageblock.th>.title{padding:0 .125em}
.image.left,.image.right{margin-top:.25em;margin-bottom:.25em;display:inline-block;line-height:0}
.image.left{margin-right:.625em}
.image.right{margin-left:.625em}
a.image{text-decoration:none;display:inline-block}
a.image object{pointer-events:none}
sup.footnote,sup.footnoteref{font-size:.875em;position:static;vertical-align:super}
sup.footnote a,sup.footnoteref a{text-decoration:none}
sup.footnote a:active,sup.footnoteref a:active,#footnotes .footnote a:first-of-type:active{text-decoration:underline}
#footnotes{padding-top:.75em;padding-bottom:.75em;margin-bottom:.625em}
#footnotes hr{width:20%;min-width:6.25em;margin:-.25em 0 .75em;border-width:1px 0 0}
#footnotes .footnote{padding:0 .375em 0 .225em;line-height:1.3334;font-size:.875em;margin-left:1.2em;margin-bottom:.2em}
#footnotes .footnote a:first-of-type{font-weight:bold;text-decoration:none;margin-left:-1.05em}
#footnotes .footnote:last-of-type{margin-bottom:0}
#content #footnotes{margin-top:-.625em;margin-bottom:0;padding:.75em 0}
div.unbreakable{page-break-inside:avoid}
.big{font-size:larger}
.small{font-size:smaller}
.underline{text-decoration:underline}
.overline{text-decoration:overline}
.line-through{text-decoration:line-through}
.aqua{color:#00bfbf}
.aqua-background{background:#00fafa}
.black{color:#000}
.black-background{background:#000}
.blue{color:#0000bf}
.blue-background{background:#0000fa}
.fuchsia{color:#bf00bf}
.fuchsia-background{background:#fa00fa}
.gray{color:#606060}
.gray-background{background:#7d7d7d}
.green{color:#006000}
.green-background{background:#007d00}
.lime{color:#00bf00}
.lime-background{background:#00fa00}
.maroon{color:#600000}
.maroon-background{background:#7d0000}
.navy{color:#000060}
.navy-background{background:#00007d}
.olive{color:#606000}
.olive-background{background:#7d7d00}
.purple{color:#600060}
.purple-background{background:#7d007d}
.red{color:#bf0000}
.red-background{background:#fa0000}
.silver{color:#909090}
.silver-background{background:#bcbcbc}
.teal{color:#006060}
.teal-background{background:#007d7d}
.white{color:#bfbfbf}
.white-background{background:#fafafa}
.yellow{color:#bfbf00}
.yellow-background{background:#fafa00}
span.icon>.fa{cursor:default}
a span.icon>.fa{cursor:inherit}
.admonitionblock td.icon [class^="fa icon-"]{font-size:2.5em;text-shadow:1px 1px 2px rgba(0,0,0,.5);cursor:default}
.admonitionblock td.icon .icon-note::before{content:"\f05a";color:#19407c}
.admonitionblock td.icon .icon-tip::before{content:"\f0eb";text-shadow:1px 1px 2px rgba(155,155,0,.8);color:#111}
.admonitionblock td.icon .icon-warning::before{content:"\f071";color:#bf6900}
.admonitionblock td.icon .icon-caution::before{content:"\f06d";color:#bf3400}
.admonitionblock td.icon .icon-important::before{content:"\f06a";color:#bf0000}
.conum[data-value]{display:inline-block;color:#fff!important;background:rgba(0,0,0,.8);border-radius:50%;text-align:center;font-size:.75em;width:1.67em;height:1.67em;line-height:1.67em;font-family:"Open Sans","DejaVu Sans",sans-serif;font-style:normal;font-weight:bold}
.conum[data-value] *{color:#fff!important}
.conum[data-value]+b{display:none}
.conum[data-value]::after{content:attr(data-value)}
pre .conum[data-value]{position:relative;top:-.125em}
b.conum *{color:inherit!important}
.conum:not([data-value]):empty{display:none}
dt,th.tableblock,td.content,div.footnote{text-rendering:optimizeLegibility}
h1,h2,p,td.content,span.alt,summary{letter-spacing:-.01em}
p strong,td.content strong,div.footnote strong{letter-spacing:-.005em}
p,blockquote,dt,td.content,td.hdlist1,span.alt,summary{font-size:1.0625rem}
p{margin-bottom:1.25rem}
.sidebarblock p,.sidebarblock dt,.sidebarblock td.content,p.tableblock{font-size:1em}
.exampleblock>.content{background:#fffef7;border-color:#e0e0dc;box-shadow:0 1px 4px #e0e0dc}
.print-only{display:none!important}
@page{margin:1.25cm .75cm}
@media print{*{box-shadow:none!important;text-shadow:none!important}
html{font-size:80%}
a{color:inherit!important;text-decoration:underline!important}
a.bare,a[href^="#"],a[href^="mailto:"]{text-decoration:none!important}
a[href^="http:"]:not(.bare)::after,a[href^="https:"]:not(.bare)::after{content:"(" attr(href) ")";display:inline-block;font-size:.875em;padding-left:.25em}
abbr[title]{border-bottom:1px dotted}
abbr[title]::after{content:" (" attr(title) ")"}
pre,blockquote,tr,img,object,svg{page-break-inside:avoid}
thead{display:table-header-group}
svg{max-width:100%}
p,blockquote,dt,td.content{font-size:1em;orphans:3;widows:3}
h2,h3,#toctitle,.sidebarblock>.content>.title{page-break-after:avoid}
#header,#content,#footnotes,#footer{max-width:none}
#toc,.sidebarblock,.exampleblock>.content{background:none!important}
#toc{border-bottom:1px solid #dddddf!important;padding-bottom:0!important}
body.book #header{text-align:center}
body.book #header>h1:first-child{border:0!important;margin:2.5em 0 1em}
body.book #header .details{border:0!important;display:block;padding:0!important}
body.book #header .details span:first-child{margin-left:0!important}
body.book #header .details br{display:block}
body.book #header .details br+span::before{content:none!important}
body.book #toc{border:0!important;text-align:left!important;padding:0!important;margin:0!important}
body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-break-before:always}
.listingblock code[data-lang]::before{display:block}
#footer{padding:0 .9375em}
.hide-on-print{display:none!important}
.print-only{display:block!important}
.hide-for-print{display:none!important}
.show-for-print{display:inherit!important}}
@media amzn-kf8,print{#header>h1:first-child{margin-top:1.25rem}
.sect1{padding:0!important}
.sect1+.sect1{border:0}
#footer{background:none}
#footer-text{color:rgba(0,0,0,.6);font-size:.9em}}
@media amzn-kf8{#header,#content,#footnotes,#footer{padding:0}}
</style>
</head>
<body class="article">
<div id="header">
<h1>Build Systems</h1>
</div>
<div id="content">
<div id="preamble">
<div class="sectionbody">
<div class="paragraph">
<p>The build system is the primary way for both developers and system integrators
to interact with the Git project. As such, being easy to use and extend for
those who are not directly developing Git itself is just as important as other
requirements we have on any potential build system.</p>
</div>
<div class="paragraph">
<p>This document outlines the different requirements that we have for the build
system and then compares available build systems using these criteria.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_requirements">Requirements</h2>
<div class="sectionbody">
<div class="paragraph">
<p>The following subsections present a list of requirements that we have for any
potential build system. Sections are sorted by decreasing priority.</p>
</div>
<div class="sect2">
<h3 id="_platform_support">Platform support</h3>
<div class="paragraph">
<p>The build system must have support for all of our platforms that we continually
test against as outlined by our platform support policy. These platforms are:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Linux</p>
</li>
<li>
<p>Windows</p>
</li>
<li>
<p>macOS</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Furthermore, the build system should have support for the following platforms
that generally have somebody running test pipelines against regularly:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>AIX</p>
</li>
<li>
<p>FreeBSD</p>
</li>
<li>
<p>NetBSD</p>
</li>
<li>
<p>NonStop</p>
</li>
<li>
<p>OpenBSD</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>The platforms which must be supported by the tool should be aligned with our
[platform support policy](platform-support.txt).</p>
</div>
</div>
<div class="sect2">
<h3 id="_auto_detection_of_supported_features">Auto-detection of supported features</h3>
<div class="paragraph">
<p>The build system must support auto-detection of features which are or aren&#8217;t
available on the current platform. Platform maintainers should not be required
to manually configure the complete build.</p>
</div>
<div class="paragraph">
<p>Auto-detection of the following items is considered to be important:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Check for the existence of headers.</p>
</li>
<li>
<p>Check for the existence of libraries.</p>
</li>
<li>
<p>Check for the existence of exectuables.</p>
</li>
<li>
<p>Check for the runtime behavior of specific functions.</p>
</li>
<li>
<p>Check for specific link order requirements when multiple libraries are
involved.</p>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_ease_of_use">Ease of use</h3>
<div class="paragraph">
<p>The build system should be both easy to use and easy to extend. While this is
naturally a subjective metric it is likely not controversial to say that some
build systems are considerably harder to use than others.</p>
</div>
</div>
<div class="sect2">
<h3 id="_ide_support">IDE support</h3>
<div class="paragraph">
<p>The build system should integrate with well-known IDEs. Well-known IDEs include:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Microsoft Visual Studio</p>
</li>
<li>
<p>Visual Studio Code</p>
</li>
<li>
<p>Xcode</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>There are four levels of support:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Native integration into the IDE.</p>
</li>
<li>
<p>Integration into the IDE via a plugin.</p>
</li>
<li>
<p>Integration into the IDE via generating a project description with the build
system.</p>
</li>
<li>
<p>No integration.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Native integration is preferable, but integration via either a plugin or by
generating a project description via the build system are considered feasible
alternatives.</p>
</div>
<div class="paragraph">
<p>Another important distinction is the level of integration. There are two
features that one generally wants to have:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Integration of build targets.</p>
</li>
<li>
<p>Automatic setup of features like code completion with detected build
dependencies.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>The first bullet point is the bare minimum, but is not sufficient to be
considered proper integration.</p>
</div>
</div>
<div class="sect2">
<h3 id="_out_of_tree_builds">Out-of-tree builds</h3>
<div class="paragraph">
<p>The build system should support out-of-tree builds. Out-of-tree builds allow a
developer to configure multiple different build directories with different
configuration, e.g. one "debug" build and one "release" build.</p>
</div>
</div>
<div class="sect2">
<h3 id="_cross_platform_builds">Cross-platform builds</h3>
<div class="paragraph">
<p>The build system should support cross-platform builds, e.g. building for arm on
an x86-64 host.</p>
</div>
</div>
<div class="sect2">
<h3 id="_language_support">Language support</h3>
<div class="paragraph">
<p>The following languages and toolchains are of relevance and should be supported
by the build system:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>C: the primary compiled language used by Git, must be supported. Relevant
toolchains are GCC, Clang and MSVC.</p>
</li>
<li>
<p>Rust: candidate as a second compiled lanugage, should be supported. Relevant
toolchains is the LLVM-based rustc.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Built-in support for the respective languages is preferred over support that
needs to be wired up manually to avoid unnecessary complexity. Native support
includes the following features:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Compiling objects.</p>
</li>
<li>
<p>Dependency tracking.</p>
</li>
<li>
<p>Detection of available features.</p>
</li>
<li>
<p>Discovery of relevant toolchains.</p>
</li>
<li>
<p>Linking libraries and executables.</p>
</li>
<li>
<p>Templating placeholders in scripts.</p>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_test_integration">Test integration</h3>
<div class="paragraph">
<p>It should be possible to integrate tests into the build system such that it is
possible to build and test Git within the build system. Features which are nice
to have:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Track build-time dependencies for respective tests. Unit tests have
different requirements than integration tests.</p>
</li>
<li>
<p>Allow filtering of which tests to run.</p>
</li>
<li>
<p>Allow running tests such that utilities like <code>test_pause</code> or <code>debug</code> work.</p>
</li>
</ul>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_comparison">Comparison</h2>
<div class="sectionbody">
<div class="paragraph">
<p>The following list of build systems are considered:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>GNU Make</p>
</li>
<li>
<p>autoconf</p>
</li>
<li>
<p>CMake</p>
</li>
<li>
<p>Meson</p>
</li>
</ul>
</div>
<div class="sect2">
<h3 id="_gnu_make">GNU Make</h3>
<div class="ulist">
<ul>
<li>
<p>Platform support: ubitquitous on all platforms, but not well-integrated into Windows.</p>
</li>
<li>
<p>Auto-detection: no built-in support for auto-detection of features.</p>
</li>
<li>
<p>Ease of use: easy to use, but discovering available options is hard. Makefile
rules can quickly get out of hand once reaching a certain scope.</p>
</li>
<li>
<p>IDE support: execution of Makefile targets is supported by many IDEs</p>
</li>
<li>
<p>Out-of-tree builds: supported in theory, not wired up in practice.</p>
</li>
<li>
<p>Cross-platform builds: supported in theory, not wired up in practice.</p>
</li>
<li>
<p>Language support:</p>
</li>
<li>
<p>C: Limited built-in support, many parts need to be wired up manually.</p>
</li>
<li>
<p>Rust: No built-in support, needs to be wired up manually.</p>
</li>
<li>
<p>Test integration: partially supported, many parts need to be wired up
manually.</p>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_autoconf">autoconf</h3>
<div class="ulist">
<ul>
<li>
<p>Platform support: ubiquitous on all platforms, but not well-integrated into Windows.</p>
</li>
<li>
<p>Auto-detection: supported.</p>
</li>
<li>
<p>Ease of use: easy to use, discovering available options is comparatively
easy. The autoconf syntax is prohibitively hard to extend though due to its
complex set of interacting files and the hard-to-understand M4 language.</p>
</li>
<li>
<p>IDE support: no integration into IDEs at generation time. The generated
Makefiles have the same level of support as GNU Make.</p>
</li>
<li>
<p>Out-of-tree builds: supported in theory, not wired up in practice.</p>
</li>
<li>
<p>Cross-platform builds: supported.</p>
</li>
<li>
<p>Language support:</p>
</li>
<li>
<p>C: Limited built-in support, many parts need to be wired up manually.</p>
</li>
<li>
<p>Rust: No built-in support, needs to be wired up manually.</p>
</li>
<li>
<p>Test integration: partially supported, many parts need to be wired up
manually.</p>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_cmake">CMake</h3>
<div class="ulist">
<ul>
<li>
<p>Platform support: not as extensive as GNU Make or autoconf, but all major
platforms are supported.</p>
</li>
<li>
<p>AIX</p>
</li>
<li>
<p>Cygwin</p>
</li>
<li>
<p>FreeBSD</p>
</li>
<li>
<p>Linux</p>
</li>
<li>
<p>OpenBSD</p>
</li>
<li>
<p>Solaris</p>
</li>
<li>
<p>Windows</p>
</li>
<li>
<p>macOS</p>
</li>
<li>
<p>Ease of use: easy to use, discovering available options is not always
trivial. The scripting language used by CMake is somewhat cumbersome to use,
but extending CMake build instructions is doable.</p>
</li>
<li>
<p>IDE support: natively integrated into Microsoft Visual Studio. Can generate
project descriptions for Xcode. An extension is available for Visual Studio
Code. Many other IDEs have plugins for CMake.</p>
</li>
<li>
<p>Out-of-tree builds: supported.</p>
</li>
<li>
<p>Cross-platform builds: supported.</p>
</li>
<li>
<p>Language support:</p>
</li>
<li>
<p>C: Supported for GCC, Clang, MSVC and other toolchains.</p>
</li>
<li>
<p>Rust: No built-in support, needs to be wired up manually.</p>
</li>
<li>
<p>Test integration: supported, even though test dependencies are a bit
cumbersome to use via "test fixtures". Interactive test runs are not
supported.</p>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_meson">Meson</h3>
<div class="ulist">
<ul>
<li>
<p>Platform: not as extensive as GNU Make or autoconf, but all major platforms
and some smaller ones are supported.</p>
</li>
<li>
<p>AIX</p>
</li>
<li>
<p>Cygwin</p>
</li>
<li>
<p>DragonflyBSD</p>
</li>
<li>
<p>FreeBSD</p>
</li>
<li>
<p>Haiku</p>
</li>
<li>
<p>Linux</p>
</li>
<li>
<p>NetBSD</p>
</li>
<li>
<p>OpenBSD</p>
</li>
<li>
<p>Solaris</p>
</li>
<li>
<p>Windows</p>
</li>
<li>
<p>macOS</p>
</li>
<li>
<p>Ease of use: easy to use, discovering available options is easy. The
scripting language is straight-forward to use.</p>
</li>
<li>
<p>IDE support: Supports generating build instructions for Xcode and Microsoft
Visual Studio, a plugin exists for Visual Studio Code.</p>
</li>
<li>
<p>Out-of-tree builds: supported.</p>
</li>
<li>
<p>Cross-platform builds: supported.</p>
</li>
<li>
<p>Language support:</p>
</li>
<li>
<p>C: Supported for GCC, Clang, MSVC and other toolchains.</p>
</li>
<li>
<p>Rust: Supported for rustc.</p>
</li>
<li>
<p>Test integration: supported. Interactive tests are supported starting with
Meson 1.5.0 via the <code>--interactive</code> flag.</p>
</li>
</ul>
</div>
</div>
</div>
</div>
</div>
<div id="footer">
<div id="footer-text">
Last updated 2025-03-17 07:09:38 UTC
</div>
</div>
</body>
</html>

View File

@@ -0,0 +1,572 @@
Bundle URIs
===========
Git bundles are files that store a pack-file along with some extra metadata,
including a set of refs and a (possibly empty) set of necessary commits. See
linkgit:git-bundle[1] and linkgit:gitformat-bundle[5] for more information.
Bundle URIs are locations where Git can download one or more bundles in
order to bootstrap the object database in advance of fetching the remaining
objects from a remote.
One goal is to speed up clones and fetches for users with poor network
connectivity to the origin server. Another benefit is to allow heavy users,
such as CI build farms, to use local resources for the majority of Git data
and thereby reducing the load on the origin server.
To enable the bundle URI feature, users can specify a bundle URI using
command-line options or the origin server can advertise one or more URIs
via a protocol v2 capability.
Design Goals
------------
The bundle URI standard aims to be flexible enough to satisfy multiple
workloads. The bundle provider and the Git client have several choices in
how they create and consume bundle URIs.
* Bundles can have whatever name the server desires. This name could refer
to immutable data by using a hash of the bundle contents. However, this
means that a new URI will be needed after every update of the content.
This might be acceptable if the server is advertising the URI (and the
server is aware of new bundles being generated) but would not be
ergonomic for users using the command line option.
* The bundles could be organized specifically for bootstrapping full
clones, but could also be organized with the intention of bootstrapping
incremental fetches. The bundle provider must decide on one of several
organization schemes to minimize client downloads during incremental
fetches, but the Git client can also choose whether to use bundles for
either of these operations.
* The bundle provider can choose to support full clones, partial clones,
or both. The client can detect which bundles are appropriate for the
repository's partial clone filter, if any.
* The bundle provider can use a single bundle (for clones only), or a
list of bundles. When using a list of bundles, the provider can specify
whether or not the client needs _all_ of the bundle URIs for a full
clone, or if _any_ one of the bundle URIs is sufficient. This allows the
bundle provider to use different URIs for different geographies.
* The bundle provider can organize the bundles using heuristics, such as
creation tokens, to help the client prevent downloading bundles it does
not need. When the bundle provider does not provide these heuristics,
the client can use optimizations to minimize how much of the data is
downloaded.
* The bundle provider does not need to be associated with the Git server.
The client can choose to use the bundle provider without it being
advertised by the Git server.
* The client can choose to discover bundle providers that are advertised
by the Git server. This could happen during `git clone`, during
`git fetch`, both, or neither. The user can choose which combination
works best for them.
* The client can choose to configure a bundle provider manually at any
time. The client can also choose to specify a bundle provider manually
as a command-line option to `git clone`.
Each repository is different and every Git server has different needs.
Hopefully the bundle URI feature is flexible enough to satisfy all needs.
If not, then the feature can be extended through its versioning mechanism.
Server requirements
-------------------
To provide a server-side implementation of bundle servers, no other parts
of the Git protocol are required. This allows server maintainers to use
static content solutions such as CDNs in order to serve the bundle files.
At the current scope of the bundle URI feature, all URIs are expected to
be HTTP(S) URLs where content is downloaded to a local file using a `GET`
request to that URL. The server could include authentication requirements
to those requests with the aim of triggering the configured credential
helper for secure access. (Future extensions could use "file://" URIs or
SSH URIs.)
Assuming a `200 OK` response from the server, the content at the URL is
inspected. First, Git attempts to parse the file as a bundle file of
version 2 or higher. If the file is not a bundle, then the file is parsed
as a plain-text file using Git's config parser. The key-value pairs in
that config file are expected to describe a list of bundle URIs. If
neither of these parse attempts succeed, then Git will report an error to
the user that the bundle URI provided erroneous data.
Any other data provided by the server is considered erroneous.
Bundle Lists
------------
The Git server can advertise bundle URIs using a set of `key=value` pairs.
A bundle URI can also serve a plain-text file in the Git config format
containing these same `key=value` pairs. In both cases, we consider this
to be a _bundle list_. The pairs specify information about the bundles
that the client can use to make decisions for which bundles to download
and which to ignore.
A few keys focus on properties of the list itself.
bundle.version::
(Required) This value provides a version number for the bundle
list. If a future Git change enables a feature that needs the Git
client to react to a new key in the bundle list file, then this version
will increment. The only current version number is 1, and if any other
value is specified then Git will fail to use this file.
bundle.mode::
(Required) This value has one of two values: `all` and `any`. When `all`
is specified, then the client should expect to need all of the listed
bundle URIs that match their repository's requirements. When `any` is
specified, then the client should expect that any one of the bundle URIs
that match their repository's requirements will suffice. Typically, the
`any` option is used to list a number of different bundle servers
located in different geographies.
bundle.heuristic::
If this string-valued key exists, then the bundle list is designed to
work well with incremental `git fetch` commands. The heuristic signals
that there are additional keys available for each bundle that help
determine which subset of bundles the client should download. The only
heuristic currently planned is `creationToken`.
The remaining keys include an `<id>` segment which is a server-designated
name for each available bundle. The `<id>` must contain only alphanumeric
and `-` characters.
bundle.<id>.uri::
(Required) This string value is the URI for downloading bundle `<id>`.
If the URI begins with a protocol (`http://` or `https://`) then the URI
is absolute. Otherwise, the URI is interpreted as relative to the URI
used for the bundle list. If the URI begins with `/`, then that relative
path is relative to the domain name used for the bundle list. (This use
of relative paths is intended to make it easier to distribute a set of
bundles across a large number of servers or CDNs with different domain
names.)
bundle.<id>.filter::
This string value represents an object filter that should also appear in
the header of this bundle. The server uses this value to differentiate
different kinds of bundles from which the client can choose those that
match their object filters.
bundle.<id>.creationToken::
This value is a nonnegative 64-bit integer used for sorting the bundles
list. This is used to download a subset of bundles during a fetch when
`bundle.heuristic=creationToken`.
bundle.<id>.location::
This string value advertises a real-world location from where the bundle
URI is served. This can be used to present the user with an option for
which bundle URI to use or simply as an informative indicator of which
bundle URI was selected by Git. This is only valuable when
`bundle.mode` is `any`.
Here is an example bundle list using the Git config format:
[bundle]
version = 1
mode = all
heuristic = creationToken
[bundle "2022-02-09-1644442601-daily"]
uri = https://bundles.example.com/git/git/2022-02-09-1644442601-daily.bundle
creationToken = 1644442601
[bundle "2022-02-02-1643842562"]
uri = https://bundles.example.com/git/git/2022-02-02-1643842562.bundle
creationToken = 1643842562
[bundle "2022-02-09-1644442631-daily-blobless"]
uri = 2022-02-09-1644442631-daily-blobless.bundle
creationToken = 1644442631
filter = blob:none
[bundle "2022-02-02-1643842568-blobless"]
uri = /git/git/2022-02-02-1643842568-blobless.bundle
creationToken = 1643842568
filter = blob:none
This example uses `bundle.mode=all` as well as the
`bundle.<id>.creationToken` heuristic. It also uses the `bundle.<id>.filter`
options to present two parallel sets of bundles: one for full clones and
another for blobless partial clones.
Suppose that this bundle list was found at the URI
`https://bundles.example.com/git/git/` and so the two blobless bundles have
the following fully-expanded URIs:
* `https://bundles.example.com/git/git/2022-02-09-1644442631-daily-blobless.bundle`
* `https://bundles.example.com/git/git/2022-02-02-1643842568-blobless.bundle`
Advertising Bundle URIs
-----------------------
If a user knows a bundle URI for the repository they are cloning, then
they can specify that URI manually through a command-line option. However,
a Git host may want to advertise bundle URIs during the clone operation,
helping users unaware of the feature.
The only thing required for this feature is that the server can advertise
one or more bundle URIs. This advertisement takes the form of a new
protocol v2 capability specifically for discovering bundle URIs.
The client could choose an arbitrary bundle URI as an option _or_ select
the URI with best performance by some exploratory checks. It is up to the
bundle provider to decide if having multiple URIs is preferable to a
single URI that is geodistributed through server-side infrastructure.
Cloning with Bundle URIs
------------------------
The primary need for bundle URIs is to speed up clones. The Git client
will interact with bundle URIs according to the following flow:
1. The user specifies a bundle URI with the `--bundle-uri` command-line
option _or_ the client discovers a bundle list advertised by the
Git server.
2. If the downloaded data from a bundle URI is a bundle, then the client
inspects the bundle headers to check that the prerequisite commit OIDs
are present in the client repository. If some are missing, then the
client delays unbundling until other bundles have been unbundled,
making those OIDs present. When all required OIDs are present, the
client unbundles that data using a refspec. The default refspec is
`+refs/heads/*:refs/bundles/*`, but this can be configured. These refs
are stored so that later `git fetch` negotiations can communicate each
bundled ref as a `have`, reducing the size of the fetch over the Git
protocol. To allow pruning refs from this ref namespace, Git may
introduce a numbered namespace (such as `refs/bundles/<i>/*`) such that
stale bundle refs can be deleted.
3. If the file is instead a bundle list, then the client inspects the
`bundle.mode` to see if the list is of the `all` or `any` form.
a. If `bundle.mode=all`, then the client considers all bundle
URIs. The list is reduced based on the `bundle.<id>.filter` options
matching the client repository's partial clone filter. Then, all
bundle URIs are requested. If the `bundle.<id>.creationToken`
heuristic is provided, then the bundles are downloaded in decreasing
order by the creation token, stopping when a bundle has all required
OIDs. The bundles can then be unbundled in increasing creation token
order. The client stores the latest creation token as a heuristic
for avoiding future downloads if the bundle list does not advertise
bundles with larger creation tokens.
b. If `bundle.mode=any`, then the client can choose any one of the
bundle URIs to inspect. The client can use a variety of ways to
choose among these URIs. The client can also fallback to another URI
if the initial choice fails to return a result.
Note that during a clone we expect that all bundles will be required, and
heuristics such as `bundle.<uri>.creationToken` can be used to download
bundles in chronological order or in parallel.
If a given bundle URI is a bundle list with a `bundle.heuristic`
value, then the client can choose to store that URI as its chosen bundle
URI. The client can then navigate directly to that URI during later `git
fetch` calls.
When downloading bundle URIs, the client can choose to inspect the initial
content before committing to downloading the entire content. This may
provide enough information to determine if the URI is a bundle list or
a bundle. In the case of a bundle, the client may inspect the bundle
header to determine that all advertised tips are already in the client
repository and cancel the remaining download.
Fetching with Bundle URIs
-------------------------
When the client fetches new data, it can decide to fetch from bundle
servers before fetching from the origin remote. This could be done via a
command-line option, but it is more likely useful to use a config value
such as the one specified during the clone.
The fetch operation follows the same procedure to download bundles from a
bundle list (although we do _not_ want to use parallel downloads here). We
expect that the process will end when all prerequisite commit OIDs in a
thin bundle are already in the object database.
When using the `creationToken` heuristic, the client can avoid downloading
any bundles if their creation tokens are not larger than the stored
creation token. After fetching new bundles, Git updates this local
creation token.
If the bundle provider does not provide a heuristic, then the client
should attempt to inspect the bundle headers before downloading the full
bundle data in case the bundle tips already exist in the client
repository.
Error Conditions
----------------
If the Git client discovers something unexpected while downloading
information according to a bundle URI or the bundle list found at that
location, then Git can ignore that data and continue as if it was not
given a bundle URI. The remote Git server is the ultimate source of truth,
not the bundle URI.
Here are a few example error conditions:
* The client fails to connect with a server at the given URI or a connection
is lost without any chance to recover.
* The client receives a 400-level response (such as `404 Not Found` or
`401 Not Authorized`). The client should use the credential helper to
find and provide a credential for the URI, but match the semantics of
Git's other HTTP protocols in terms of handling specific 400-level
errors.
* The server reports any other failure response.
* The client receives data that is not parsable as a bundle or bundle list.
* A bundle includes a filter that does not match expectations.
* The client cannot unbundle the bundles because the prerequisite commit OIDs
are not in the object database and there are no more bundles to download.
There are also situations that could be seen as wasteful, but are not
error conditions:
* The downloaded bundles contain more information than is requested by
the clone or fetch request. A primary example is if the user requests
a clone with `--single-branch` but downloads bundles that store every
reachable commit from all `refs/heads/*` references. This might be
initially wasteful, but perhaps these objects will become reachable by
a later ref update that the client cares about.
* A bundle download during a `git fetch` contains objects already in the
object database. This is probably unavoidable if we are using bundles
for fetches, since the client will almost always be slightly ahead of
the bundle servers after performing its "catch-up" fetch to the remote
server. This extra work is most wasteful when the client is fetching
much more frequently than the server is computing bundles, such as if
the client is using hourly prefetches with background maintenance, but
the server is computing bundles weekly. For this reason, the client
should not use bundle URIs for fetch unless the server has explicitly
recommended it through a `bundle.heuristic` value.
Example Bundle Provider organization
------------------------------------
The bundle URI feature is intentionally designed to be flexible to
different ways a bundle provider wants to organize the object data.
However, it can be helpful to have a complete organization model described
here so providers can start from that base.
This example organization is a simplified model of what is used by the
GVFS Cache Servers (see section near the end of this document) which have
been beneficial in speeding up clones and fetches for very large
repositories, although using extra software outside of Git.
The bundle provider deploys servers across multiple geographies. Each
server manages its own bundle set. The server can track a number of Git
repositories, but provides a bundle list for each based on a pattern. For
example, when mirroring a repository at `https://<domain>/<org>/<repo>`
the bundle server could have its bundle list available at
`https://<server-url>/<domain>/<org>/<repo>`. The origin Git server can
list all of these servers under the "any" mode:
[bundle]
version = 1
mode = any
[bundle "eastus"]
uri = https://eastus.example.com/<domain>/<org>/<repo>
[bundle "europe"]
uri = https://europe.example.com/<domain>/<org>/<repo>
[bundle "apac"]
uri = https://apac.example.com/<domain>/<org>/<repo>
This "list of lists" is static and only changes if a bundle server is
added or removed.
Each bundle server manages its own set of bundles. The initial bundle list
contains only a single bundle, containing all of the objects received from
cloning the repository from the origin server. The list uses the
`creationToken` heuristic and a `creationToken` is made for the bundle
based on the server's timestamp.
The bundle server runs regularly-scheduled updates for the bundle list,
such as once a day. During this task, the server fetches the latest
contents from the origin server and generates a bundle containing the
objects reachable from the latest origin refs, but not contained in a
previously-computed bundle. This bundle is added to the list, with care
that the `creationToken` is strictly greater than the previous maximum
`creationToken`.
When the bundle list grows too large, say more than 30 bundles, then the
oldest "_N_ minus 30" bundles are combined into a single bundle. This
bundle's `creationToken` is equal to the maximum `creationToken` among the
merged bundles.
An example bundle list is provided here, although it only has two daily
bundles and not a full list of 30:
[bundle]
version = 1
mode = all
heuristic = creationToken
[bundle "2022-02-13-1644770820-daily"]
uri = https://eastus.example.com/<domain>/<org>/<repo>/2022-02-09-1644770820-daily.bundle
creationToken = 1644770820
[bundle "2022-02-09-1644442601-daily"]
uri = https://eastus.example.com/<domain>/<org>/<repo>/2022-02-09-1644442601-daily.bundle
creationToken = 1644442601
[bundle "2022-02-02-1643842562"]
uri = https://eastus.example.com/<domain>/<org>/<repo>/2022-02-02-1643842562.bundle
creationToken = 1643842562
To avoid storing and serving object data in perpetuity despite becoming
unreachable in the origin server, this bundle merge can be more careful.
Instead of taking an absolute union of the old bundles, instead the bundle
can be created by looking at the newer bundles and ensuring that their
necessary commits are all available in this merged bundle (or in another
one of the newer bundles). This allows "expiring" object data that is not
being used by new commits in this window of time. That data could be
reintroduced by a later push.
The intention of this data organization has two main goals. First, initial
clones of the repository become faster by downloading precomputed object
data from a closer source. Second, `git fetch` commands can be faster,
especially if the client has not fetched for a few days. However, if a
client does not fetch for 30 days, then the bundle list organization would
cause redownloading a large amount of object data.
One way to make this organization more useful to users who fetch frequently
is to have more frequent bundle creation. For example, bundles could be
created every hour, and then once a day those "hourly" bundles could be
merged into a "daily" bundle. The daily bundles are merged into the
oldest bundle after 30 days.
It is recommended that this bundle strategy is repeated with the `blob:none`
filter if clients of this repository are expecting to use blobless partial
clones. This list of blobless bundles stays in the same list as the full
bundles, but uses the `bundle.<id>.filter` key to separate the two groups.
For very large repositories, the bundle provider may want to _only_ provide
blobless bundles.
Implementation Plan
-------------------
This design document is being submitted on its own as an aspirational
document, with the goal of implementing all of the mentioned client
features over the course of several patch series. Here is a potential
outline for submitting these features:
1. Integrate bundle URIs into `git clone` with a `--bundle-uri` option.
This will include a new `git fetch --bundle-uri` mode for use as the
implementation underneath `git clone`. The initial version here will
expect a single bundle at the given URI.
2. Implement the ability to parse a bundle list from a bundle URI and
update the `git fetch --bundle-uri` logic to properly distinguish
between `bundle.mode` options. Specifically design the feature so
that the config format parsing feeds a list of key-value pairs into the
bundle list logic.
3. Create the `bundle-uri` protocol v2 command so Git servers can advertise
bundle URIs using the key-value pairs. Plug into the existing key-value
input to the bundle list logic. Allow `git clone` to discover these
bundle URIs and bootstrap the client repository from the bundle data.
(This choice is an opt-in via a config option and a command-line
option.)
4. Allow the client to understand the `bundle.heuristic` configuration key
and the `bundle.<id>.creationToken` heuristic. When `git clone`
discovers a bundle URI with `bundle.heuristic`, it configures the client
repository to check that bundle URI during later `git fetch <remote>`
commands.
5. Allow clients to discover bundle URIs during `git fetch` and configure
a bundle URI for later fetches if `bundle.heuristic` is set.
6. Implement the "inspect headers" heuristic to reduce data downloads when
the `bundle.<id>.creationToken` heuristic is not available.
As these features are reviewed, this plan might be updated. We also expect
that new designs will be discovered and implemented as this feature
matures and becomes used in real-world scenarios.
Related Work: Packfile URIs
---------------------------
The Git protocol already has a capability where the Git server can list
a set of URLs along with the packfile response when serving a client
request. The client is then expected to download the packfiles at those
locations in order to have a complete understanding of the response.
This mechanism is used by the Gerrit server (implemented with JGit) and
has been effective at reducing CPU load and improving user performance for
clones.
A major downside to this mechanism is that the origin server needs to know
_exactly_ what is in those packfiles, and the packfiles need to be available
to the user for some time after the server has responded. This coupling
between the origin and the packfile data is difficult to manage.
Further, this implementation is extremely hard to make work with fetches.
Related Work: GVFS Cache Servers
--------------------------------
The GVFS Protocol [2] is a set of HTTP endpoints designed independently of
the Git project before Git's partial clone was created. One feature of this
protocol is the idea of a "cache server" which can be colocated with build
machines or developer offices to transfer Git data without overloading the
central server.
The endpoint that VFS for Git is famous for is the `GET /gvfs/objects/{oid}`
endpoint, which allows downloading an object on-demand. This is a critical
piece of the filesystem virtualization of that product.
However, a more subtle need is the `GET /gvfs/prefetch?lastPackTimestamp=<t>`
endpoint. Given an optional timestamp, the cache server responds with a list
of precomputed packfiles containing the commits and trees that were introduced
in those time intervals.
The cache server computes these "prefetch" packfiles using the following
strategy:
1. Every hour, an "hourly" pack is generated with a given timestamp.
2. Nightly, the previous 24 hourly packs are rolled up into a "daily" pack.
3. Nightly, all prefetch packs more than 30 days old are rolled up into
one pack.
When a user runs `gvfs clone` or `scalar clone` against a repo with cache
servers, the client requests all prefetch packfiles, which is at most
`24 + 30 + 1` packfiles downloading only commits and trees. The client
then follows with a request to the origin server for the references, and
attempts to checkout that tip reference. (There is an extra endpoint that
helps get all reachable trees from a given commit, in case that commit
was not already in a prefetch packfile.)
During a `git fetch`, a hook requests the prefetch endpoint using the
most-recent timestamp from a previously-downloaded prefetch packfile.
Only the list of packfiles with later timestamps are downloaded. Most
users fetch hourly, so they get at most one hourly prefetch pack. Users
whose machines have been off or otherwise have not fetched in over 30 days
might redownload all prefetch packfiles. This is rare.
It is important to note that the clients always contact the origin server
for the refs advertisement, so the refs are frequently "ahead" of the
prefetched pack data. The missing objects are downloaded on-demand using
the `GET gvfs/objects/{oid}` requests, when needed by a command such as
`git checkout` or `git log`. Some Git optimizations disable checks that
would cause these on-demand downloads to be too aggressive.
See Also
--------
[1] https://lore.kernel.org/git/RFC-cover-00.13-0000000000-20210805T150534Z-avarab@gmail.com/
An earlier RFC for a bundle URI feature.
[2] https://github.com/microsoft/VFSForGit/blob/master/Protocol.md
The GVFS Protocol

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,401 @@
Git Commit-Graph Design Notes
=============================
Git walks the commit graph for many reasons, including:
1. Listing and filtering commit history.
2. Computing merge bases.
These operations can become slow as the commit count grows. The merge
base calculation shows up in many user-facing commands, such as 'merge-base'
or 'status' and can take minutes to compute depending on history shape.
There are two main costs here:
1. Decompressing and parsing commits.
2. Walking the entire graph to satisfy topological order constraints.
The commit-graph file is a supplemental data structure that accelerates
commit graph walks. If a user downgrades or disables the 'core.commitGraph'
config setting, then the existing object database is sufficient. The file is stored
as "commit-graph" either in the .git/objects/info directory or in the info
directory of an alternate.
The commit-graph file stores the commit graph structure along with some
extra metadata to speed up graph walks. By listing commit OIDs in
lexicographic order, we can identify an integer position for each commit
and refer to the parents of a commit using those integer positions. We
use binary search to find initial commits and then use the integer
positions for fast lookups during the walk.
A consumer may load the following info for a commit from the graph:
1. The commit OID.
2. The list of parents, along with their integer position.
3. The commit date.
4. The root tree OID.
5. The generation number (see definition below).
Values 1-4 satisfy the requirements of parse_commit_gently().
There are two definitions of generation number:
1. Corrected committer dates (generation number v2)
2. Topological levels (generation number v1)
Define "corrected committer date" of a commit recursively as follows:
* A commit with no parents (a root commit) has corrected committer date
equal to its committer date.
* A commit with at least one parent has corrected committer date equal to
the maximum of its committer date and one more than the largest corrected
committer date among its parents.
* As a special case, a root commit with timestamp zero has corrected commit
date of 1, to be able to distinguish it from GENERATION_NUMBER_ZERO
(that is, an uncomputed corrected commit date).
Define the "topological level" of a commit recursively as follows:
* A commit with no parents (a root commit) has topological level of one.
* A commit with at least one parent has topological level one more than
the largest topological level among its parents.
Equivalently, the topological level of a commit A is one more than the
length of a longest path from A to a root commit. The recursive definition
is easier to use for computation and observing the following property:
If A and B are commits with generation numbers N and M, respectively,
and N <= M, then A cannot reach B. That is, we know without searching
that B is not an ancestor of A because it is further from a root commit
than A.
Conversely, when checking if A is an ancestor of B, then we only need
to walk commits until all commits on the walk boundary have generation
number at most N. If we walk commits using a priority queue seeded by
generation numbers, then we always expand the boundary commit with highest
generation number and can easily detect the stopping condition.
The property applies to both versions of generation number, that is both
corrected committer dates and topological levels.
This property can be used to significantly reduce the time it takes to
walk commits and determine topological relationships. Without generation
numbers, the general heuristic is the following:
If A and B are commits with commit time X and Y, respectively, and
X < Y, then A _probably_ cannot reach B.
In absence of corrected commit dates (for example, old versions of Git or
mixed generation graph chains),
this heuristic is currently used whenever the computation is allowed to
violate topological relationships due to clock skew (such as "git log"
with default order), but is not used when the topological order is
required (such as merge base calculations, "git log --graph").
In practice, we expect some commits to be created recently and not stored
in the commit-graph. We can treat these commits as having "infinite"
generation number and walk until reaching commits with known generation
number.
We use the macro GENERATION_NUMBER_INFINITY to mark commits not
in the commit-graph file. If a commit-graph file was written by a version
of Git that did not compute generation numbers, then those commits will
have generation number represented by the macro GENERATION_NUMBER_ZERO = 0.
Since the commit-graph file is closed under reachability, we can guarantee
the following weaker condition on all commits:
If A and B are commits with generation numbers N and M, respectively,
and N < M, then A cannot reach B.
Note how the strict inequality differs from the inequality when we have
fully-computed generation numbers. Using strict inequality may result in
walking a few extra commits, but the simplicity in dealing with commits
with generation number *_INFINITY or *_ZERO is valuable.
We use the macro GENERATION_NUMBER_V1_MAX = 0x3FFFFFFF for commits whose
topological levels (generation number v1) are computed to be at least
this value. We limit at this value since it is the largest value that
can be stored in the commit-graph file using the 30 bits available
to topological levels. This presents another case where a commit can
have generation number equal to that of a parent.
Design Details
--------------
- The commit-graph file is stored in a file named 'commit-graph' in the
.git/objects/info directory. This could be stored in the info directory
of an alternate.
- The core.commitGraph config setting must be on to consume graph files.
- The file format includes parameters for the object ID hash function,
so a future change of hash algorithm does not require a change in format.
- Commit grafts and replace objects can change the shape of the commit
history. The latter can also be enabled/disabled on the fly using
`--no-replace-objects`. This leads to difficulty storing both possible
interpretations of a commit id, especially when computing generation
numbers. The commit-graph will not be read or written when
replace-objects or grafts are present.
- Shallow clones create grafts of commits by dropping their parents. This
leads the commit-graph to think those commits have generation number 1.
If and when those commits are made unshallow, those generation numbers
become invalid. Since shallow clones are intended to restrict the commit
history to a very small set of commits, the commit-graph feature is less
helpful for these clones, anyway. The commit-graph will not be read or
written when shallow commits are present.
Commit-Graphs Chains
--------------------
Typically, repos grow with near-constant velocity (commits per day). Over time,
the number of commits added by a fetch operation is much smaller than the
number of commits in the full history. By creating a "chain" of commit-graphs,
we enable fast writes of new commit data without rewriting the entire commit
history -- at least, most of the time.
## File Layout
A commit-graph chain uses multiple files, and we use a fixed naming convention
to organize these files. Each commit-graph file has a name
`$OBJDIR/info/commit-graphs/graph-{hash}.graph` where `{hash}` is the hex-
valued hash stored in the footer of that file (which is a hash of the file's
contents before that hash). For a chain of commit-graph files, a plain-text
file at `$OBJDIR/info/commit-graphs/commit-graph-chain` contains the
hashes for the files in order from "lowest" to "highest".
For example, if the `commit-graph-chain` file contains the lines
```
{hash0}
{hash1}
{hash2}
```
then the commit-graph chain looks like the following diagram:
+-----------------------+
| graph-{hash2}.graph |
+-----------------------+
|
+-----------------------+
| |
| graph-{hash1}.graph |
| |
+-----------------------+
|
+-----------------------+
| |
| |
| |
| graph-{hash0}.graph |
| |
| |
| |
+-----------------------+
Let X0 be the number of commits in `graph-{hash0}.graph`, X1 be the number of
commits in `graph-{hash1}.graph`, and X2 be the number of commits in
`graph-{hash2}.graph`. If a commit appears in position i in `graph-{hash2}.graph`,
then we interpret this as being the commit in position (X0 + X1 + i), and that
will be used as its "graph position". The commits in `graph-{hash2}.graph` use these
positions to refer to their parents, which may be in `graph-{hash1}.graph` or
`graph-{hash0}.graph`. We can navigate to an arbitrary commit in position j by checking
its containment in the intervals [0, X0), [X0, X0 + X1), [X0 + X1, X0 + X1 +
X2).
Each commit-graph file (except the base, `graph-{hash0}.graph`) contains data
specifying the hashes of all files in the lower layers. In the above example,
`graph-{hash1}.graph` contains `{hash0}` while `graph-{hash2}.graph` contains
`{hash0}` and `{hash1}`.
## Merging commit-graph files
If we only added a new commit-graph file on every write, we would run into a
linear search problem through many commit-graph files. Instead, we use a merge
strategy to decide when the stack should collapse some number of levels.
The diagram below shows such a collapse. As a set of new commits are added, it
is determined by the merge strategy that the files should collapse to
`graph-{hash1}`. Thus, the new commits, the commits in `graph-{hash2}` and
the commits in `graph-{hash1}` should be combined into a new `graph-{hash3}`
file.
+---------------------+
| |
| (new commits) |
| |
+---------------------+
| |
+-----------------------+ +---------------------+
| graph-{hash2} |->| |
+-----------------------+ +---------------------+
| | |
+-----------------------+ +---------------------+
| | | |
| graph-{hash1} |->| |
| | | |
+-----------------------+ +---------------------+
| tmp_graphXXX
+-----------------------+
| |
| |
| |
| graph-{hash0} |
| |
| |
| |
+-----------------------+
During this process, the commits to write are combined, sorted and we write the
contents to a temporary file, all while holding a `commit-graph-chain.lock`
lock-file. When the file is flushed, we rename it to `graph-{hash3}`
according to the computed `{hash3}`. Finally, we write the new chain data to
`commit-graph-chain.lock`:
```
{hash3}
{hash0}
```
We then close the lock-file.
## Merge Strategy
When writing a set of commits that do not exist in the commit-graph stack of
height N, we default to creating a new file at level N + 1. We then decide to
merge with the Nth level if one of two conditions hold:
1. `--size-multiple=<X>` is specified or X = 2, and the number of commits in
level N is less than X times the number of commits in level N + 1.
2. `--max-commits=<C>` is specified with non-zero C and the number of commits
in level N + 1 is more than C commits.
This decision cascades down the levels: when we merge a level we create a new
set of commits that then compares to the next level.
The first condition bounds the number of levels to be logarithmic in the total
number of commits. The second condition bounds the total number of commits in
a `graph-{hashN}` file and not in the `commit-graph` file, preventing
significant performance issues when the stack merges and another process only
partially reads the previous stack.
The merge strategy values (2 for the size multiple, 64,000 for the maximum
number of commits) could be extracted into config settings for full
flexibility.
## Handling Mixed Generation Number Chains
With the introduction of generation number v2 and generation data chunk, the
following scenario is possible:
1. "New" Git writes a commit-graph with the corrected commit dates.
2. "Old" Git writes a split commit-graph on top without corrected commit dates.
A naive approach of using the newest available generation number from
each layer would lead to violated expectations: the lower layer would
use corrected commit dates which are much larger than the topological
levels of the higher layer. For this reason, Git inspects the topmost
layer to see if the layer is missing corrected commit dates. In such a case
Git only uses topological level for generation numbers.
When writing a new layer in split commit-graph, we write corrected commit
dates if the topmost layer has corrected commit dates written. This
guarantees that if a layer has corrected commit dates, all lower layers
must have corrected commit dates as well.
When merging layers, we do not consider whether the merged layers had corrected
commit dates. Instead, the new layer will have corrected commit dates if the
layer below the new layer has corrected commit dates.
While writing or merging layers, if the new layer is the only layer, it will
have corrected commit dates when written by compatible versions of Git. Thus,
rewriting split commit-graph as a single file (`--split=replace`) creates a
single layer with corrected commit dates.
## Deleting graph-{hash} files
After a new tip file is written, some `graph-{hash}` files may no longer
be part of a chain. It is important to remove these files from disk, eventually.
The main reason to delay removal is that another process could read the
`commit-graph-chain` file before it is rewritten, but then look for the
`graph-{hash}` files after they are deleted.
To allow holding old split commit-graphs for a while after they are unreferenced,
we update the modified times of the files when they become unreferenced. Then,
we scan the `$OBJDIR/info/commit-graphs/` directory for `graph-{hash}`
files whose modified times are older than a given expiry window. This window
defaults to zero, but can be changed using command-line arguments or a config
setting.
## Chains across multiple object directories
In a repo with alternates, we look for the `commit-graph-chain` file starting
in the local object directory and then in each alternate. The first file that
exists defines our chain. As we look for the `graph-{hash}` files for
each `{hash}` in the chain file, we follow the same pattern for the host
directories.
This allows commit-graphs to be split across multiple forks in a fork network.
The typical case is a large "base" repo with many smaller forks.
As the base repo advances, it will likely update and merge its commit-graph
chain more frequently than the forks. If a fork updates their commit-graph after
the base repo, then it should "reparent" the commit-graph chain onto the new
chain in the base repo. When reading each `graph-{hash}` file, we track
the object directory containing it. During a write of a new commit-graph file,
we check for any changes in the source object directory and read the
`commit-graph-chain` file for that source and create a new file based on those
files. During this "reparent" operation, we necessarily need to collapse all
levels in the fork, as all of the files are invalid against the new base file.
It is crucial to be careful when cleaning up "unreferenced" `graph-{hash}.graph`
files in this scenario. It falls to the user to define the proper settings for
their custom environment:
1. When merging levels in the base repo, the unreferenced files may still be
referenced by chains from fork repos.
2. The expiry time should be set to a length of time such that every fork has
time to recompute their commit-graph chain to "reparent" onto the new base
file(s).
3. If the commit-graph chain is updated in the base, the fork will not have
access to the new chain until its chain is updated to reference those files.
(This may change in the future [5].)
Related Links
-------------
[0] https://bugs.chromium.org/p/git/issues/detail?id=8
Chromium work item for: Serialized Commit Graph
[1] https://lore.kernel.org/git/20110713070517.GC18566@sigill.intra.peff.net/
An abandoned patch that introduced generation numbers.
[2] https://lore.kernel.org/git/20170908033403.q7e6dj7benasrjes@sigill.intra.peff.net/
Discussion about generation numbers on commits and how they interact
with fsck.
[3] https://lore.kernel.org/git/20170908034739.4op3w4f2ma5s65ku@sigill.intra.peff.net/
More discussion about generation numbers and not storing them inside
commit objects. A valuable quote:
"I think we should be moving more in the direction of keeping
repo-local caches for optimizations. Reachability bitmaps have been
a big performance win. I think we should be doing the same with our
properties of commits. Not just generation numbers, but making it
cheap to access the graph structure without zlib-inflating whole
commit objects (i.e., packv4 or something like the "metapacks" I
proposed a few years ago)."
[4] https://lore.kernel.org/git/20180108154822.54829-1-git@jeffhostetler.com/T/#u
A patch to remove the ahead-behind calculation from 'status'.
[5] https://lore.kernel.org/git/f27db281-abad-5043-6d71-cbb083b1c877@gmail.com/
A discussion of a "two-dimensional graph position" that can allow reading
multiple commit-graph chains at the same time.

View File

@@ -0,0 +1,118 @@
Directory rename detection
==========================
Rename detection logic in diffcore-rename that checks for renames of
individual files is also aggregated there and then analyzed in either
merge-ort or merge-recursive for cases where combinations of renames
indicate that a full directory has been renamed.
Scope of abilities
------------------
It is perhaps easiest to start with an example:
* When all of x/a, x/b and x/c have moved to z/a, z/b and z/c, it is
likely that x/d added in the meantime would also want to move to z/d by
taking the hint that the entire directory 'x' moved to 'z'.
More interesting possibilities exist, though, such as:
* one side of history renames x -> z, and the other renames some file to
x/e, causing the need for the merge to do a transitive rename so that
the rename ends up at z/e.
* one side of history renames x -> z, but also renames all files within x.
For example, x/a -> z/alpha, x/b -> z/bravo, etc.
* both 'x' and 'y' being merged into a single directory 'z', with a
directory rename being detected for both x->z and y->z.
* not all files in a directory being renamed to the same location;
i.e. perhaps most the files in 'x' are now found under 'z', but a few
are found under 'w'.
* a directory being renamed, which also contained a subdirectory that was
renamed to some entirely different location. (And perhaps the inner
directory itself contained inner directories that were renamed to yet
other locations).
* combinations of the above; see t/t6423-merge-rename-directories.sh for
various interesting cases.
Limitations -- applicability of directory renames
-------------------------------------------------
In order to prevent edge and corner cases resulting in either conflicts
that cannot be represented in the index or which might be too complex for
users to try to understand and resolve, a couple basic rules limit when
directory rename detection applies:
1) If a given directory still exists on both sides of a merge, we do
not consider it to have been renamed.
2) If a subset of to-be-renamed files have a file or directory in the
way (or would be in the way of each other), "turn off" the directory
rename for those specific sub-paths and report the conflict to the
user.
3) If the other side of history did a directory rename to a path that
your side of history renamed away, then ignore that particular
rename from the other side of history for any implicit directory
renames (but warn the user).
Limitations -- detailed rules and testcases
-------------------------------------------
t/t6423-merge-rename-directories.sh contains extensive tests and commentary
which generate and explore the rules listed above. It also lists a few
additional rules:
a) If renames split a directory into two or more others, the directory
with the most renames, "wins".
b) Only apply implicit directory renames to directories if the other side
of history is the one doing the renaming.
c) Do not perform directory rename detection for directories which had no
new paths added to them.
Limitations -- support in different commands
--------------------------------------------
Directory rename detection is supported by 'merge' and 'cherry-pick'.
Other git commands which users might be surprised to see limited or no
directory rename detection support in:
* diff
Folks have requested in the past that `git diff` detect directory
renames and somehow simplify its output. It is not clear whether this
would be desirable or how the output should be simplified, so this was
simply not implemented. Also, while diffcore-rename has most of the
logic for detecting directory renames, some of the logic is still found
within merge-ort and merge-recursive. Fully supporting directory
rename detection in diffs would require copying or moving the remaining
bits of logic to the diff machinery.
* am
git-am tries to avoid a full three way merge, instead calling
git-apply. That prevents us from detecting renames at all, which may
defeat the directory rename detection. There is a fallback, though; if
the initial git-apply fails and the user has specified the -3 option,
git-am will fall back to a three way merge. However, git-am lacks the
necessary information to do a "real" three way merge. Instead, it has
to use build_fake_ancestor() to get a merge base that is missing files
whose rename may have been important to detect for directory rename
detection to function.
* rebase
Since am-based rebases work by first generating a bunch of patches
(which no longer record what the original commits were and thus don't
have the necessary info from which we can find a real merge-base), and
then calling git-am, this implies that am-based rebases will not always
successfully detect directory renames either (see the 'am' section
above). merged-based rebases (rebase -m) and cherry-pick-based rebases
(rebase -i) are not affected by this shortcoming, and fully support
directory rename detection.

View File

@@ -0,0 +1,830 @@
Git hash function transition
============================
Objective
---------
Migrate Git from SHA-1 to a stronger hash function.
Background
----------
At its core, the Git version control system is a content addressable
filesystem. It uses the SHA-1 hash function to name content. For
example, files, directories, and revisions are referred to by hash
values unlike in other traditional version control systems where files
or versions are referred to via sequential numbers. The use of a hash
function to address its content delivers a few advantages:
* Integrity checking is easy. Bit flips, for example, are easily
detected, as the hash of corrupted content does not match its name.
* Lookup of objects is fast.
Using a cryptographically secure hash function brings additional
advantages:
* Object names can be signed and third parties can trust the hash to
address the signed object and all objects it references.
* Communication using Git protocol and out of band communication
methods have a short reliable string that can be used to reliably
address stored content.
Over time some flaws in SHA-1 have been discovered by security
researchers. On 23 February 2017 the SHAttered attack
(https://shattered.io) demonstrated a practical SHA-1 hash collision.
Git v2.13.0 and later subsequently moved to a hardened SHA-1
implementation by default, which isn't vulnerable to the SHAttered
attack, but SHA-1 is still weak.
Thus it's considered prudent to move past any variant of SHA-1
to a new hash. There's no guarantee that future attacks on SHA-1 won't
be published in the future, and those attacks may not have viable
mitigations.
If SHA-1 and its variants were to be truly broken, Git's hash function
could not be considered cryptographically secure any more. This would
impact the communication of hash values because we could not trust
that a given hash value represented the known good version of content
that the speaker intended.
SHA-1 still possesses the other properties such as fast object lookup
and safe error checking, but other hash functions are equally suitable
that are believed to be cryptographically secure.
Choice of Hash
--------------
The hash to replace the hardened SHA-1 should be stronger than SHA-1
was: we would like it to be trustworthy and useful in practice for at
least 10 years.
Some other relevant properties:
1. A 256-bit hash (long enough to match common security practice; not
excessively long to hurt performance and disk usage).
2. High quality implementations should be widely available (e.g., in
OpenSSL and Apple CommonCrypto).
3. The hash function's properties should match Git's needs (e.g. Git
requires collision and 2nd preimage resistance and does not require
length extension resistance).
4. As a tiebreaker, the hash should be fast to compute (fortunately
many contenders are faster than SHA-1).
There were several contenders for a successor hash to SHA-1, including
SHA-256, SHA-512/256, SHA-256x16, K12, and BLAKE2bp-256.
In late 2018 the project picked SHA-256 as its successor hash.
See 0ed8d8da374 (doc hash-function-transition: pick SHA-256 as
NewHash, 2018-08-04) and numerous mailing list threads at the time,
particularly the one starting at
https://lore.kernel.org/git/20180609224913.GC38834@genre.crustytoothpaste.net/
for more information.
Goals
-----
1. The transition to SHA-256 can be done one local repository at a time.
a. Requiring no action by any other party.
b. A SHA-256 repository can communicate with SHA-1 Git servers
(push/fetch).
c. Users can use SHA-1 and SHA-256 identifiers for objects
interchangeably (see "Object names on the command line", below).
d. New signed objects make use of a stronger hash function than
SHA-1 for their security guarantees.
2. Allow a complete transition away from SHA-1.
a. Local metadata for SHA-1 compatibility can be removed from a
repository if compatibility with SHA-1 is no longer needed.
3. Maintainability throughout the process.
a. The object format is kept simple and consistent.
b. Creation of a generalized repository conversion tool.
Non-Goals
---------
1. Add SHA-256 support to Git protocol. This is valuable and the
logical next step but it is out of scope for this initial design.
2. Transparently improving the security of existing SHA-1 signed
objects.
3. Intermixing objects using multiple hash functions in a single
repository.
4. Taking the opportunity to fix other bugs in Git's formats and
protocols.
5. Shallow clones and fetches into a SHA-256 repository. (This will
change when we add SHA-256 support to Git protocol.)
6. Skip fetching some submodules of a project into a SHA-256
repository. (This also depends on SHA-256 support in Git
protocol.)
Overview
--------
We introduce a new repository format extension. Repositories with this
extension enabled use SHA-256 instead of SHA-1 to name their objects.
This affects both object names and object content -- both the names
of objects and all references to other objects within an object are
switched to the new hash function.
SHA-256 repositories cannot be read by older versions of Git.
Alongside the packfile, a SHA-256 repository stores a bidirectional
mapping between SHA-256 and SHA-1 object names. The mapping is generated
locally and can be verified using "git fsck". Object lookups use this
mapping to allow naming objects using either their SHA-1 and SHA-256 names
interchangeably.
"git cat-file" and "git hash-object" gain options to display an object
in its SHA-1 form and write an object given its SHA-1 form. This
requires all objects referenced by that object to be present in the
object database so that they can be named using the appropriate name
(using the bidirectional hash mapping).
Fetches from a SHA-1 based server convert the fetched objects into
SHA-256 form and record the mapping in the bidirectional mapping table
(see below for details). Pushes to a SHA-1 based server convert the
objects being pushed into SHA-1 form so the server does not have to be
aware of the hash function the client is using.
Detailed Design
---------------
Repository format extension
~~~~~~~~~~~~~~~~~~~~~~~~~~~
A SHA-256 repository uses repository format version `1` (see
linkgit:gitrepository-layout[5]) with `extensions.objectFormat` and
`extensions.compatObjectFormat` (see linkgit:git-config[1]) set to:
[core]
repositoryFormatVersion = 1
[extensions]
objectFormat = sha256
compatObjectFormat = sha1
The combination of setting `core.repositoryFormatVersion=1` and
populating `extensions.*` ensures that all versions of Git later than
`v0.99.9l` will die instead of trying to operate on the SHA-256
repository, instead producing an error message.
# Between v0.99.9l and v2.7.0
$ git status
fatal: Expected git repo version <= 0, found 1
# After v2.7.0
$ git status
fatal: unknown repository extensions found:
objectformat
compatobjectformat
See the "Transition plan" section below for more details on these
repository extensions.
Object names
~~~~~~~~~~~~
Objects can be named by their 40 hexadecimal digit SHA-1 name or 64
hexadecimal digit SHA-256 name, plus names derived from those (see
gitrevisions(7)).
The SHA-1 name of an object is the SHA-1 of the concatenation of its
type, length, a nul byte, and the object's SHA-1 content. This is the
traditional <sha1> used in Git to name objects.
The SHA-256 name of an object is the SHA-256 of the concatenation of its
type, length, a nul byte, and the object's SHA-256 content.
Object format
~~~~~~~~~~~~~
The content as a byte sequence of a tag, commit, or tree object named
by SHA-1 and SHA-256 differ because an object named by SHA-256 name refers to
other objects by their SHA-256 names and an object named by SHA-1 name
refers to other objects by their SHA-1 names.
The SHA-256 content of an object is the same as its SHA-1 content, except
that objects referenced by the object are named using their SHA-256 names
instead of SHA-1 names. Because a blob object does not refer to any
other object, its SHA-1 content and SHA-256 content are the same.
The format allows round-trip conversion between SHA-256 content and
SHA-1 content.
Object storage
~~~~~~~~~~~~~~
Loose objects use zlib compression and packed objects use the packed
format described in linkgit:gitformat-pack[5], just like
today. The content that is compressed and stored uses SHA-256 content
instead of SHA-1 content.
Pack index
~~~~~~~~~~
Pack index (.idx) files use a new v3 format that supports multiple
hash functions. They have the following format (all integers are in
network byte order):
- A header appears at the beginning and consists of the following:
* The 4-byte pack index signature: '\377t0c'
* 4-byte version number: 3
* 4-byte length of the header section, including the signature and
version number
* 4-byte number of objects contained in the pack
* 4-byte number of object formats in this pack index: 2
* For each object format:
** 4-byte format identifier (e.g., 'sha1' for SHA-1)
** 4-byte length in bytes of shortened object names. This is the
shortest possible length needed to make names in the shortened
object name table unambiguous.
** 4-byte integer, recording where tables relating to this format
are stored in this index file, as an offset from the beginning.
* 4-byte offset to the trailer from the beginning of this file.
* Zero or more additional key/value pairs (4-byte key, 4-byte
value). Only one key is supported: 'PSRC'. See the "Loose objects
and unreachable objects" section for supported values and how this
is used. All other keys are reserved. Readers must ignore
unrecognized keys.
- Zero or more NUL bytes. This can optionally be used to improve the
alignment of the full object name table below.
- Tables for the first object format:
* A sorted table of shortened object names. These are prefixes of
the names of all objects in this pack file, packed together
without offset values to reduce the cache footprint of the binary
search for a specific object name.
* A table of full object names in pack order. This allows resolving
a reference to "the nth object in the pack file" (from a
reachability bitmap or from the next table of another object
format) to its object name.
* A table of 4-byte values mapping object name order to pack order.
For an object in the table of sorted shortened object names, the
value at the corresponding index in this table is the index in the
previous table for that same object.
This can be used to look up the object in reachability bitmaps or
to look up its name in another object format.
* A table of 4-byte CRC32 values of the packed object data, in the
order that the objects appear in the pack file. This is to allow
compressed data to be copied directly from pack to pack during
repacking without undetected data corruption.
* A table of 4-byte offset values. For an object in the table of
sorted shortened object names, the value at the corresponding
index in this table indicates where that object can be found in
the pack file. These are usually 31-bit pack file offsets, but
large offsets are encoded as an index into the next table with the
most significant bit set.
* A table of 8-byte offset entries (empty for pack files less than
2 GiB). Pack files are organized with heavily used objects toward
the front, so most object references should not need to refer to
this table.
- Zero or more NUL bytes.
- Tables for the second object format, with the same layout as above,
up to and not including the table of CRC32 values.
- Zero or more NUL bytes.
- The trailer consists of the following:
* A copy of the 20-byte SHA-256 checksum at the end of the
corresponding packfile.
* 20-byte SHA-256 checksum of all of the above.
Loose object index
~~~~~~~~~~~~~~~~~~
A new file $GIT_OBJECT_DIR/loose-object-idx contains information about
all loose objects. Its format is
# loose-object-idx
(sha256-name SP sha1-name LF)*
where the object names are in hexadecimal format. The file is not
sorted.
The loose object index is protected against concurrent writes by a
lock file $GIT_OBJECT_DIR/loose-object-idx.lock. To add a new loose
object:
1. Write the loose object to a temporary file, like today.
2. Open loose-object-idx.lock with O_CREAT | O_EXCL to acquire the lock.
3. Rename the loose object into place.
4. Open loose-object-idx with O_APPEND and write the new object
5. Unlink loose-object-idx.lock to release the lock.
To remove entries (e.g. in "git pack-refs" or "git-prune"):
1. Open loose-object-idx.lock with O_CREAT | O_EXCL to acquire the
lock.
2. Write the new content to loose-object-idx.lock.
3. Unlink any loose objects being removed.
4. Rename to replace loose-object-idx, releasing the lock.
Translation table
~~~~~~~~~~~~~~~~~
The index files support a bidirectional mapping between SHA-1 names
and SHA-256 names. The lookup proceeds similarly to ordinary object
lookups. For example, to convert a SHA-1 name to a SHA-256 name:
1. Look for the object in idx files. If a match is present in the
idx's sorted list of truncated SHA-1 names, then:
a. Read the corresponding entry in the SHA-1 name order to pack
name order mapping.
b. Read the corresponding entry in the full SHA-1 name table to
verify we found the right object. If it is, then
c. Read the corresponding entry in the full SHA-256 name table.
That is the object's SHA-256 name.
2. Check for a loose object. Read lines from loose-object-idx until
we find a match.
Step (1) takes the same amount of time as an ordinary object lookup:
O(number of packs * log(objects per pack)). Step (2) takes O(number of
loose objects) time. To maintain good performance it will be necessary
to keep the number of loose objects low. See the "Loose objects and
unreachable objects" section below for more details.
Since all operations that make new objects (e.g., "git commit") add
the new objects to the corresponding index, this mapping is possible
for all objects in the object store.
Reading an object's SHA-1 content
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The SHA-1 content of an object can be read by converting all SHA-256 names
of its SHA-256 content references to SHA-1 names using the translation table.
Fetch
~~~~~
Fetching from a SHA-1 based server requires translating between SHA-1
and SHA-256 based representations on the fly.
SHA-1s named in the ref advertisement that are present on the client
can be translated to SHA-256 and looked up as local objects using the
translation table.
Negotiation proceeds as today. Any "have"s generated locally are
converted to SHA-1 before being sent to the server, and SHA-1s
mentioned by the server are converted to SHA-256 when looking them up
locally.
After negotiation, the server sends a packfile containing the
requested objects. We convert the packfile to SHA-256 format using
the following steps:
1. index-pack: inflate each object in the packfile and compute its
SHA-1. Objects can contain deltas in OBJ_REF_DELTA format against
objects the client has locally. These objects can be looked up
using the translation table and their SHA-1 content read as
described above to resolve the deltas.
2. topological sort: starting at the "want"s from the negotiation
phase, walk through objects in the pack and emit a list of them,
excluding blobs, in reverse topologically sorted order, with each
object coming later in the list than all objects it references.
(This list only contains objects reachable from the "wants". If the
pack from the server contained additional extraneous objects, then
they will be discarded.)
3. convert to SHA-256: open a new SHA-256 packfile. Read the topologically
sorted list just generated. For each object, inflate its
SHA-1 content, convert to SHA-256 content, and write it to the SHA-256
pack. Record the new SHA-1<-->SHA-256 mapping entry for use in the idx.
4. sort: reorder entries in the new pack to match the order of objects
in the pack the server generated and include blobs. Write a SHA-256 idx
file
5. clean up: remove the SHA-1 based pack file, index, and
topologically sorted list obtained from the server in steps 1
and 2.
Step 3 requires every object referenced by the new object to be in the
translation table. This is why the topological sort step is necessary.
As an optimization, step 1 could write a file describing what non-blob
objects each object it has inflated from the packfile references. This
makes the topological sort in step 2 possible without inflating the
objects in the packfile for a second time. The objects need to be
inflated again in step 3, for a total of two inflations.
Step 4 is probably necessary for good read-time performance. "git
pack-objects" on the server optimizes the pack file for good data
locality (see Documentation/technical/pack-heuristics.adoc).
Details of this process are likely to change. It will take some
experimenting to get this to perform well.
Push
~~~~
Push is simpler than fetch because the objects referenced by the
pushed objects are already in the translation table. The SHA-1 content
of each object being pushed can be read as described in the "Reading
an object's SHA-1 content" section to generate the pack written by git
send-pack.
Signed Commits
~~~~~~~~~~~~~~
We add a new field "gpgsig-sha256" to the commit object format to allow
signing commits without relying on SHA-1. It is similar to the
existing "gpgsig" field. Its signed payload is the SHA-256 content of the
commit object with any "gpgsig" and "gpgsig-sha256" fields removed.
This means commits can be signed
1. using SHA-1 only, as in existing signed commit objects
2. using both SHA-1 and SHA-256, by using both gpgsig-sha256 and gpgsig
fields.
3. using only SHA-256, by only using the gpgsig-sha256 field.
Old versions of "git verify-commit" can verify the gpgsig signature in
cases (1) and (2) without modifications and view case (3) as an
ordinary unsigned commit.
Signed Tags
~~~~~~~~~~~
We add a new field "gpgsig-sha256" to the tag object format to allow
signing tags without relying on SHA-1. Its signed payload is the
SHA-256 content of the tag with its gpgsig-sha256 field and "-----BEGIN PGP
SIGNATURE-----" delimited in-body signature removed.
This means tags can be signed
1. using SHA-1 only, as in existing signed tag objects
2. using both SHA-1 and SHA-256, by using gpgsig-sha256 and an in-body
signature.
3. using only SHA-256, by only using the gpgsig-sha256 field.
Mergetag embedding
~~~~~~~~~~~~~~~~~~
The mergetag field in the SHA-1 content of a commit contains the
SHA-1 content of a tag that was merged by that commit.
The mergetag field in the SHA-256 content of the same commit contains the
SHA-256 content of the same tag.
Submodules
~~~~~~~~~~
To convert recorded submodule pointers, you need to have the converted
submodule repository in place. The translation table of the submodule
can be used to look up the new hash.
Loose objects and unreachable objects
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Fast lookups in the loose-object-idx require that the number of loose
objects not grow too high.
"git gc --auto" currently waits for there to be 6700 loose objects
present before consolidating them into a packfile. We will need to
measure to find a more appropriate threshold for it to use.
"git gc --auto" currently waits for there to be 50 packs present
before combining packfiles. Packing loose objects more aggressively
may cause the number of pack files to grow too quickly. This can be
mitigated by using a strategy similar to Martin Fick's exponential
rolling garbage collection script:
https://gerrit-review.googlesource.com/c/gerrit/+/35215
"git gc" currently expels any unreachable objects it encounters in
pack files to loose objects in an attempt to prevent a race when
pruning them (in case another process is simultaneously writing a new
object that refers to the about-to-be-deleted object). This leads to
an explosion in the number of loose objects present and disk space
usage due to the objects in delta form being replaced with independent
loose objects. Worse, the race is still present for loose objects.
Instead, "git gc" will need to move unreachable objects to a new
packfile marked as UNREACHABLE_GARBAGE (using the PSRC field; see
below). To avoid the race when writing new objects referring to an
about-to-be-deleted object, code paths that write new objects will
need to copy any objects from UNREACHABLE_GARBAGE packs that they
refer to new, non-UNREACHABLE_GARBAGE packs (or loose objects).
UNREACHABLE_GARBAGE are then safe to delete if their creation time (as
indicated by the file's mtime) is long enough ago.
To avoid a proliferation of UNREACHABLE_GARBAGE packs, they can be
combined under certain circumstances. If "gc.garbageTtl" is set to
greater than one day, then packs created within a single calendar day,
UTC, can be coalesced together. The resulting packfile would have an
mtime before midnight on that day, so this makes the effective maximum
ttl the garbageTtl + 1 day. If "gc.garbageTtl" is less than one day,
then we divide the calendar day into intervals one-third of that ttl
in duration. Packs created within the same interval can be coalesced
together. The resulting packfile would have an mtime before the end of
the interval, so this makes the effective maximum ttl equal to the
garbageTtl * 4/3.
This rule comes from Thirumala Reddy Mutchukota's JGit change
https://git.eclipse.org/r/90465.
The UNREACHABLE_GARBAGE setting goes in the PSRC field of the pack
index. More generally, that field indicates where a pack came from:
- 1 (PACK_SOURCE_RECEIVE) for a pack received over the network
- 2 (PACK_SOURCE_AUTO) for a pack created by a lightweight
"gc --auto" operation
- 3 (PACK_SOURCE_GC) for a pack created by a full gc
- 4 (PACK_SOURCE_UNREACHABLE_GARBAGE) for potential garbage
discovered by gc
- 5 (PACK_SOURCE_INSERT) for locally created objects that were
written directly to a pack file, e.g. from "git add ."
This information can be useful for debugging and for "gc --auto" to
make appropriate choices about which packs to coalesce.
Caveats
-------
Invalid objects
~~~~~~~~~~~~~~~
The conversion from SHA-1 content to SHA-256 content retains any
brokenness in the original object (e.g., tree entry modes encoded with
leading 0, tree objects whose paths are not sorted correctly, and
commit objects without an author or committer). This is a deliberate
feature of the design to allow the conversion to round-trip.
More profoundly broken objects (e.g., a commit with a truncated "tree"
header line) cannot be converted but were not usable by current Git
anyway.
Shallow clone and submodules
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Because it requires all referenced objects to be available in the
locally generated translation table, this design does not support
shallow clone or unfetched submodules. Protocol improvements might
allow lifting this restriction.
Alternates
~~~~~~~~~~
For the same reason, a SHA-256 repository cannot borrow objects from a
SHA-1 repository using objects/info/alternates or
$GIT_ALTERNATE_OBJECT_REPOSITORIES.
git notes
~~~~~~~~~
The "git notes" tool annotates objects using their SHA-1 name as key.
This design does not describe a way to migrate notes trees to use
SHA-256 names. That migration is expected to happen separately (for
example using a file at the root of the notes tree to describe which
hash it uses).
Server-side cost
~~~~~~~~~~~~~~~~
Until Git protocol gains SHA-256 support, using SHA-256 based storage
on public-facing Git servers is strongly discouraged. Once Git
protocol gains SHA-256 support, SHA-256 based servers are likely not
to support SHA-1 compatibility, to avoid what may be a very expensive
hash re-encode during clone and to encourage peers to modernize.
The design described here allows fetches by SHA-1 clients of a
personal SHA-256 repository because it's not much more difficult than
allowing pushes from that repository. This support needs to be guarded
by a configuration option -- servers like git.kernel.org that serve a
large number of clients would not be expected to bear that cost.
Meaning of signatures
~~~~~~~~~~~~~~~~~~~~~
The signed payload for signed commits and tags does not explicitly
name the hash used to identify objects. If some day Git adopts a new
hash function with the same length as the current SHA-1 (40
hexadecimal digit) or SHA-256 (64 hexadecimal digit) objects then the
intent behind the PGP signed payload in an object signature is
unclear:
object e7e07d5a4fcc2a203d9873968ad3e6bd4d7419d7
type commit
tag v2.12.0
tagger Junio C Hamano <gitster@pobox.com> 1487962205 -0800
Git 2.12
Does this mean Git v2.12.0 is the commit with SHA-1 name
e7e07d5a4fcc2a203d9873968ad3e6bd4d7419d7 or the commit with
new-40-digit-hash-name e7e07d5a4fcc2a203d9873968ad3e6bd4d7419d7?
Fortunately SHA-256 and SHA-1 have different lengths. If Git starts
using another hash with the same length to name objects, then it will
need to change the format of signed payloads using that hash to
address this issue.
Object names on the command line
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To support the transition (see Transition plan below), this design
supports four different modes of operation:
1. ("dark launch") Treat object names input by the user as SHA-1 and
convert any object names written to output to SHA-1, but store
objects using SHA-256. This allows users to test the code with no
visible behavior change except for performance. This allows
running even tests that assume the SHA-1 hash function, to
sanity-check the behavior of the new mode.
2. ("early transition") Allow both SHA-1 and SHA-256 object names in
input. Any object names written to output use SHA-1. This allows
users to continue to make use of SHA-1 to communicate with peers
(e.g. by email) that have not migrated yet and prepares for mode 3.
3. ("late transition") Allow both SHA-1 and SHA-256 object names in
input. Any object names written to output use SHA-256. In this
mode, users are using a more secure object naming method by
default. The disruption is minimal as long as most of their peers
are in mode 2 or mode 3.
4. ("post-transition") Treat object names input by the user as
SHA-256 and write output using SHA-256. This is safer than mode 3
because there is less risk that input is incorrectly interpreted
using the wrong hash function.
The mode is specified in configuration.
The user can also explicitly specify which format to use for a
particular revision specifier and for output, overriding the mode. For
example:
git --output-format=sha1 log abac87a^{sha1}..f787cac^{sha256}
Transition plan
---------------
Some initial steps can be implemented independently of one another:
- adding a hash function API (vtable)
- teaching fsck to tolerate the gpgsig-sha256 field
- excluding gpgsig-* from the fields copied by "git commit --amend"
- annotating tests that depend on SHA-1 values with a SHA1 test
prerequisite
- using "struct object_id", GIT_MAX_RAWSZ, and GIT_MAX_HEXSZ
consistently instead of "unsigned char *" and the hardcoded
constants 20 and 40.
- introducing index v3
- adding support for the PSRC field and safer object pruning
The first user-visible change is the introduction of the objectFormat
extension (without compatObjectFormat). This requires:
- teaching fsck about this mode of operation
- using the hash function API (vtable) when computing object names
- signing objects and verifying signatures
- rejecting attempts to fetch from or push to an incompatible
repository
Next comes introduction of compatObjectFormat:
- implementing the loose-object-idx
- translating object names between object formats
- translating object content between object formats
- generating and verifying signatures in the compat format
- adding appropriate index entries when adding a new object to the
object store
- --output-format option
- ^{sha1} and ^{sha256} revision notation
- configuration to specify default input and output format (see
"Object names on the command line" above)
The next step is supporting fetches and pushes to SHA-1 repositories:
- allow pushes to a repository using the compat format
- generate a topologically sorted list of the SHA-1 names of fetched
objects
- convert the fetched packfile to SHA-256 format and generate an idx
file
- re-sort to match the order of objects in the fetched packfile
The infrastructure supporting fetch also allows converting an existing
repository. In converted repositories and new clones, end users can
gain support for the new hash function without any visible change in
behavior (see "dark launch" in the "Object names on the command line"
section). In particular this allows users to verify SHA-256 signatures
on objects in the repository, and it should ensure the transition code
is stable in production in preparation for using it more widely.
Over time projects would encourage their users to adopt the "early
transition" and then "late transition" modes to take advantage of the
new, more futureproof SHA-256 object names.
When objectFormat and compatObjectFormat are both set, commands
generating signatures would generate both SHA-1 and SHA-256 signatures
by default to support both new and old users.
In projects using SHA-256 heavily, users could be encouraged to adopt
the "post-transition" mode to avoid accidentally making implicit use
of SHA-1 object names.
Once a critical mass of users have upgraded to a version of Git that
can verify SHA-256 signatures and have converted their existing
repositories to support verifying them, we can add support for a
setting to generate only SHA-256 signatures. This is expected to be at
least a year later.
That is also a good moment to advertise the ability to convert
repositories to use SHA-256 only, stripping out all SHA-1 related
metadata. This improves performance by eliminating translation
overhead and security by avoiding the possibility of accidentally
relying on the safety of SHA-1.
Updating Git's protocols to allow a server to specify which hash
functions it supports is also an important part of this transition. It
is not discussed in detail in this document but this transition plan
assumes it happens. :)
Alternatives considered
-----------------------
Upgrading everyone working on a particular project on a flag day
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Projects like the Linux kernel are large and complex enough that
flipping the switch for all projects based on the repository at once
is infeasible.
Not only would all developers and server operators supporting
developers have to switch on the same flag day, but supporting tooling
(continuous integration, code review, bug trackers, etc) would have to
be adapted as well. This also makes it difficult to get early feedback
from some project participants testing before it is time for mass
adoption.
Using hash functions in parallel
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
(e.g. https://lore.kernel.org/git/22708.8913.864049.452252@chiark.greenend.org.uk/ )
Objects newly created would be addressed by the new hash, but inside
such an object (e.g. commit) it is still possible to address objects
using the old hash function.
* You cannot trust its history (needed for bisectability) in the
future without further work
* Maintenance burden as the number of supported hash functions grows
(they will never go away, so they accumulate). In this proposal, by
comparison, converted objects lose all references to SHA-1.
Signed objects with multiple hashes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Instead of introducing the gpgsig-sha256 field in commit and tag objects
for SHA-256 content based signatures, an earlier version of this design
added "hash sha256 <SHA-256 name>" fields to strengthen the existing
SHA-1 content based signatures.
In other words, a single signature was used to attest to the object
content using both hash functions. This had some advantages:
* Using one signature instead of two speeds up the signing process.
* Having one signed payload with both hashes allows the signer to
attest to the SHA-1 name and SHA-256 name referring to the same object.
* All users consume the same signature. Broken signatures are likely
to be detected quickly using current versions of git.
However, it also came with disadvantages:
* Verifying a signed object requires access to the SHA-1 names of all
objects it references, even after the transition is complete and
translation table is no longer needed for anything else. To support
this, the design added fields such as "hash sha1 tree <SHA-1 name>"
and "hash sha1 parent <SHA-1 name>" to the SHA-256 content of a signed
commit, complicating the conversion process.
* Allowing signed objects without a SHA-1 (for after the transition is
complete) complicated the design further, requiring a "nohash sha1"
field to suppress including "hash sha1" fields in the SHA-256 content
and signed payload.
Lazily populated translation table
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Some of the work of building the translation table could be deferred to
push time, but that would significantly complicate and slow down pushes.
Calculating the SHA-1 name at object creation time at the same time it is
being streamed to disk and having its SHA-256 name calculated should be
an acceptable cost.
Document History
----------------
2017-03-03
bmwill@google.com, jonathantanmy@google.com, jrnieder@gmail.com,
sbeller@google.com
* Initial version sent to https://lore.kernel.org/git/20170304011251.GA26789@aiede.mtv.corp.google.com
2017-03-03 jrnieder@gmail.com
Incorporated suggestions from jonathantanmy and sbeller:
* Describe purpose of signed objects with each hash type
* Redefine signed object verification using object content under the
first hash function
2017-03-06 jrnieder@gmail.com
* Use SHA3-256 instead of SHA2 (thanks, Linus and brian m. carlson).[1][2]
* Make SHA3-based signatures a separate field, avoiding the need for
"hash" and "nohash" fields (thanks to peff[3]).
* Add a sorting phase to fetch (thanks to Junio for noticing the need
for this).
* Omit blobs from the topological sort during fetch (thanks to peff).
* Discuss alternates, git notes, and git servers in the caveats
section (thanks to Junio Hamano, brian m. carlson[4], and Shawn
Pearce).
* Clarify language throughout (thanks to various commenters,
especially Junio).
2017-09-27 jrnieder@gmail.com, sbeller@google.com
* Use placeholder NewHash instead of SHA3-256
* Describe criteria for picking a hash function.
* Include a transition plan (thanks especially to Brandon Williams
for fleshing these ideas out)
* Define the translation table (thanks, Shawn Pearce[5], Jonathan
Tan, and Masaya Suzuki)
* Avoid loose object overhead by packing more aggressively in
"git gc --auto"
Later history:
* See the history of this file in git.git for the history of subsequent
edits. This document history is no longer being maintained as it
would now be superfluous to the commit log
References:
[1] https://lore.kernel.org/git/CA+55aFzJtejiCjV0e43+9oR3QuJK2PiFiLQemytoLpyJWe6P9w@mail.gmail.com/
[2] https://lore.kernel.org/git/CA+55aFz+gkAsDZ24zmePQuEs1XPS9BP_s8O7Q4wQ7LV7X5-oDA@mail.gmail.com/
[3] https://lore.kernel.org/git/20170306084353.nrns455dvkdsfgo5@sigill.intra.peff.net/
[4] https://lore.kernel.org/git/20170304224936.rqqtkdvfjgyezsht@genre.crustytoothpaste.net
[5] https://lore.kernel.org/git/CAJo=hJtoX9=AyLHHpUJS7fueV9ciZ_MNpnEPHUz8Whui6g9F0A@mail.gmail.com/

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,656 @@
Large Object Promisors
======================
Since Git has been created, users have been complaining about issues
with storing large files in Git. Some solutions have been created to
help, but they haven't helped much with some issues.
Git currently supports multiple promisor remotes, which could help
with some of these remaining issues, but it's very hard to use them to
help, because a number of important features are missing.
The goal of the effort described in this document is to add these
important features.
We will call a "Large Object Promisor", or "LOP" in short, a promisor
remote which is used to store only large blobs and which is separate
from the main remote that should store the other Git objects and the
rest of the repos.
By extension, we will also call "Large Object Promisor", or LOP, the
effort described in this document to add a set of features to make it
easier to handle large blobs/files in Git by using LOPs.
This effort aims to especially improve things on the server side, and
especially for large blobs that are already compressed in a binary
format.
This effort aims to provide an alternative to Git LFS
(https://git-lfs.com/) and similar tools like git-annex
(https://git-annex.branchable.com/) for handling large files, even
though a complete alternative would very likely require other efforts
especially on the client side, where it would likely help to implement
a new object representation for large blobs as discussed in:
https://lore.kernel.org/git/xmqqbkdometi.fsf@gitster.g/
0) Non goals
------------
- We will not discuss those client side improvements here, as they
would require changes in different parts of Git than this effort.
+
So we don't pretend to fully replace Git LFS with only this effort,
but we nevertheless believe that it can significantly improve the
current situation on the server side, and that other separate
efforts could also improve the situation on the client side.
- In the same way, we are not going to discuss all the possible ways
to implement a LOP or their underlying object storage, or to
optimize how LOP works.
+
Our opinion is that the simplest solution for now is for LOPs to use
object storage through a remote helper (see section II.2 below for
more details) to store their objects. So we consider that this is the
default implementation. If there are improvements on top of this,
that's great, but our opinion is that such improvements are not
necessary for LOPs to already be useful. Such improvements are likely
a different technical topic, and can be taken care of separately
anyway.
+
So in particular we are not going to discuss pluggable ODBs or other
object database backends that could chunk large blobs, dedup the
chunks and store them efficiently. Sure, that would be a nice
improvement to store large blobs on the server side, but we believe
it can just be a separate effort as it's also not technically very
related to this effort.
+
We are also not going to discuss data transfer improvements between
LOPs and clients or servers. Sure, there might be some easy and very
effective optimizations there (as we know that objects on LOPs are
very likely incompressible and not deltifying well), but this can be
dealt with separately in a separate effort.
In other words, the goal of this document is not to talk about all the
possible ways to optimize how Git could handle large blobs, but to
describe how a LOP based solution can already work well and alleviate
a number of current issues in the context of Git clients and servers
sharing Git objects.
Even if LOPs are used not very efficiently, they can still be useful
and worth using in some cases, as we will see in more details
later in this document:
- they can make it simpler for clients to use promisor remotes and
therefore avoid fetching a lot of large blobs they might not need
locally,
- they can make it significantly cheaper or easier for servers to
host a significant part of the current repository content, and
even more to host content with larger blobs or more large blobs
than currently.
I) Issues with the current situation
------------------------------------
- Some statistics made on GitLab repos have shown that more than 75%
of the disk space is used by blobs that are larger than 1MB and
often in a binary format.
- So even if users could use Git LFS or similar tools to store a lot
of large blobs out of their repos, it's a fact that in practice they
don't do it as much as they probably should.
- On the server side ideally, the server should be able to decide for
itself how it stores things. It should not depend on users deciding
to use tools like Git LFS on some blobs or not.
- It's much more expensive to store large blobs that don't delta
compress well on regular fast seeking drives (like SSDs) than on
object storage (like Amazon S3 or GCP Buckets). Using fast drives
for regular Git repos makes sense though, as serving regular Git
content (blobs containing text or code) needs drives where seeking
is fast, but the content is relatively small. On the other hand,
object storage for Git LFS blobs makes sense as seeking speed is not
as important when dealing with large files, while costs are more
important. So the fact that users don't use Git LFS or similar tools
for a significant number of large blobs has likely some bad
consequences on the cost of repo storage for most Git hosting
platforms.
- Having large blobs handled in the same way as other blobs and Git
objects in Git repos instead of on object storage also has a cost in
increased memory and CPU usage, and therefore decreased performance,
when creating packfiles. (This is because Git tries to use delta
compression or zlib compression which is unlikely to work well on
already compressed binary content.) So it's not just a storage cost
increase.
- When a large blob has been committed into a repo, it might not be
possible to remove this blob from the repo without rewriting
history, even if the user then decides to use Git LFS or a similar
tool to handle it.
- In fact Git LFS and similar tools are not very flexible in letting
users change their minds about the blobs they should handle or not.
- Even when users are using Git LFS or similar tools, they are often
complaining that these tools require significant effort to set up,
learn and use correctly.
II) Main features of the "Large Object Promisors" solution
----------------------------------------------------------
The main features below should give a rough overview of how the
solution may work. Details about needed elements can be found in
following sections.
Even if each feature below is very useful for the full solution, it is
very likely to be also useful on its own in some cases where the full
solution is not required. However, we'll focus primarily on the big
picture here.
Also each feature doesn't need to be implemented entirely in Git
itself. Some could be scripts, hooks or helpers that are not part of
the Git repo. It would be helpful if those could be shared and
improved on collaboratively though. So we want to encourage sharing
them.
1) Large blobs are stored on LOPs
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Large blobs should be stored on special promisor remotes that we will
call "Large Object Promisors" or LOPs. These LOPs should be additional
remotes dedicated to contain large blobs especially those in binary
format. They should be used along with main remotes that contain the
other objects.
Note 1
++++++
To clarify, a LOP is a normal promisor remote, except that:
- it should store only large blobs,
- it should be separate from the main remote, so that the main remote
can focus on serving other objects and the rest of the repos (see
feature 4) below) and can use the LOP as a promisor remote for
itself.
Note 2
++++++
Git already makes it possible for a main remote to also be a promisor
remote storing both regular objects and large blobs for a client that
clones from it with a filter on blob size. But here we explicitly want
to avoid that.
Rationale
+++++++++
LOPs aim to be good at handling large blobs while main remotes are
already good at handling other objects.
Implementation
++++++++++++++
Git already has support for multiple promisor remotes, see
link:partial-clone.html#using-many-promisor-remotes[the partial clone documentation].
Also, Git already has support for partial clone using a filter on the
size of the blobs (with `git clone --filter=blob:limit=<size>`). Most
of the other main features below are based on these existing features
and are about making them easy and efficient to use for the purpose of
better handling large blobs.
2) LOPs can use object storage
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
LOPs can be implemented using object storage, like an Amazon S3 or GCP
Bucket or MinIO (which is open source under the GNU AGPLv3 license) to
actually store the large blobs, and can be accessed through a Git
remote helper (see linkgit:gitremote-helpers[7]) which makes the
underlying object storage appear like a remote to Git.
Note
++++
A LOP can be a promisor remote accessed using a remote helper by
both some clients and the main remote.
Rationale
+++++++++
This looks like the simplest way to create LOPs that can cheaply
handle many large blobs.
Implementation
++++++++++++++
Remote helpers are quite easy to write as shell scripts, but it might
be more efficient and maintainable to write them using other languages
like Go.
Some already exist under open source licenses, for example:
- https://github.com/awslabs/git-remote-s3
- https://gitlab.com/eric.p.ju/git-remote-gs
Other ways to implement LOPs are certainly possible, but the goal of
this document is not to discuss how to best implement a LOP or its
underlying object storage (see the "0) Non goals" section above).
3) LOP object storage can be Git LFS storage
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The underlying object storage that a LOP uses could also serve as
storage for large files handled by Git LFS.
Rationale
+++++++++
This would simplify the server side if it wants to both use a LOP and
act as a Git LFS server.
4) A main remote can offload to a LOP with a configurable threshold
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
On the server side, a main remote should have a way to offload to a
LOP all its blobs with a size over a configurable threshold.
Rationale
+++++++++
This makes it easy to set things up and to clean things up. For
example, an admin could use this to manually convert a repo not using
LOPs to a repo using a LOP. On a repo already using a LOP but where
some users would sometimes push large blobs, a cron job could use this
to regularly make sure the large blobs are moved to the LOP.
Implementation
++++++++++++++
Using something based on `git repack --filter=...` to separate the
blobs we want to offload from the other Git objects could be a good
idea. The missing part is to connect to the LOP, check if the blobs we
want to offload are already there and if not send them.
5) A main remote should try to remain clean from large blobs
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
A main remote should try to avoid containing a lot of oversize
blobs. For that purpose, it should offload as needed to a LOP and it
should have ways to prevent oversize blobs to be fetched, and also
perhaps pushed, into it.
Rationale
+++++++++
A main remote containing many oversize blobs would defeat the purpose
of LOPs.
Implementation
++++++++++++++
The way to offload to a LOP discussed in 4) above can be used to
regularly offload oversize blobs. About preventing oversize blobs from
being fetched into the repo see 6) below. About preventing oversize
blob pushes, a pre-receive hook could be used.
Also there are different scenarios in which large blobs could get
fetched into the main remote, for example:
- A client that doesn't implement the "promisor-remote" protocol
(described in 6) below) clones from the main remote.
- The main remote gets a request for information about a large blob
and is not able to get that information without fetching the blob
from the LOP.
It might not be possible to completely prevent all these scenarios
from happening. So the goal here should be to implement features that
make the fetching of large blobs less likely. For example adding a
`remote-object-info` command in the `git cat-file --batch` protocol
and its variants might make it possible for a main repo to respond to
some requests about large blobs without fetching them.
6) A protocol negotiation should happen when a client clones
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
When a client clones from a main repo, there should be a protocol
negotiation so that the server can advertise one or more LOPs and so
that the client and the server can discuss if the client could
directly use a LOP the server is advertising. If the client and the
server can agree on that, then the client would be able to get the
large blobs directly from the LOP and the server would not need to
fetch those blobs from the LOP to be able to serve the client.
Note
++++
For fetches instead of clones, a protocol negotiation might not always
happen, see the "What about fetches?" FAQ entry below for details.
Rationale
+++++++++
Security, configurability and efficiency of setting things up.
Implementation
++++++++++++++
A "promisor-remote" protocol v2 capability looks like a good way to
implement this. The way the client and server use this capability
could be controlled by configuration variables.
Information that the server could send to the client through that
protocol could be things like: LOP name, LOP URL, filter-spec (for
example `blob:limit=<size>`) or just size limit that should be used as
a filter when cloning, token to be used with the LOP, etc.
7) A client can offload to a LOP
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
When a client is using a LOP that is also a LOP of its main remote,
the client should be able to offload some large blobs it has fetched,
but might not need anymore, to the LOP.
Note
++++
It might depend on the context if it should be OK or not for clients
to offload large blobs they have created, instead of fetched, directly
to the LOP without the main remote checking them in some ways
(possibly using hooks or other tools).
This should be discussed and refined when we get closer to
implementing this feature.
Rationale
+++++++++
On the client, the easiest way to deal with unneeded large blobs is to
offload them.
Implementation
++++++++++++++
This is very similar to what 4) above is about, except on the client
side instead of the server side. So a good solution to 4) could likely
be adapted to work on the client side too.
There might be some security issues here, as there is no negotiation,
but they might be mitigated if the client can reuse a token it got
when cloning (see 6) above). Also if the large blobs were fetched from
a LOP, it is likely, and can easily be confirmed, that the LOP still
has them, so that they can just be removed from the client.
III) Benefits of using LOPs
---------------------------
Many benefits are related to the issues discussed in "I) Issues with
the current situation" above:
- No need to rewrite history when deciding which blobs are worth
handling separately than other objects, or when moving or removing
the threshold.
- If the protocol between client and server is developed and secured
enough, then many details might be setup on the server side only and
all the clients could then easily get all the configuration
information and use it to set themselves up mostly automatically.
- Storage costs benefits on the server side.
- Reduced memory and CPU needs on main remotes on the server side.
- Reduced storage needs on the client side.
IV) FAQ
-------
What about using multiple LOPs on the server and client side?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
That could perhaps be useful in some cases, but for now it's more
likely that in most cases a single LOP will be advertised by the
server and should be used by the client.
A case where it could be useful for a server to advertise multiple
LOPs is if a LOP is better for some users while a different LOP is
better for other users. For example some clients might have a better
connection to a LOP than others.
In those cases it's the responsibility of the server to have some
documentation to help clients. It could say for example something like
"Users in this part of the world might want to pick only LOP A as it
is likely to be better connected to them, while users in other parts
of the world should pick only LOP B for the same reason."
When should we trust or not trust the LOPs advertised by the server?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In some contexts, like in corporate setup where the server and all the
clients are parts of an internal network in a company where admins
have all the rights on every system, it's OK, and perhaps even a good
thing, if the clients fully trust the server, as it can help ensure
that all the clients are on the same page.
There are also contexts in which clients trust a code hosting platform
serving them some repos, but might not fully trust other users
managing or contributing to some of these repos. For example, the code
hosting platform could have hooks in place to check that any object it
receives doesn't contain malware or otherwise bad content. In this
case it might be OK for the client to use a main remote and its LOP if
they are both hosted by the code hosting platform, but not if the LOP
is hosted elsewhere (where the content is not checked).
In other contexts, a client should just not trust a server.
So there should be different ways to configure how the client should
behave when a server advertises a LOP to it at clone time.
As the basic elements that a server can advertise about a LOP are a
LOP name and a LOP URL, the client should base its decision about
accepting a LOP on these elements.
One simple way to be very strict in the LOP it accepts is for example
for the client to check that the LOP is already configured on the
client with the same name and URL as what the server advertises.
In general default and "safe" settings should require that the LOP are
configured on the client separately from the "promisor-remote"
protocol and that the client accepts a LOP only when information about
it from the protocol matches what has been already configured
separately.
What about LOP names?
~~~~~~~~~~~~~~~~~~~~~
In some contexts, for example if the clients sometimes fetch from each
other, it can be a good idea for all the clients to use the same names
for all the remotes they use, including LOPs.
In other contexts, each client might want to be able to give the name
it wants to each remote, including each LOP, it interacts with.
So there should be different ways to configure how the client accepts
or not the LOP name the server advertises.
If a default or "safe" setting is used, then as such a setting should
require that the LOP be configured separately, then the name would be
configured separately and there is no risk that the server could
dictate a name to a client.
Could the main remote be bogged down by old or paranoid clients?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Yes, it could happen if there are too many clients that are either
unwilling to trust the main remote or that just don't implement the
"promisor-remote" protocol because they are too old or not fully
compatible with the 'git' client.
When serving such a client, the main remote has no other choice than
to first fetch from its LOP, to then be able to provide to the client
everything it requested. So the main remote, even if it has cleanup
mechanisms (see section II.4 above), would be burdened at least
temporarily with the large blobs it had to fetch from its LOP.
Not behaving like this would be breaking backward compatibility, and
could be seen as segregating clients. For example, it might be
possible to implement a special mode that allows the server to just
reject clients that don't implement the "promisor-remote" protocol or
aren't willing to trust the main remote. This mode might be useful in
a special context like a corporate environment. There is no plan to
implement such a mode though, and this should be discussed separately
later anyway.
A better way to proceed is probably for the main remote to show a
message telling clients that don't implement the protocol or are
unwilling to accept the advertised LOP(s) that they would get faster
clone and fetches by upgrading client software or properly setting
them up to accept LOP(s).
Waiting for clients to upgrade, monitoring these upgrades and limiting
the use of LOPs to repos that are not very frequently accessed might
be other good ways to make sure that some benefits are still reaped
from LOPs. Over time, as more and more clients upgrade and benefit
from LOPs, using them in more and more frequently accessed repos will
become worth it.
Corporate environments, where it might be easier to make sure that all
the clients are up-to-date and properly configured, could hopefully
benefit more and earlier from using LOPs.
What about fetches?
~~~~~~~~~~~~~~~~~~~
There are different kinds of fetches. A regular fetch happens when
some refs have been updated on the server and the client wants the ref
updates and possibly the new objects added with them. A "backfill" or
"lazy" fetch, on the contrary, happens when the client needs to use
some objects it already knows about but doesn't have because they are
on a promisor remote.
Regular fetch
+++++++++++++
In a regular fetch, the client will contact the main remote and a
protocol negotiation will happen between them. It's a good thing that
a protocol negotiation happens every time, as the configuration on the
client or the main remote could have changed since the previous
protocol negotiation. In this case, the new protocol negotiation
should ensure that the new fetch will happen in a way that satisfies
the new configuration of both the client and the server.
In most cases though, the configurations on the client and the main
remote will not have changed between 2 fetches or between the initial
clone and a subsequent fetch. This means that the result of a new
protocol negotiation will be the same as the previous result, so the
new fetch will happen in the same way as the previous clone or fetch,
using, or not using, the same LOP(s) as last time.
"Backfill" or "lazy" fetch
++++++++++++++++++++++++++
When there is a backfill fetch, the client doesn't necessarily contact
the main remote first. It will try to fetch from its promisor remotes
in the order they appear in the config file, except that a remote
configured using the `extensions.partialClone` config variable will be
tried last. See
link:partial-clone.html#using-many-promisor-remotes[the partial clone documentation].
This is not new with this effort. In fact this is how multiple remotes
have already been working for around 5 years.
When using LOPs, having the main remote configured using
`extensions.partialClone`, so it's tried last, makes sense, as missing
objects should only be large blobs that are on LOPs.
This means that a protocol negotiation will likely not happen as the
missing objects will be fetched from the LOPs, and then there will be
nothing left to fetch from the main remote.
To secure that, it could be a good idea for LOPs to require a token
from the client when it fetches from them. The client could get the
token when performing a protocol negotiation with the main remote (see
section II.6 above).
V) Future improvements
----------------------
It is expected that at the beginning using LOPs will be mostly worth
it either in a corporate context where the Git version that clients
use can easily be controlled, or on repos that are infrequently
accessed. (See the "Could the main remote be bogged down by old or
paranoid clients?" section in the FAQ above.)
Over time, as more and more clients upgrade to a version that
implements the "promisor-remote" protocol v2 capability described
above in section II.6), it will be worth it to use LOPs more widely.
A lot of improvements may also help using LOPs more widely. Some of
these improvements are part of the scope of this document like the
following:
- Implementing a "remote-object-info" command in the
`git cat-file --batch` protocol and its variants to allow main
remotes to respond to requests about large blobs without fetching
them. (Eric Ju has started working on this based on previous work
by Calvin Wan.)
- Creating better cleanup and offload mechanisms for main remotes
and clients to prevent accumulation of large blobs.
- Developing more sophisticated protocol negotiation capabilities
between clients and servers for handling LOPs, for example adding
a filter-spec (e.g., blob:limit=<size>) or size limit for
filtering when cloning, or adding a token for LOP authentication.
- Improving security measures for LOP access, particularly around
token handling and authentication.
- Developing standardized ways to configure and manage multiple LOPs
across different environments. Especially in the case where
different LOPs serve the same content to clients in different
geographical locations, there is a need for replication or
synchronization between LOPs.
Some improvements, including some that have been mentioned in the "0)
Non Goals" section of this document, are out of the scope of this
document:
- Implementing a new object representation for large blobs on the
client side.
- Developing pluggable ODBs or other object database backends that
could chunk large blobs, dedup the chunks and store them
efficiently.
- Optimizing data transfer between LOPs and clients/servers,
particularly for incompressible and non-deltifying content.
- Creating improved client side tools for managing large objects
more effectively, for example tools for migrating from Git LFS or
git-annex, or tools to find which objects could be offloaded and
how much disk space could be reclaimed by offloading them.
Some improvements could be seen as part of the scope of this document,
but might already have their own separate projects from the Git
project, like:
- Improving existing remote helpers to access object storage or
developing new ones.
- Improving existing object storage solutions or developing new
ones.
Even though all the above improvements may help, this document and the
LOP effort should try to focus, at least first, on a relatively small
number of improvements mostly those that are in its current scope.
For example introducing pluggable ODBs and a new object database
backend is likely a multi-year effort on its own that can happen
separately in parallel. It has different technical requirements,
touches other part of the Git code base and should have its own design
document(s).

View File

@@ -0,0 +1,50 @@
Long-running process protocol
=============================
This protocol is used when Git needs to communicate with an external
process throughout the entire life of a single Git command. All
communication is in pkt-line format (see linkgit:gitprotocol-common[5])
over standard input and standard output.
Handshake
---------
Git starts by sending a welcome message (for example,
"git-filter-client"), a list of supported protocol version numbers, and
a flush packet. Git expects to read the welcome message with "server"
instead of "client" (for example, "git-filter-server"), exactly one
protocol version number from the previously sent list, and a flush
packet. All further communication will be based on the selected version.
The remaining protocol description below documents "version=2". Please
note that "version=42" in the example below does not exist and is only
there to illustrate how the protocol would look like with more than one
version.
After the version negotiation Git sends a list of all capabilities that
it supports and a flush packet. Git expects to read a list of desired
capabilities, which must be a subset of the supported capabilities list,
and a flush packet as response:
------------------------
packet: git> git-filter-client
packet: git> version=2
packet: git> version=42
packet: git> 0000
packet: git< git-filter-server
packet: git< version=2
packet: git< 0000
packet: git> capability=clean
packet: git> capability=smudge
packet: git> capability=not-yet-invented
packet: git> 0000
packet: git< capability=clean
packet: git< capability=smudge
packet: git< 0000
------------------------
Shutdown
--------
Git will close
the command pipe on exit. The filter is expected to detect EOF
and exit gracefully on its own. Git will wait until the filter
process has stopped.

View File

@@ -0,0 +1,511 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="en">
<head>
<meta charset="UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=edge"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<meta name="generator" content="Asciidoctor 2.0.23"/>
<title>Long-running process protocol</title>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"/>
<style>
/*! Asciidoctor default stylesheet | MIT License | https://asciidoctor.org */
/* Uncomment the following line when using as a custom stylesheet */
/* @import "https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"; */
html{font-family:sans-serif;-webkit-text-size-adjust:100%}
a{background:none}
a:focus{outline:thin dotted}
a:active,a:hover{outline:0}
h1{font-size:2em;margin:.67em 0}
b,strong{font-weight:bold}
abbr{font-size:.9em}
abbr[title]{cursor:help;border-bottom:1px dotted #dddddf;text-decoration:none}
dfn{font-style:italic}
hr{height:0}
mark{background:#ff0;color:#000}
code,kbd,pre,samp{font-family:monospace;font-size:1em}
pre{white-space:pre-wrap}
q{quotes:"\201C" "\201D" "\2018" "\2019"}
small{font-size:80%}
sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline}
sup{top:-.5em}
sub{bottom:-.25em}
img{border:0}
svg:not(:root){overflow:hidden}
figure{margin:0}
audio,video{display:inline-block}
audio:not([controls]){display:none;height:0}
fieldset{border:1px solid silver;margin:0 2px;padding:.35em .625em .75em}
legend{border:0;padding:0}
button,input,select,textarea{font-family:inherit;font-size:100%;margin:0}
button,input{line-height:normal}
button,select{text-transform:none}
button,html input[type=button],input[type=reset],input[type=submit]{-webkit-appearance:button;cursor:pointer}
button[disabled],html input[disabled]{cursor:default}
input[type=checkbox],input[type=radio]{padding:0}
button::-moz-focus-inner,input::-moz-focus-inner{border:0;padding:0}
textarea{overflow:auto;vertical-align:top}
table{border-collapse:collapse;border-spacing:0}
*,::before,::after{box-sizing:border-box}
html,body{font-size:100%}
body{background:#fff;color:rgba(0,0,0,.8);padding:0;margin:0;font-family:"Noto Serif","DejaVu Serif",serif;line-height:1;position:relative;cursor:auto;-moz-tab-size:4;-o-tab-size:4;tab-size:4;word-wrap:anywhere;-moz-osx-font-smoothing:grayscale;-webkit-font-smoothing:antialiased}
a:hover{cursor:pointer}
img,object,embed{max-width:100%;height:auto}
object,embed{height:100%}
img{-ms-interpolation-mode:bicubic}
.left{float:left!important}
.right{float:right!important}
.text-left{text-align:left!important}
.text-right{text-align:right!important}
.text-center{text-align:center!important}
.text-justify{text-align:justify!important}
.hide{display:none}
img,object,svg{display:inline-block;vertical-align:middle}
textarea{height:auto;min-height:50px}
select{width:100%}
.subheader,.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{line-height:1.45;color:#7a2518;font-weight:400;margin-top:0;margin-bottom:.25em}
div,dl,dt,dd,ul,ol,li,h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6,pre,form,p,blockquote,th,td{margin:0;padding:0}
a{color:#2156a5;text-decoration:underline;line-height:inherit}
a:hover,a:focus{color:#1d4b8f}
a img{border:0}
p{line-height:1.6;margin-bottom:1.25em;text-rendering:optimizeLegibility}
p aside{font-size:.875em;line-height:1.35;font-style:italic}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{font-family:"Open Sans","DejaVu Sans",sans-serif;font-weight:300;font-style:normal;color:#ba3925;text-rendering:optimizeLegibility;margin-top:1em;margin-bottom:.5em;line-height:1.0125em}
h1 small,h2 small,h3 small,#toctitle small,.sidebarblock>.content>.title small,h4 small,h5 small,h6 small{font-size:60%;color:#e99b8f;line-height:0}
h1{font-size:2.125em}
h2{font-size:1.6875em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.375em}
h4,h5{font-size:1.125em}
h6{font-size:1em}
hr{border:solid #dddddf;border-width:1px 0 0;clear:both;margin:1.25em 0 1.1875em}
em,i{font-style:italic;line-height:inherit}
strong,b{font-weight:bold;line-height:inherit}
small{font-size:60%;line-height:inherit}
code{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:400;color:rgba(0,0,0,.9)}
ul,ol,dl{line-height:1.6;margin-bottom:1.25em;list-style-position:outside;font-family:inherit}
ul,ol{margin-left:1.5em}
ul li ul,ul li ol{margin-left:1.25em;margin-bottom:0}
ul.circle{list-style-type:circle}
ul.disc{list-style-type:disc}
ul.square{list-style-type:square}
ul.circle ul:not([class]),ul.disc ul:not([class]),ul.square ul:not([class]){list-style:inherit}
ol li ul,ol li ol{margin-left:1.25em;margin-bottom:0}
dl dt{margin-bottom:.3125em;font-weight:bold}
dl dd{margin-bottom:1.25em}
blockquote{margin:0 0 1.25em;padding:.5625em 1.25em 0 1.1875em;border-left:1px solid #ddd}
blockquote,blockquote p{line-height:1.6;color:rgba(0,0,0,.85)}
@media screen and (min-width:768px){h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2}
h1{font-size:2.75em}
h2{font-size:2.3125em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.6875em}
h4{font-size:1.4375em}}
table{background:#fff;margin-bottom:1.25em;border:1px solid #dedede;word-wrap:normal}
table thead,table tfoot{background:#f7f8f7}
table thead tr th,table thead tr td,table tfoot tr th,table tfoot tr td{padding:.5em .625em .625em;font-size:inherit;color:rgba(0,0,0,.8);text-align:left}
table tr th,table tr td{padding:.5625em .625em;font-size:inherit;color:rgba(0,0,0,.8)}
table tr.even,table tr.alt{background:#f8f8f7}
table thead tr th,table tfoot tr th,table tbody tr td,table tr td,table tfoot tr td{line-height:1.6}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2;word-spacing:-.05em}
h1 strong,h2 strong,h3 strong,#toctitle strong,.sidebarblock>.content>.title strong,h4 strong,h5 strong,h6 strong{font-weight:400}
.center{margin-left:auto;margin-right:auto}
.stretch{width:100%}
.clearfix::before,.clearfix::after,.float-group::before,.float-group::after{content:" ";display:table}
.clearfix::after,.float-group::after{clear:both}
:not(pre).nobreak{word-wrap:normal}
:not(pre).nowrap{white-space:nowrap}
:not(pre).pre-wrap{white-space:pre-wrap}
:not(pre):not([class^=L])>code{font-size:.9375em;font-style:normal!important;letter-spacing:0;padding:.1em .5ex;word-spacing:-.15em;background:#f7f7f8;border-radius:4px;line-height:1.45;text-rendering:optimizeSpeed}
pre{color:rgba(0,0,0,.9);font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;line-height:1.45;text-rendering:optimizeSpeed}
pre code,pre pre{color:inherit;font-size:inherit;line-height:inherit}
pre>code{display:block}
pre.nowrap,pre.nowrap pre{white-space:pre;word-wrap:normal}
em em{font-style:normal}
strong strong{font-weight:400}
.keyseq{color:rgba(51,51,51,.8)}
kbd{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;display:inline-block;color:rgba(0,0,0,.8);font-size:.65em;line-height:1.45;background:#f7f7f7;border:1px solid #ccc;border-radius:3px;box-shadow:0 1px 0 rgba(0,0,0,.2),inset 0 0 0 .1em #fff;margin:0 .15em;padding:.2em .5em;vertical-align:middle;position:relative;top:-.1em;white-space:nowrap}
.keyseq kbd:first-child{margin-left:0}
.keyseq kbd:last-child{margin-right:0}
.menuseq,.menuref{color:#000}
.menuseq b:not(.caret),.menuref{font-weight:inherit}
.menuseq{word-spacing:-.02em}
.menuseq b.caret{font-size:1.25em;line-height:.8}
.menuseq i.caret{font-weight:bold;text-align:center;width:.45em}
b.button::before,b.button::after{position:relative;top:-1px;font-weight:400}
b.button::before{content:"[";padding:0 3px 0 2px}
b.button::after{content:"]";padding:0 2px 0 3px}
p a>code:hover{color:rgba(0,0,0,.9)}
#header,#content,#footnotes,#footer{width:100%;margin:0 auto;max-width:62.5em;*zoom:1;position:relative;padding-left:.9375em;padding-right:.9375em}
#header::before,#header::after,#content::before,#content::after,#footnotes::before,#footnotes::after,#footer::before,#footer::after{content:" ";display:table}
#header::after,#content::after,#footnotes::after,#footer::after{clear:both}
#content{margin-top:1.25em}
#content::before{content:none}
#header>h1:first-child{color:rgba(0,0,0,.85);margin-top:2.25rem;margin-bottom:0}
#header>h1:first-child+#toc{margin-top:8px;border-top:1px solid #dddddf}
#header>h1:only-child{border-bottom:1px solid #dddddf;padding-bottom:8px}
#header .details{border-bottom:1px solid #dddddf;line-height:1.45;padding-top:.25em;padding-bottom:.25em;padding-left:.25em;color:rgba(0,0,0,.6);display:flex;flex-flow:row wrap}
#header .details span:first-child{margin-left:-.125em}
#header .details span.email a{color:rgba(0,0,0,.85)}
#header .details br{display:none}
#header .details br+span::before{content:"\00a0\2013\00a0"}
#header .details br+span.author::before{content:"\00a0\22c5\00a0";color:rgba(0,0,0,.85)}
#header .details br+span#revremark::before{content:"\00a0|\00a0"}
#header #revnumber{text-transform:capitalize}
#header #revnumber::after{content:"\00a0"}
#content>h1:first-child:not([class]){color:rgba(0,0,0,.85);border-bottom:1px solid #dddddf;padding-bottom:8px;margin-top:0;padding-top:1rem;margin-bottom:1.25rem}
#toc{border-bottom:1px solid #e7e7e9;padding-bottom:.5em}
#toc>ul{margin-left:.125em}
#toc ul.sectlevel0>li>a{font-style:italic}
#toc ul.sectlevel0 ul.sectlevel1{margin:.5em 0}
#toc ul{font-family:"Open Sans","DejaVu Sans",sans-serif;list-style-type:none}
#toc li{line-height:1.3334;margin-top:.3334em}
#toc a{text-decoration:none}
#toc a:active{text-decoration:underline}
#toctitle{color:#7a2518;font-size:1.2em}
@media screen and (min-width:768px){#toctitle{font-size:1.375em}
body.toc2{padding-left:15em;padding-right:0}
body.toc2 #header>h1:nth-last-child(2){border-bottom:1px solid #dddddf;padding-bottom:8px}
#toc.toc2{margin-top:0!important;background:#f8f8f7;position:fixed;width:15em;left:0;top:0;border-right:1px solid #e7e7e9;border-top-width:0!important;border-bottom-width:0!important;z-index:1000;padding:1.25em 1em;height:100%;overflow:auto}
#toc.toc2 #toctitle{margin-top:0;margin-bottom:.8rem;font-size:1.2em}
#toc.toc2>ul{font-size:.9em;margin-bottom:0}
#toc.toc2 ul ul{margin-left:0;padding-left:1em}
#toc.toc2 ul.sectlevel0 ul.sectlevel1{padding-left:0;margin-top:.5em;margin-bottom:.5em}
body.toc2.toc-right{padding-left:0;padding-right:15em}
body.toc2.toc-right #toc.toc2{border-right-width:0;border-left:1px solid #e7e7e9;left:auto;right:0}}
@media screen and (min-width:1280px){body.toc2{padding-left:20em;padding-right:0}
#toc.toc2{width:20em}
#toc.toc2 #toctitle{font-size:1.375em}
#toc.toc2>ul{font-size:.95em}
#toc.toc2 ul ul{padding-left:1.25em}
body.toc2.toc-right{padding-left:0;padding-right:20em}}
#content #toc{border:1px solid #e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;border-radius:4px}
#content #toc>:first-child{margin-top:0}
#content #toc>:last-child{margin-bottom:0}
#footer{max-width:none;background:rgba(0,0,0,.8);padding:1.25em}
#footer-text{color:hsla(0,0%,100%,.8);line-height:1.44}
#content{margin-bottom:.625em}
.sect1{padding-bottom:.625em}
@media screen and (min-width:768px){#content{margin-bottom:1.25em}
.sect1{padding-bottom:1.25em}}
.sect1:last-child{padding-bottom:0}
.sect1+.sect1{border-top:1px solid #e7e7e9}
#content h1>a.anchor,h2>a.anchor,h3>a.anchor,#toctitle>a.anchor,.sidebarblock>.content>.title>a.anchor,h4>a.anchor,h5>a.anchor,h6>a.anchor{position:absolute;z-index:1001;width:1.5ex;margin-left:-1.5ex;display:block;text-decoration:none!important;visibility:hidden;text-align:center;font-weight:400}
#content h1>a.anchor::before,h2>a.anchor::before,h3>a.anchor::before,#toctitle>a.anchor::before,.sidebarblock>.content>.title>a.anchor::before,h4>a.anchor::before,h5>a.anchor::before,h6>a.anchor::before{content:"\00A7";font-size:.85em;display:block;padding-top:.1em}
#content h1:hover>a.anchor,#content h1>a.anchor:hover,h2:hover>a.anchor,h2>a.anchor:hover,h3:hover>a.anchor,#toctitle:hover>a.anchor,.sidebarblock>.content>.title:hover>a.anchor,h3>a.anchor:hover,#toctitle>a.anchor:hover,.sidebarblock>.content>.title>a.anchor:hover,h4:hover>a.anchor,h4>a.anchor:hover,h5:hover>a.anchor,h5>a.anchor:hover,h6:hover>a.anchor,h6>a.anchor:hover{visibility:visible}
#content h1>a.link,h2>a.link,h3>a.link,#toctitle>a.link,.sidebarblock>.content>.title>a.link,h4>a.link,h5>a.link,h6>a.link{color:#ba3925;text-decoration:none}
#content h1>a.link:hover,h2>a.link:hover,h3>a.link:hover,#toctitle>a.link:hover,.sidebarblock>.content>.title>a.link:hover,h4>a.link:hover,h5>a.link:hover,h6>a.link:hover{color:#a53221}
details,.audioblock,.imageblock,.literalblock,.listingblock,.stemblock,.videoblock{margin-bottom:1.25em}
details{margin-left:1.25rem}
details>summary{cursor:pointer;display:block;position:relative;line-height:1.6;margin-bottom:.625rem;outline:none;-webkit-tap-highlight-color:transparent}
details>summary::-webkit-details-marker{display:none}
details>summary::before{content:"";border:solid transparent;border-left:solid;border-width:.3em 0 .3em .5em;position:absolute;top:.5em;left:-1.25rem;transform:translateX(15%)}
details[open]>summary::before{border:solid transparent;border-top:solid;border-width:.5em .3em 0;transform:translateY(15%)}
details>summary::after{content:"";width:1.25rem;height:1em;position:absolute;top:.3em;left:-1.25rem}
.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{text-rendering:optimizeLegibility;text-align:left;font-family:"Noto Serif","DejaVu Serif",serif;font-size:1rem;font-style:italic}
table.tableblock.fit-content>caption.title{white-space:nowrap;width:0}
.paragraph.lead>p,#preamble>.sectionbody>[class=paragraph]:first-of-type p{font-size:1.21875em;line-height:1.6;color:rgba(0,0,0,.85)}
.admonitionblock>table{border-collapse:separate;border:0;background:none;width:100%}
.admonitionblock>table td.icon{text-align:center;width:80px}
.admonitionblock>table td.icon img{max-width:none}
.admonitionblock>table td.icon .title{font-weight:bold;font-family:"Open Sans","DejaVu Sans",sans-serif;text-transform:uppercase}
.admonitionblock>table td.content{padding-left:1.125em;padding-right:1.25em;border-left:1px solid #dddddf;color:rgba(0,0,0,.6);word-wrap:anywhere}
.admonitionblock>table td.content>:last-child>:last-child{margin-bottom:0}
.exampleblock>.content{border:1px solid #e6e6e6;margin-bottom:1.25em;padding:1.25em;background:#fff;border-radius:4px}
.sidebarblock{border:1px solid #dbdbd6;margin-bottom:1.25em;padding:1.25em;background:#f3f3f2;border-radius:4px}
.sidebarblock>.content>.title{color:#7a2518;margin-top:0;text-align:center}
.exampleblock>.content>:first-child,.sidebarblock>.content>:first-child{margin-top:0}
.exampleblock>.content>:last-child,.exampleblock>.content>:last-child>:last-child,.exampleblock>.content .olist>ol>li:last-child>:last-child,.exampleblock>.content .ulist>ul>li:last-child>:last-child,.exampleblock>.content .qlist>ol>li:last-child>:last-child,.sidebarblock>.content>:last-child,.sidebarblock>.content>:last-child>:last-child,.sidebarblock>.content .olist>ol>li:last-child>:last-child,.sidebarblock>.content .ulist>ul>li:last-child>:last-child,.sidebarblock>.content .qlist>ol>li:last-child>:last-child{margin-bottom:0}
.literalblock pre,.listingblock>.content>pre{border-radius:4px;overflow-x:auto;padding:1em;font-size:.8125em}
@media screen and (min-width:768px){.literalblock pre,.listingblock>.content>pre{font-size:.90625em}}
@media screen and (min-width:1280px){.literalblock pre,.listingblock>.content>pre{font-size:1em}}
.literalblock pre,.listingblock>.content>pre:not(.highlight),.listingblock>.content>pre[class=highlight],.listingblock>.content>pre[class^="highlight "]{background:#f7f7f8}
.literalblock.output pre{color:#f7f7f8;background:rgba(0,0,0,.9)}
.listingblock>.content{position:relative}
.listingblock code[data-lang]::before{display:none;content:attr(data-lang);position:absolute;font-size:.75em;top:.425rem;right:.5rem;line-height:1;text-transform:uppercase;color:inherit;opacity:.5}
.listingblock:hover code[data-lang]::before{display:block}
.listingblock.terminal pre .command::before{content:attr(data-prompt);padding-right:.5em;color:inherit;opacity:.5}
.listingblock.terminal pre .command:not([data-prompt])::before{content:"$"}
.listingblock pre.highlightjs{padding:0}
.listingblock pre.highlightjs>code{padding:1em;border-radius:4px}
.listingblock pre.prettyprint{border-width:0}
.prettyprint{background:#f7f7f8}
pre.prettyprint .linenums{line-height:1.45;margin-left:2em}
pre.prettyprint li{background:none;list-style-type:inherit;padding-left:0}
pre.prettyprint li code[data-lang]::before{opacity:1}
pre.prettyprint li:not(:first-child) code[data-lang]::before{display:none}
table.linenotable{border-collapse:separate;border:0;margin-bottom:0;background:none}
table.linenotable td[class]{color:inherit;vertical-align:top;padding:0;line-height:inherit;white-space:normal}
table.linenotable td.code{padding-left:.75em}
table.linenotable td.linenos,pre.pygments .linenos{border-right:1px solid;opacity:.35;padding-right:.5em;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}
pre.pygments span.linenos{display:inline-block;margin-right:.75em}
.quoteblock{margin:0 1em 1.25em 1.5em;display:table}
.quoteblock:not(.excerpt)>.title{margin-left:-1.5em;margin-bottom:.75em}
.quoteblock blockquote,.quoteblock p{color:rgba(0,0,0,.85);font-size:1.15rem;line-height:1.75;word-spacing:.1em;letter-spacing:0;font-style:italic;text-align:justify}
.quoteblock blockquote{margin:0;padding:0;border:0}
.quoteblock blockquote::before{content:"\201c";float:left;font-size:2.75em;font-weight:bold;line-height:.6em;margin-left:-.6em;color:#7a2518;text-shadow:0 1px 2px rgba(0,0,0,.1)}
.quoteblock blockquote>.paragraph:last-child p{margin-bottom:0}
.quoteblock .attribution{margin-top:.75em;margin-right:.5ex;text-align:right}
.verseblock{margin:0 1em 1.25em}
.verseblock pre{font-family:"Open Sans","DejaVu Sans",sans-serif;font-size:1.15rem;color:rgba(0,0,0,.85);font-weight:300;text-rendering:optimizeLegibility}
.verseblock pre strong{font-weight:400}
.verseblock .attribution{margin-top:1.25rem;margin-left:.5ex}
.quoteblock .attribution,.verseblock .attribution{font-size:.9375em;line-height:1.45;font-style:italic}
.quoteblock .attribution br,.verseblock .attribution br{display:none}
.quoteblock .attribution cite,.verseblock .attribution cite{display:block;letter-spacing:-.025em;color:rgba(0,0,0,.6)}
.quoteblock.abstract blockquote::before,.quoteblock.excerpt blockquote::before,.quoteblock .quoteblock blockquote::before{display:none}
.quoteblock.abstract blockquote,.quoteblock.abstract p,.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{line-height:1.6;word-spacing:0}
.quoteblock.abstract{margin:0 1em 1.25em;display:block}
.quoteblock.abstract>.title{margin:0 0 .375em;font-size:1.15em;text-align:center}
.quoteblock.excerpt>blockquote,.quoteblock .quoteblock{padding:0 0 .25em 1em;border-left:.25em solid #dddddf}
.quoteblock.excerpt,.quoteblock .quoteblock{margin-left:0}
.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{color:inherit;font-size:1.0625rem}
.quoteblock.excerpt .attribution,.quoteblock .quoteblock .attribution{color:inherit;font-size:.85rem;text-align:left;margin-right:0}
p.tableblock:last-child{margin-bottom:0}
td.tableblock>.content{margin-bottom:1.25em;word-wrap:anywhere}
td.tableblock>.content>:last-child{margin-bottom:-1.25em}
table.tableblock,th.tableblock,td.tableblock{border:0 solid #dedede}
table.grid-all>*>tr>*{border-width:1px}
table.grid-cols>*>tr>*{border-width:0 1px}
table.grid-rows>*>tr>*{border-width:1px 0}
table.frame-all{border-width:1px}
table.frame-ends{border-width:1px 0}
table.frame-sides{border-width:0 1px}
table.frame-none>colgroup+*>:first-child>*,table.frame-sides>colgroup+*>:first-child>*{border-top-width:0}
table.frame-none>:last-child>:last-child>*,table.frame-sides>:last-child>:last-child>*{border-bottom-width:0}
table.frame-none>*>tr>:first-child,table.frame-ends>*>tr>:first-child{border-left-width:0}
table.frame-none>*>tr>:last-child,table.frame-ends>*>tr>:last-child{border-right-width:0}
table.stripes-all>*>tr,table.stripes-odd>*>tr:nth-of-type(odd),table.stripes-even>*>tr:nth-of-type(even),table.stripes-hover>*>tr:hover{background:#f8f8f7}
th.halign-left,td.halign-left{text-align:left}
th.halign-right,td.halign-right{text-align:right}
th.halign-center,td.halign-center{text-align:center}
th.valign-top,td.valign-top{vertical-align:top}
th.valign-bottom,td.valign-bottom{vertical-align:bottom}
th.valign-middle,td.valign-middle{vertical-align:middle}
table thead th,table tfoot th{font-weight:bold}
tbody tr th{background:#f7f8f7}
tbody tr th,tbody tr th p,tfoot tr th,tfoot tr th p{color:rgba(0,0,0,.8);font-weight:bold}
p.tableblock>code:only-child{background:none;padding:0}
p.tableblock{font-size:1em}
ol{margin-left:1.75em}
ul li ol{margin-left:1.5em}
dl dd{margin-left:1.125em}
dl dd:last-child,dl dd:last-child>:last-child{margin-bottom:0}
li p,ul dd,ol dd,.olist .olist,.ulist .ulist,.ulist .olist,.olist .ulist{margin-bottom:.625em}
ul.checklist,ul.none,ol.none,ul.no-bullet,ol.no-bullet,ol.unnumbered,ul.unstyled,ol.unstyled{list-style-type:none}
ul.no-bullet,ol.no-bullet,ol.unnumbered{margin-left:.625em}
ul.unstyled,ol.unstyled{margin-left:0}
li>p:empty:only-child::before{content:"";display:inline-block}
ul.checklist>li>p:first-child{margin-left:-1em}
ul.checklist>li>p:first-child>.fa-square-o:first-child,ul.checklist>li>p:first-child>.fa-check-square-o:first-child{width:1.25em;font-size:.8em;position:relative;bottom:.125em}
ul.checklist>li>p:first-child>input[type=checkbox]:first-child{margin-right:.25em}
ul.inline{display:flex;flex-flow:row wrap;list-style:none;margin:0 0 .625em -1.25em}
ul.inline>li{margin-left:1.25em}
.unstyled dl dt{font-weight:400;font-style:normal}
ol.arabic{list-style-type:decimal}
ol.decimal{list-style-type:decimal-leading-zero}
ol.loweralpha{list-style-type:lower-alpha}
ol.upperalpha{list-style-type:upper-alpha}
ol.lowerroman{list-style-type:lower-roman}
ol.upperroman{list-style-type:upper-roman}
ol.lowergreek{list-style-type:lower-greek}
.hdlist>table,.colist>table{border:0;background:none}
.hdlist>table>tbody>tr,.colist>table>tbody>tr{background:none}
td.hdlist1,td.hdlist2{vertical-align:top;padding:0 .625em}
td.hdlist1{font-weight:bold;padding-bottom:1.25em}
td.hdlist2{word-wrap:anywhere}
.literalblock+.colist,.listingblock+.colist{margin-top:-.5em}
.colist td:not([class]):first-child{padding:.4em .75em 0;line-height:1;vertical-align:top}
.colist td:not([class]):first-child img{max-width:none}
.colist td:not([class]):last-child{padding:.25em 0}
.thumb,.th{line-height:0;display:inline-block;border:4px solid #fff;box-shadow:0 0 0 1px #ddd}
.imageblock.left{margin:.25em .625em 1.25em 0}
.imageblock.right{margin:.25em 0 1.25em .625em}
.imageblock>.title{margin-bottom:0}
.imageblock.thumb,.imageblock.th{border-width:6px}
.imageblock.thumb>.title,.imageblock.th>.title{padding:0 .125em}
.image.left,.image.right{margin-top:.25em;margin-bottom:.25em;display:inline-block;line-height:0}
.image.left{margin-right:.625em}
.image.right{margin-left:.625em}
a.image{text-decoration:none;display:inline-block}
a.image object{pointer-events:none}
sup.footnote,sup.footnoteref{font-size:.875em;position:static;vertical-align:super}
sup.footnote a,sup.footnoteref a{text-decoration:none}
sup.footnote a:active,sup.footnoteref a:active,#footnotes .footnote a:first-of-type:active{text-decoration:underline}
#footnotes{padding-top:.75em;padding-bottom:.75em;margin-bottom:.625em}
#footnotes hr{width:20%;min-width:6.25em;margin:-.25em 0 .75em;border-width:1px 0 0}
#footnotes .footnote{padding:0 .375em 0 .225em;line-height:1.3334;font-size:.875em;margin-left:1.2em;margin-bottom:.2em}
#footnotes .footnote a:first-of-type{font-weight:bold;text-decoration:none;margin-left:-1.05em}
#footnotes .footnote:last-of-type{margin-bottom:0}
#content #footnotes{margin-top:-.625em;margin-bottom:0;padding:.75em 0}
div.unbreakable{page-break-inside:avoid}
.big{font-size:larger}
.small{font-size:smaller}
.underline{text-decoration:underline}
.overline{text-decoration:overline}
.line-through{text-decoration:line-through}
.aqua{color:#00bfbf}
.aqua-background{background:#00fafa}
.black{color:#000}
.black-background{background:#000}
.blue{color:#0000bf}
.blue-background{background:#0000fa}
.fuchsia{color:#bf00bf}
.fuchsia-background{background:#fa00fa}
.gray{color:#606060}
.gray-background{background:#7d7d7d}
.green{color:#006000}
.green-background{background:#007d00}
.lime{color:#00bf00}
.lime-background{background:#00fa00}
.maroon{color:#600000}
.maroon-background{background:#7d0000}
.navy{color:#000060}
.navy-background{background:#00007d}
.olive{color:#606000}
.olive-background{background:#7d7d00}
.purple{color:#600060}
.purple-background{background:#7d007d}
.red{color:#bf0000}
.red-background{background:#fa0000}
.silver{color:#909090}
.silver-background{background:#bcbcbc}
.teal{color:#006060}
.teal-background{background:#007d7d}
.white{color:#bfbfbf}
.white-background{background:#fafafa}
.yellow{color:#bfbf00}
.yellow-background{background:#fafa00}
span.icon>.fa{cursor:default}
a span.icon>.fa{cursor:inherit}
.admonitionblock td.icon [class^="fa icon-"]{font-size:2.5em;text-shadow:1px 1px 2px rgba(0,0,0,.5);cursor:default}
.admonitionblock td.icon .icon-note::before{content:"\f05a";color:#19407c}
.admonitionblock td.icon .icon-tip::before{content:"\f0eb";text-shadow:1px 1px 2px rgba(155,155,0,.8);color:#111}
.admonitionblock td.icon .icon-warning::before{content:"\f071";color:#bf6900}
.admonitionblock td.icon .icon-caution::before{content:"\f06d";color:#bf3400}
.admonitionblock td.icon .icon-important::before{content:"\f06a";color:#bf0000}
.conum[data-value]{display:inline-block;color:#fff!important;background:rgba(0,0,0,.8);border-radius:50%;text-align:center;font-size:.75em;width:1.67em;height:1.67em;line-height:1.67em;font-family:"Open Sans","DejaVu Sans",sans-serif;font-style:normal;font-weight:bold}
.conum[data-value] *{color:#fff!important}
.conum[data-value]+b{display:none}
.conum[data-value]::after{content:attr(data-value)}
pre .conum[data-value]{position:relative;top:-.125em}
b.conum *{color:inherit!important}
.conum:not([data-value]):empty{display:none}
dt,th.tableblock,td.content,div.footnote{text-rendering:optimizeLegibility}
h1,h2,p,td.content,span.alt,summary{letter-spacing:-.01em}
p strong,td.content strong,div.footnote strong{letter-spacing:-.005em}
p,blockquote,dt,td.content,td.hdlist1,span.alt,summary{font-size:1.0625rem}
p{margin-bottom:1.25rem}
.sidebarblock p,.sidebarblock dt,.sidebarblock td.content,p.tableblock{font-size:1em}
.exampleblock>.content{background:#fffef7;border-color:#e0e0dc;box-shadow:0 1px 4px #e0e0dc}
.print-only{display:none!important}
@page{margin:1.25cm .75cm}
@media print{*{box-shadow:none!important;text-shadow:none!important}
html{font-size:80%}
a{color:inherit!important;text-decoration:underline!important}
a.bare,a[href^="#"],a[href^="mailto:"]{text-decoration:none!important}
a[href^="http:"]:not(.bare)::after,a[href^="https:"]:not(.bare)::after{content:"(" attr(href) ")";display:inline-block;font-size:.875em;padding-left:.25em}
abbr[title]{border-bottom:1px dotted}
abbr[title]::after{content:" (" attr(title) ")"}
pre,blockquote,tr,img,object,svg{page-break-inside:avoid}
thead{display:table-header-group}
svg{max-width:100%}
p,blockquote,dt,td.content{font-size:1em;orphans:3;widows:3}
h2,h3,#toctitle,.sidebarblock>.content>.title{page-break-after:avoid}
#header,#content,#footnotes,#footer{max-width:none}
#toc,.sidebarblock,.exampleblock>.content{background:none!important}
#toc{border-bottom:1px solid #dddddf!important;padding-bottom:0!important}
body.book #header{text-align:center}
body.book #header>h1:first-child{border:0!important;margin:2.5em 0 1em}
body.book #header .details{border:0!important;display:block;padding:0!important}
body.book #header .details span:first-child{margin-left:0!important}
body.book #header .details br{display:block}
body.book #header .details br+span::before{content:none!important}
body.book #toc{border:0!important;text-align:left!important;padding:0!important;margin:0!important}
body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-break-before:always}
.listingblock code[data-lang]::before{display:block}
#footer{padding:0 .9375em}
.hide-on-print{display:none!important}
.print-only{display:block!important}
.hide-for-print{display:none!important}
.show-for-print{display:inherit!important}}
@media amzn-kf8,print{#header>h1:first-child{margin-top:1.25rem}
.sect1{padding:0!important}
.sect1+.sect1{border:0}
#footer{background:none}
#footer-text{color:rgba(0,0,0,.6);font-size:.9em}}
@media amzn-kf8{#header,#content,#footnotes,#footer{padding:0}}
</style>
</head>
<body class="article">
<div id="header">
<h1>Long-running process protocol</h1>
</div>
<div id="content">
<div id="preamble">
<div class="sectionbody">
<div class="paragraph">
<p>This protocol is used when Git needs to communicate with an external
process throughout the entire life of a single Git command. All
communication is in pkt-line format (see <a href="../gitprotocol-common.html">gitprotocol-common(5)</a>)
over standard input and standard output.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_handshake">Handshake</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Git starts by sending a welcome message (for example,
"git-filter-client"), a list of supported protocol version numbers, and
a flush packet. Git expects to read the welcome message with "server"
instead of "client" (for example, "git-filter-server"), exactly one
protocol version number from the previously sent list, and a flush
packet. All further communication will be based on the selected version.
The remaining protocol description below documents "version=2". Please
note that "version=42" in the example below does not exist and is only
there to illustrate how the protocol would look like with more than one
version.</p>
</div>
<div class="paragraph">
<p>After the version negotiation Git sends a list of all capabilities that
it supports and a flush packet. Git expects to read a list of desired
capabilities, which must be a subset of the supported capabilities list,
and a flush packet as response:</p>
</div>
<div class="listingblock">
<div class="content">
<pre>packet: git&gt; git-filter-client
packet: git&gt; version=2
packet: git&gt; version=42
packet: git&gt; 0000
packet: git&lt; git-filter-server
packet: git&lt; version=2
packet: git&lt; 0000
packet: git&gt; capability=clean
packet: git&gt; capability=smudge
packet: git&gt; capability=not-yet-invented
packet: git&gt; 0000
packet: git&lt; capability=clean
packet: git&lt; capability=smudge
packet: git&lt; 0000</pre>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_shutdown">Shutdown</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Git will close
the command pipe on exit. The filter is expected to detect EOF
and exit gracefully on its own. Git will wait until the filter
process has stopped.</p>
</div>
</div>
</div>
</div>
<div id="footer">
<div id="footer-text">
Last updated 2025-03-17 07:09:38 UTC
</div>
</div>
</body>
</html>

View File

@@ -0,0 +1,203 @@
Multi-Pack-Index (MIDX) Design Notes
====================================
The Git object directory contains a 'pack' directory containing
packfiles (with suffix ".pack") and pack-indexes (with suffix
".idx"). The pack-indexes provide a way to lookup objects and
navigate to their offset within the pack, but these must come
in pairs with the packfiles. This pairing depends on the file
names, as the pack-index differs only in suffix with its pack-
file. While the pack-indexes provide fast lookup per packfile,
this performance degrades as the number of packfiles increases,
because abbreviations need to inspect every packfile and we are
more likely to have a miss on our most-recently-used packfile.
For some large repositories, repacking into a single packfile
is not feasible due to storage space or excessive repack times.
The multi-pack-index (MIDX for short) stores a list of objects
and their offsets into multiple packfiles. It contains:
* A list of packfile names.
* A sorted list of object IDs.
* A list of metadata for the ith object ID including:
** A value j referring to the jth packfile.
** An offset within the jth packfile for the object.
* If large offsets are required, we use another list of large
offsets similar to version 2 pack-indexes.
- An optional list of objects in pseudo-pack order (used with MIDX bitmaps).
Thus, we can provide O(log N) lookup time for any number
of packfiles.
Design Details
--------------
- The MIDX is stored in a file named 'multi-pack-index' in the
.git/objects/pack directory. This could be stored in the pack
directory of an alternate. It refers only to packfiles in that
same directory.
- The core.multiPackIndex config setting must be on (which is the
default) to consume MIDX files. Setting it to `false` prevents
Git from reading a MIDX file, even if one exists.
- The file format includes parameters for the object ID hash
function, so a future change of hash algorithm does not require
a change in format.
- The MIDX keeps only one record per object ID. If an object appears
in multiple packfiles, then the MIDX selects the copy in the
preferred packfile, otherwise selecting from the most-recently
modified packfile.
- If there exist packfiles in the pack directory not registered in
the MIDX, then those packfiles are loaded into the `packed_git`
list and `packed_git_mru` cache.
- The pack-indexes (.idx files) remain in the pack directory so we
can delete the MIDX file, set core.midx to false, or downgrade
without any loss of information.
- The MIDX file format uses a chunk-based approach (similar to the
commit-graph file) that allows optional data to be added.
Incremental multi-pack indexes
------------------------------
As repositories grow in size, it becomes more expensive to write a
multi-pack index (MIDX) that includes all packfiles. To accommodate
this, the "incremental multi-pack indexes" feature allows for combining
a "chain" of multi-pack indexes.
Each individual component of the chain need only contain a small number
of packfiles. Appending to the chain does not invalidate earlier parts
of the chain, so repositories can control how much time is spent
updating the MIDX chain by determining the number of packs in each layer
of the MIDX chain.
=== Design state
At present, the incremental multi-pack indexes feature is missing two
important components:
- The ability to rewrite earlier portions of the MIDX chain (i.e., to
"compact" some collection of adjacent MIDX layers into a single
MIDX). At present the only supported way of shrinking a MIDX chain
is to rewrite the entire chain from scratch without the `--split`
flag.
+
There are no fundamental limitations that stand in the way of being able
to implement this feature. It is omitted from the initial implementation
in order to reduce the complexity, but will be added later.
- Support for reachability bitmaps. The classic single MIDX
implementation does support reachability bitmaps (see the section
titled "multi-pack-index reverse indexes" in
linkgit:gitformat-pack[5] for more details).
+
As above, there are no fundamental limitations that stand in the way of
extending the incremental MIDX format to support reachability bitmaps.
The design below specifically takes this into account, and support for
reachability bitmaps will be added in a future patch series. It is
omitted from the current implementation for the same reason as above.
+
In brief, to support reachability bitmaps with the incremental MIDX
feature, the concept of the pseudo-pack order is extended across each
layer of the incremental MIDX chain to form a concatenated pseudo-pack
order. This concatenation takes place in the same order as the chain
itself (in other words, the concatenated pseudo-pack order for a chain
`{$H1, $H2, $H3}` would be the pseudo-pack order for `$H1`, followed by
the pseudo-pack order for `$H2`, followed by the pseudo-pack order for
`$H3`).
+
The layout will then be extended so that each layer of the incremental
MIDX chain can write a `*.bitmap`. The objects in each layer's bitmap
are offset by the number of objects in the previous layers of the chain.
=== File layout
Instead of storing a single `multi-pack-index` file (with an optional
`.rev` and `.bitmap` extension) in `$GIT_DIR/objects/pack`, incremental
MIDXs are stored in the following layout:
----
$GIT_DIR/objects/pack/multi-pack-index.d/
$GIT_DIR/objects/pack/multi-pack-index.d/multi-pack-index-chain
$GIT_DIR/objects/pack/multi-pack-index.d/multi-pack-index-$H1.midx
$GIT_DIR/objects/pack/multi-pack-index.d/multi-pack-index-$H2.midx
$GIT_DIR/objects/pack/multi-pack-index.d/multi-pack-index-$H3.midx
----
The `multi-pack-index-chain` file contains a list of the incremental
MIDX files in the chain, in order. The above example shows a chain whose
`multi-pack-index-chain` file would contain the following lines:
----
$H1
$H2
$H3
----
The `multi-pack-index-$H1.midx` file contains the first layer of the
multi-pack-index chain. The `multi-pack-index-$H2.midx` file contains
the second layer of the chain, and so on.
When both an incremental- and non-incremental MIDX are present, the
non-incremental MIDX is always read first.
=== Object positions for incremental MIDXs
In the original multi-pack-index design, we refer to objects via their
lexicographic position (by object IDs) within the repository's singular
multi-pack-index. In the incremental multi-pack-index design, we refer
to objects via their index into a concatenated lexicographic ordering
among each component in the MIDX chain.
If `objects_nr()` is a function that returns the number of objects in a
given MIDX layer, then the index of an object at lexicographic position
`i` within, say, $H3 is defined as:
----
objects_nr($H2) + objects_nr($H1) + i
----
(in the C implementation, this is often computed as `i +
m->num_objects_in_base`).
Future Work
-----------
- The multi-pack-index allows many packfiles, especially in a context
where repacking is expensive (such as a very large repo), or
unexpected maintenance time is unacceptable (such as a high-demand
build machine). However, the multi-pack-index needs to be rewritten
in full every time. We can extend the format to be incremental, so
writes are fast. By storing a small "tip" multi-pack-index that
points to large "base" MIDX files, we can keep writes fast while
still reducing the number of binary searches required for object
lookups.
- If the multi-pack-index is extended to store a "stable object order"
(a function Order(hash) = integer that is constant for a given hash,
even as the multi-pack-index is updated) then MIDX bitmaps could be
updated independently of the MIDX.
- Packfiles can be marked as "special" using empty files that share
the initial name but replace ".pack" with ".keep" or ".promisor".
We can add an optional chunk of data to the multi-pack-index that
records flags of information about the packfiles. This allows new
states, such as 'repacked' or 'redeltified', that can help with
pack maintenance in a multi-pack environment. It may also be
helpful to organize packfiles by object type (commit, tree, blob,
etc.) and use this metadata to help that maintenance.
Related Links
-------------
[0] https://bugs.chromium.org/p/git/issues/detail?id=6
Chromium work item for: Multi-Pack Index (MIDX)
[1] https://lore.kernel.org/git/20180107181459.222909-1-dstolee@microsoft.com/
An earlier RFC for the multi-pack-index feature
[2] https://lore.kernel.org/git/alpine.DEB.2.20.1803091557510.23109@alexmv-linux/
Git Merge 2018 Contributor's summit notes (includes discussion of MIDX)

View File

@@ -0,0 +1,738 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="en">
<head>
<meta charset="UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=edge"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<meta name="generator" content="Asciidoctor 2.0.23"/>
<title>Multi-Pack-Index (MIDX) Design Notes</title>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"/>
<style>
/*! Asciidoctor default stylesheet | MIT License | https://asciidoctor.org */
/* Uncomment the following line when using as a custom stylesheet */
/* @import "https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"; */
html{font-family:sans-serif;-webkit-text-size-adjust:100%}
a{background:none}
a:focus{outline:thin dotted}
a:active,a:hover{outline:0}
h1{font-size:2em;margin:.67em 0}
b,strong{font-weight:bold}
abbr{font-size:.9em}
abbr[title]{cursor:help;border-bottom:1px dotted #dddddf;text-decoration:none}
dfn{font-style:italic}
hr{height:0}
mark{background:#ff0;color:#000}
code,kbd,pre,samp{font-family:monospace;font-size:1em}
pre{white-space:pre-wrap}
q{quotes:"\201C" "\201D" "\2018" "\2019"}
small{font-size:80%}
sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline}
sup{top:-.5em}
sub{bottom:-.25em}
img{border:0}
svg:not(:root){overflow:hidden}
figure{margin:0}
audio,video{display:inline-block}
audio:not([controls]){display:none;height:0}
fieldset{border:1px solid silver;margin:0 2px;padding:.35em .625em .75em}
legend{border:0;padding:0}
button,input,select,textarea{font-family:inherit;font-size:100%;margin:0}
button,input{line-height:normal}
button,select{text-transform:none}
button,html input[type=button],input[type=reset],input[type=submit]{-webkit-appearance:button;cursor:pointer}
button[disabled],html input[disabled]{cursor:default}
input[type=checkbox],input[type=radio]{padding:0}
button::-moz-focus-inner,input::-moz-focus-inner{border:0;padding:0}
textarea{overflow:auto;vertical-align:top}
table{border-collapse:collapse;border-spacing:0}
*,::before,::after{box-sizing:border-box}
html,body{font-size:100%}
body{background:#fff;color:rgba(0,0,0,.8);padding:0;margin:0;font-family:"Noto Serif","DejaVu Serif",serif;line-height:1;position:relative;cursor:auto;-moz-tab-size:4;-o-tab-size:4;tab-size:4;word-wrap:anywhere;-moz-osx-font-smoothing:grayscale;-webkit-font-smoothing:antialiased}
a:hover{cursor:pointer}
img,object,embed{max-width:100%;height:auto}
object,embed{height:100%}
img{-ms-interpolation-mode:bicubic}
.left{float:left!important}
.right{float:right!important}
.text-left{text-align:left!important}
.text-right{text-align:right!important}
.text-center{text-align:center!important}
.text-justify{text-align:justify!important}
.hide{display:none}
img,object,svg{display:inline-block;vertical-align:middle}
textarea{height:auto;min-height:50px}
select{width:100%}
.subheader,.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{line-height:1.45;color:#7a2518;font-weight:400;margin-top:0;margin-bottom:.25em}
div,dl,dt,dd,ul,ol,li,h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6,pre,form,p,blockquote,th,td{margin:0;padding:0}
a{color:#2156a5;text-decoration:underline;line-height:inherit}
a:hover,a:focus{color:#1d4b8f}
a img{border:0}
p{line-height:1.6;margin-bottom:1.25em;text-rendering:optimizeLegibility}
p aside{font-size:.875em;line-height:1.35;font-style:italic}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{font-family:"Open Sans","DejaVu Sans",sans-serif;font-weight:300;font-style:normal;color:#ba3925;text-rendering:optimizeLegibility;margin-top:1em;margin-bottom:.5em;line-height:1.0125em}
h1 small,h2 small,h3 small,#toctitle small,.sidebarblock>.content>.title small,h4 small,h5 small,h6 small{font-size:60%;color:#e99b8f;line-height:0}
h1{font-size:2.125em}
h2{font-size:1.6875em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.375em}
h4,h5{font-size:1.125em}
h6{font-size:1em}
hr{border:solid #dddddf;border-width:1px 0 0;clear:both;margin:1.25em 0 1.1875em}
em,i{font-style:italic;line-height:inherit}
strong,b{font-weight:bold;line-height:inherit}
small{font-size:60%;line-height:inherit}
code{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:400;color:rgba(0,0,0,.9)}
ul,ol,dl{line-height:1.6;margin-bottom:1.25em;list-style-position:outside;font-family:inherit}
ul,ol{margin-left:1.5em}
ul li ul,ul li ol{margin-left:1.25em;margin-bottom:0}
ul.circle{list-style-type:circle}
ul.disc{list-style-type:disc}
ul.square{list-style-type:square}
ul.circle ul:not([class]),ul.disc ul:not([class]),ul.square ul:not([class]){list-style:inherit}
ol li ul,ol li ol{margin-left:1.25em;margin-bottom:0}
dl dt{margin-bottom:.3125em;font-weight:bold}
dl dd{margin-bottom:1.25em}
blockquote{margin:0 0 1.25em;padding:.5625em 1.25em 0 1.1875em;border-left:1px solid #ddd}
blockquote,blockquote p{line-height:1.6;color:rgba(0,0,0,.85)}
@media screen and (min-width:768px){h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2}
h1{font-size:2.75em}
h2{font-size:2.3125em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.6875em}
h4{font-size:1.4375em}}
table{background:#fff;margin-bottom:1.25em;border:1px solid #dedede;word-wrap:normal}
table thead,table tfoot{background:#f7f8f7}
table thead tr th,table thead tr td,table tfoot tr th,table tfoot tr td{padding:.5em .625em .625em;font-size:inherit;color:rgba(0,0,0,.8);text-align:left}
table tr th,table tr td{padding:.5625em .625em;font-size:inherit;color:rgba(0,0,0,.8)}
table tr.even,table tr.alt{background:#f8f8f7}
table thead tr th,table tfoot tr th,table tbody tr td,table tr td,table tfoot tr td{line-height:1.6}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2;word-spacing:-.05em}
h1 strong,h2 strong,h3 strong,#toctitle strong,.sidebarblock>.content>.title strong,h4 strong,h5 strong,h6 strong{font-weight:400}
.center{margin-left:auto;margin-right:auto}
.stretch{width:100%}
.clearfix::before,.clearfix::after,.float-group::before,.float-group::after{content:" ";display:table}
.clearfix::after,.float-group::after{clear:both}
:not(pre).nobreak{word-wrap:normal}
:not(pre).nowrap{white-space:nowrap}
:not(pre).pre-wrap{white-space:pre-wrap}
:not(pre):not([class^=L])>code{font-size:.9375em;font-style:normal!important;letter-spacing:0;padding:.1em .5ex;word-spacing:-.15em;background:#f7f7f8;border-radius:4px;line-height:1.45;text-rendering:optimizeSpeed}
pre{color:rgba(0,0,0,.9);font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;line-height:1.45;text-rendering:optimizeSpeed}
pre code,pre pre{color:inherit;font-size:inherit;line-height:inherit}
pre>code{display:block}
pre.nowrap,pre.nowrap pre{white-space:pre;word-wrap:normal}
em em{font-style:normal}
strong strong{font-weight:400}
.keyseq{color:rgba(51,51,51,.8)}
kbd{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;display:inline-block;color:rgba(0,0,0,.8);font-size:.65em;line-height:1.45;background:#f7f7f7;border:1px solid #ccc;border-radius:3px;box-shadow:0 1px 0 rgba(0,0,0,.2),inset 0 0 0 .1em #fff;margin:0 .15em;padding:.2em .5em;vertical-align:middle;position:relative;top:-.1em;white-space:nowrap}
.keyseq kbd:first-child{margin-left:0}
.keyseq kbd:last-child{margin-right:0}
.menuseq,.menuref{color:#000}
.menuseq b:not(.caret),.menuref{font-weight:inherit}
.menuseq{word-spacing:-.02em}
.menuseq b.caret{font-size:1.25em;line-height:.8}
.menuseq i.caret{font-weight:bold;text-align:center;width:.45em}
b.button::before,b.button::after{position:relative;top:-1px;font-weight:400}
b.button::before{content:"[";padding:0 3px 0 2px}
b.button::after{content:"]";padding:0 2px 0 3px}
p a>code:hover{color:rgba(0,0,0,.9)}
#header,#content,#footnotes,#footer{width:100%;margin:0 auto;max-width:62.5em;*zoom:1;position:relative;padding-left:.9375em;padding-right:.9375em}
#header::before,#header::after,#content::before,#content::after,#footnotes::before,#footnotes::after,#footer::before,#footer::after{content:" ";display:table}
#header::after,#content::after,#footnotes::after,#footer::after{clear:both}
#content{margin-top:1.25em}
#content::before{content:none}
#header>h1:first-child{color:rgba(0,0,0,.85);margin-top:2.25rem;margin-bottom:0}
#header>h1:first-child+#toc{margin-top:8px;border-top:1px solid #dddddf}
#header>h1:only-child{border-bottom:1px solid #dddddf;padding-bottom:8px}
#header .details{border-bottom:1px solid #dddddf;line-height:1.45;padding-top:.25em;padding-bottom:.25em;padding-left:.25em;color:rgba(0,0,0,.6);display:flex;flex-flow:row wrap}
#header .details span:first-child{margin-left:-.125em}
#header .details span.email a{color:rgba(0,0,0,.85)}
#header .details br{display:none}
#header .details br+span::before{content:"\00a0\2013\00a0"}
#header .details br+span.author::before{content:"\00a0\22c5\00a0";color:rgba(0,0,0,.85)}
#header .details br+span#revremark::before{content:"\00a0|\00a0"}
#header #revnumber{text-transform:capitalize}
#header #revnumber::after{content:"\00a0"}
#content>h1:first-child:not([class]){color:rgba(0,0,0,.85);border-bottom:1px solid #dddddf;padding-bottom:8px;margin-top:0;padding-top:1rem;margin-bottom:1.25rem}
#toc{border-bottom:1px solid #e7e7e9;padding-bottom:.5em}
#toc>ul{margin-left:.125em}
#toc ul.sectlevel0>li>a{font-style:italic}
#toc ul.sectlevel0 ul.sectlevel1{margin:.5em 0}
#toc ul{font-family:"Open Sans","DejaVu Sans",sans-serif;list-style-type:none}
#toc li{line-height:1.3334;margin-top:.3334em}
#toc a{text-decoration:none}
#toc a:active{text-decoration:underline}
#toctitle{color:#7a2518;font-size:1.2em}
@media screen and (min-width:768px){#toctitle{font-size:1.375em}
body.toc2{padding-left:15em;padding-right:0}
body.toc2 #header>h1:nth-last-child(2){border-bottom:1px solid #dddddf;padding-bottom:8px}
#toc.toc2{margin-top:0!important;background:#f8f8f7;position:fixed;width:15em;left:0;top:0;border-right:1px solid #e7e7e9;border-top-width:0!important;border-bottom-width:0!important;z-index:1000;padding:1.25em 1em;height:100%;overflow:auto}
#toc.toc2 #toctitle{margin-top:0;margin-bottom:.8rem;font-size:1.2em}
#toc.toc2>ul{font-size:.9em;margin-bottom:0}
#toc.toc2 ul ul{margin-left:0;padding-left:1em}
#toc.toc2 ul.sectlevel0 ul.sectlevel1{padding-left:0;margin-top:.5em;margin-bottom:.5em}
body.toc2.toc-right{padding-left:0;padding-right:15em}
body.toc2.toc-right #toc.toc2{border-right-width:0;border-left:1px solid #e7e7e9;left:auto;right:0}}
@media screen and (min-width:1280px){body.toc2{padding-left:20em;padding-right:0}
#toc.toc2{width:20em}
#toc.toc2 #toctitle{font-size:1.375em}
#toc.toc2>ul{font-size:.95em}
#toc.toc2 ul ul{padding-left:1.25em}
body.toc2.toc-right{padding-left:0;padding-right:20em}}
#content #toc{border:1px solid #e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;border-radius:4px}
#content #toc>:first-child{margin-top:0}
#content #toc>:last-child{margin-bottom:0}
#footer{max-width:none;background:rgba(0,0,0,.8);padding:1.25em}
#footer-text{color:hsla(0,0%,100%,.8);line-height:1.44}
#content{margin-bottom:.625em}
.sect1{padding-bottom:.625em}
@media screen and (min-width:768px){#content{margin-bottom:1.25em}
.sect1{padding-bottom:1.25em}}
.sect1:last-child{padding-bottom:0}
.sect1+.sect1{border-top:1px solid #e7e7e9}
#content h1>a.anchor,h2>a.anchor,h3>a.anchor,#toctitle>a.anchor,.sidebarblock>.content>.title>a.anchor,h4>a.anchor,h5>a.anchor,h6>a.anchor{position:absolute;z-index:1001;width:1.5ex;margin-left:-1.5ex;display:block;text-decoration:none!important;visibility:hidden;text-align:center;font-weight:400}
#content h1>a.anchor::before,h2>a.anchor::before,h3>a.anchor::before,#toctitle>a.anchor::before,.sidebarblock>.content>.title>a.anchor::before,h4>a.anchor::before,h5>a.anchor::before,h6>a.anchor::before{content:"\00A7";font-size:.85em;display:block;padding-top:.1em}
#content h1:hover>a.anchor,#content h1>a.anchor:hover,h2:hover>a.anchor,h2>a.anchor:hover,h3:hover>a.anchor,#toctitle:hover>a.anchor,.sidebarblock>.content>.title:hover>a.anchor,h3>a.anchor:hover,#toctitle>a.anchor:hover,.sidebarblock>.content>.title>a.anchor:hover,h4:hover>a.anchor,h4>a.anchor:hover,h5:hover>a.anchor,h5>a.anchor:hover,h6:hover>a.anchor,h6>a.anchor:hover{visibility:visible}
#content h1>a.link,h2>a.link,h3>a.link,#toctitle>a.link,.sidebarblock>.content>.title>a.link,h4>a.link,h5>a.link,h6>a.link{color:#ba3925;text-decoration:none}
#content h1>a.link:hover,h2>a.link:hover,h3>a.link:hover,#toctitle>a.link:hover,.sidebarblock>.content>.title>a.link:hover,h4>a.link:hover,h5>a.link:hover,h6>a.link:hover{color:#a53221}
details,.audioblock,.imageblock,.literalblock,.listingblock,.stemblock,.videoblock{margin-bottom:1.25em}
details{margin-left:1.25rem}
details>summary{cursor:pointer;display:block;position:relative;line-height:1.6;margin-bottom:.625rem;outline:none;-webkit-tap-highlight-color:transparent}
details>summary::-webkit-details-marker{display:none}
details>summary::before{content:"";border:solid transparent;border-left:solid;border-width:.3em 0 .3em .5em;position:absolute;top:.5em;left:-1.25rem;transform:translateX(15%)}
details[open]>summary::before{border:solid transparent;border-top:solid;border-width:.5em .3em 0;transform:translateY(15%)}
details>summary::after{content:"";width:1.25rem;height:1em;position:absolute;top:.3em;left:-1.25rem}
.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{text-rendering:optimizeLegibility;text-align:left;font-family:"Noto Serif","DejaVu Serif",serif;font-size:1rem;font-style:italic}
table.tableblock.fit-content>caption.title{white-space:nowrap;width:0}
.paragraph.lead>p,#preamble>.sectionbody>[class=paragraph]:first-of-type p{font-size:1.21875em;line-height:1.6;color:rgba(0,0,0,.85)}
.admonitionblock>table{border-collapse:separate;border:0;background:none;width:100%}
.admonitionblock>table td.icon{text-align:center;width:80px}
.admonitionblock>table td.icon img{max-width:none}
.admonitionblock>table td.icon .title{font-weight:bold;font-family:"Open Sans","DejaVu Sans",sans-serif;text-transform:uppercase}
.admonitionblock>table td.content{padding-left:1.125em;padding-right:1.25em;border-left:1px solid #dddddf;color:rgba(0,0,0,.6);word-wrap:anywhere}
.admonitionblock>table td.content>:last-child>:last-child{margin-bottom:0}
.exampleblock>.content{border:1px solid #e6e6e6;margin-bottom:1.25em;padding:1.25em;background:#fff;border-radius:4px}
.sidebarblock{border:1px solid #dbdbd6;margin-bottom:1.25em;padding:1.25em;background:#f3f3f2;border-radius:4px}
.sidebarblock>.content>.title{color:#7a2518;margin-top:0;text-align:center}
.exampleblock>.content>:first-child,.sidebarblock>.content>:first-child{margin-top:0}
.exampleblock>.content>:last-child,.exampleblock>.content>:last-child>:last-child,.exampleblock>.content .olist>ol>li:last-child>:last-child,.exampleblock>.content .ulist>ul>li:last-child>:last-child,.exampleblock>.content .qlist>ol>li:last-child>:last-child,.sidebarblock>.content>:last-child,.sidebarblock>.content>:last-child>:last-child,.sidebarblock>.content .olist>ol>li:last-child>:last-child,.sidebarblock>.content .ulist>ul>li:last-child>:last-child,.sidebarblock>.content .qlist>ol>li:last-child>:last-child{margin-bottom:0}
.literalblock pre,.listingblock>.content>pre{border-radius:4px;overflow-x:auto;padding:1em;font-size:.8125em}
@media screen and (min-width:768px){.literalblock pre,.listingblock>.content>pre{font-size:.90625em}}
@media screen and (min-width:1280px){.literalblock pre,.listingblock>.content>pre{font-size:1em}}
.literalblock pre,.listingblock>.content>pre:not(.highlight),.listingblock>.content>pre[class=highlight],.listingblock>.content>pre[class^="highlight "]{background:#f7f7f8}
.literalblock.output pre{color:#f7f7f8;background:rgba(0,0,0,.9)}
.listingblock>.content{position:relative}
.listingblock code[data-lang]::before{display:none;content:attr(data-lang);position:absolute;font-size:.75em;top:.425rem;right:.5rem;line-height:1;text-transform:uppercase;color:inherit;opacity:.5}
.listingblock:hover code[data-lang]::before{display:block}
.listingblock.terminal pre .command::before{content:attr(data-prompt);padding-right:.5em;color:inherit;opacity:.5}
.listingblock.terminal pre .command:not([data-prompt])::before{content:"$"}
.listingblock pre.highlightjs{padding:0}
.listingblock pre.highlightjs>code{padding:1em;border-radius:4px}
.listingblock pre.prettyprint{border-width:0}
.prettyprint{background:#f7f7f8}
pre.prettyprint .linenums{line-height:1.45;margin-left:2em}
pre.prettyprint li{background:none;list-style-type:inherit;padding-left:0}
pre.prettyprint li code[data-lang]::before{opacity:1}
pre.prettyprint li:not(:first-child) code[data-lang]::before{display:none}
table.linenotable{border-collapse:separate;border:0;margin-bottom:0;background:none}
table.linenotable td[class]{color:inherit;vertical-align:top;padding:0;line-height:inherit;white-space:normal}
table.linenotable td.code{padding-left:.75em}
table.linenotable td.linenos,pre.pygments .linenos{border-right:1px solid;opacity:.35;padding-right:.5em;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}
pre.pygments span.linenos{display:inline-block;margin-right:.75em}
.quoteblock{margin:0 1em 1.25em 1.5em;display:table}
.quoteblock:not(.excerpt)>.title{margin-left:-1.5em;margin-bottom:.75em}
.quoteblock blockquote,.quoteblock p{color:rgba(0,0,0,.85);font-size:1.15rem;line-height:1.75;word-spacing:.1em;letter-spacing:0;font-style:italic;text-align:justify}
.quoteblock blockquote{margin:0;padding:0;border:0}
.quoteblock blockquote::before{content:"\201c";float:left;font-size:2.75em;font-weight:bold;line-height:.6em;margin-left:-.6em;color:#7a2518;text-shadow:0 1px 2px rgba(0,0,0,.1)}
.quoteblock blockquote>.paragraph:last-child p{margin-bottom:0}
.quoteblock .attribution{margin-top:.75em;margin-right:.5ex;text-align:right}
.verseblock{margin:0 1em 1.25em}
.verseblock pre{font-family:"Open Sans","DejaVu Sans",sans-serif;font-size:1.15rem;color:rgba(0,0,0,.85);font-weight:300;text-rendering:optimizeLegibility}
.verseblock pre strong{font-weight:400}
.verseblock .attribution{margin-top:1.25rem;margin-left:.5ex}
.quoteblock .attribution,.verseblock .attribution{font-size:.9375em;line-height:1.45;font-style:italic}
.quoteblock .attribution br,.verseblock .attribution br{display:none}
.quoteblock .attribution cite,.verseblock .attribution cite{display:block;letter-spacing:-.025em;color:rgba(0,0,0,.6)}
.quoteblock.abstract blockquote::before,.quoteblock.excerpt blockquote::before,.quoteblock .quoteblock blockquote::before{display:none}
.quoteblock.abstract blockquote,.quoteblock.abstract p,.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{line-height:1.6;word-spacing:0}
.quoteblock.abstract{margin:0 1em 1.25em;display:block}
.quoteblock.abstract>.title{margin:0 0 .375em;font-size:1.15em;text-align:center}
.quoteblock.excerpt>blockquote,.quoteblock .quoteblock{padding:0 0 .25em 1em;border-left:.25em solid #dddddf}
.quoteblock.excerpt,.quoteblock .quoteblock{margin-left:0}
.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{color:inherit;font-size:1.0625rem}
.quoteblock.excerpt .attribution,.quoteblock .quoteblock .attribution{color:inherit;font-size:.85rem;text-align:left;margin-right:0}
p.tableblock:last-child{margin-bottom:0}
td.tableblock>.content{margin-bottom:1.25em;word-wrap:anywhere}
td.tableblock>.content>:last-child{margin-bottom:-1.25em}
table.tableblock,th.tableblock,td.tableblock{border:0 solid #dedede}
table.grid-all>*>tr>*{border-width:1px}
table.grid-cols>*>tr>*{border-width:0 1px}
table.grid-rows>*>tr>*{border-width:1px 0}
table.frame-all{border-width:1px}
table.frame-ends{border-width:1px 0}
table.frame-sides{border-width:0 1px}
table.frame-none>colgroup+*>:first-child>*,table.frame-sides>colgroup+*>:first-child>*{border-top-width:0}
table.frame-none>:last-child>:last-child>*,table.frame-sides>:last-child>:last-child>*{border-bottom-width:0}
table.frame-none>*>tr>:first-child,table.frame-ends>*>tr>:first-child{border-left-width:0}
table.frame-none>*>tr>:last-child,table.frame-ends>*>tr>:last-child{border-right-width:0}
table.stripes-all>*>tr,table.stripes-odd>*>tr:nth-of-type(odd),table.stripes-even>*>tr:nth-of-type(even),table.stripes-hover>*>tr:hover{background:#f8f8f7}
th.halign-left,td.halign-left{text-align:left}
th.halign-right,td.halign-right{text-align:right}
th.halign-center,td.halign-center{text-align:center}
th.valign-top,td.valign-top{vertical-align:top}
th.valign-bottom,td.valign-bottom{vertical-align:bottom}
th.valign-middle,td.valign-middle{vertical-align:middle}
table thead th,table tfoot th{font-weight:bold}
tbody tr th{background:#f7f8f7}
tbody tr th,tbody tr th p,tfoot tr th,tfoot tr th p{color:rgba(0,0,0,.8);font-weight:bold}
p.tableblock>code:only-child{background:none;padding:0}
p.tableblock{font-size:1em}
ol{margin-left:1.75em}
ul li ol{margin-left:1.5em}
dl dd{margin-left:1.125em}
dl dd:last-child,dl dd:last-child>:last-child{margin-bottom:0}
li p,ul dd,ol dd,.olist .olist,.ulist .ulist,.ulist .olist,.olist .ulist{margin-bottom:.625em}
ul.checklist,ul.none,ol.none,ul.no-bullet,ol.no-bullet,ol.unnumbered,ul.unstyled,ol.unstyled{list-style-type:none}
ul.no-bullet,ol.no-bullet,ol.unnumbered{margin-left:.625em}
ul.unstyled,ol.unstyled{margin-left:0}
li>p:empty:only-child::before{content:"";display:inline-block}
ul.checklist>li>p:first-child{margin-left:-1em}
ul.checklist>li>p:first-child>.fa-square-o:first-child,ul.checklist>li>p:first-child>.fa-check-square-o:first-child{width:1.25em;font-size:.8em;position:relative;bottom:.125em}
ul.checklist>li>p:first-child>input[type=checkbox]:first-child{margin-right:.25em}
ul.inline{display:flex;flex-flow:row wrap;list-style:none;margin:0 0 .625em -1.25em}
ul.inline>li{margin-left:1.25em}
.unstyled dl dt{font-weight:400;font-style:normal}
ol.arabic{list-style-type:decimal}
ol.decimal{list-style-type:decimal-leading-zero}
ol.loweralpha{list-style-type:lower-alpha}
ol.upperalpha{list-style-type:upper-alpha}
ol.lowerroman{list-style-type:lower-roman}
ol.upperroman{list-style-type:upper-roman}
ol.lowergreek{list-style-type:lower-greek}
.hdlist>table,.colist>table{border:0;background:none}
.hdlist>table>tbody>tr,.colist>table>tbody>tr{background:none}
td.hdlist1,td.hdlist2{vertical-align:top;padding:0 .625em}
td.hdlist1{font-weight:bold;padding-bottom:1.25em}
td.hdlist2{word-wrap:anywhere}
.literalblock+.colist,.listingblock+.colist{margin-top:-.5em}
.colist td:not([class]):first-child{padding:.4em .75em 0;line-height:1;vertical-align:top}
.colist td:not([class]):first-child img{max-width:none}
.colist td:not([class]):last-child{padding:.25em 0}
.thumb,.th{line-height:0;display:inline-block;border:4px solid #fff;box-shadow:0 0 0 1px #ddd}
.imageblock.left{margin:.25em .625em 1.25em 0}
.imageblock.right{margin:.25em 0 1.25em .625em}
.imageblock>.title{margin-bottom:0}
.imageblock.thumb,.imageblock.th{border-width:6px}
.imageblock.thumb>.title,.imageblock.th>.title{padding:0 .125em}
.image.left,.image.right{margin-top:.25em;margin-bottom:.25em;display:inline-block;line-height:0}
.image.left{margin-right:.625em}
.image.right{margin-left:.625em}
a.image{text-decoration:none;display:inline-block}
a.image object{pointer-events:none}
sup.footnote,sup.footnoteref{font-size:.875em;position:static;vertical-align:super}
sup.footnote a,sup.footnoteref a{text-decoration:none}
sup.footnote a:active,sup.footnoteref a:active,#footnotes .footnote a:first-of-type:active{text-decoration:underline}
#footnotes{padding-top:.75em;padding-bottom:.75em;margin-bottom:.625em}
#footnotes hr{width:20%;min-width:6.25em;margin:-.25em 0 .75em;border-width:1px 0 0}
#footnotes .footnote{padding:0 .375em 0 .225em;line-height:1.3334;font-size:.875em;margin-left:1.2em;margin-bottom:.2em}
#footnotes .footnote a:first-of-type{font-weight:bold;text-decoration:none;margin-left:-1.05em}
#footnotes .footnote:last-of-type{margin-bottom:0}
#content #footnotes{margin-top:-.625em;margin-bottom:0;padding:.75em 0}
div.unbreakable{page-break-inside:avoid}
.big{font-size:larger}
.small{font-size:smaller}
.underline{text-decoration:underline}
.overline{text-decoration:overline}
.line-through{text-decoration:line-through}
.aqua{color:#00bfbf}
.aqua-background{background:#00fafa}
.black{color:#000}
.black-background{background:#000}
.blue{color:#0000bf}
.blue-background{background:#0000fa}
.fuchsia{color:#bf00bf}
.fuchsia-background{background:#fa00fa}
.gray{color:#606060}
.gray-background{background:#7d7d7d}
.green{color:#006000}
.green-background{background:#007d00}
.lime{color:#00bf00}
.lime-background{background:#00fa00}
.maroon{color:#600000}
.maroon-background{background:#7d0000}
.navy{color:#000060}
.navy-background{background:#00007d}
.olive{color:#606000}
.olive-background{background:#7d7d00}
.purple{color:#600060}
.purple-background{background:#7d007d}
.red{color:#bf0000}
.red-background{background:#fa0000}
.silver{color:#909090}
.silver-background{background:#bcbcbc}
.teal{color:#006060}
.teal-background{background:#007d7d}
.white{color:#bfbfbf}
.white-background{background:#fafafa}
.yellow{color:#bfbf00}
.yellow-background{background:#fafa00}
span.icon>.fa{cursor:default}
a span.icon>.fa{cursor:inherit}
.admonitionblock td.icon [class^="fa icon-"]{font-size:2.5em;text-shadow:1px 1px 2px rgba(0,0,0,.5);cursor:default}
.admonitionblock td.icon .icon-note::before{content:"\f05a";color:#19407c}
.admonitionblock td.icon .icon-tip::before{content:"\f0eb";text-shadow:1px 1px 2px rgba(155,155,0,.8);color:#111}
.admonitionblock td.icon .icon-warning::before{content:"\f071";color:#bf6900}
.admonitionblock td.icon .icon-caution::before{content:"\f06d";color:#bf3400}
.admonitionblock td.icon .icon-important::before{content:"\f06a";color:#bf0000}
.conum[data-value]{display:inline-block;color:#fff!important;background:rgba(0,0,0,.8);border-radius:50%;text-align:center;font-size:.75em;width:1.67em;height:1.67em;line-height:1.67em;font-family:"Open Sans","DejaVu Sans",sans-serif;font-style:normal;font-weight:bold}
.conum[data-value] *{color:#fff!important}
.conum[data-value]+b{display:none}
.conum[data-value]::after{content:attr(data-value)}
pre .conum[data-value]{position:relative;top:-.125em}
b.conum *{color:inherit!important}
.conum:not([data-value]):empty{display:none}
dt,th.tableblock,td.content,div.footnote{text-rendering:optimizeLegibility}
h1,h2,p,td.content,span.alt,summary{letter-spacing:-.01em}
p strong,td.content strong,div.footnote strong{letter-spacing:-.005em}
p,blockquote,dt,td.content,td.hdlist1,span.alt,summary{font-size:1.0625rem}
p{margin-bottom:1.25rem}
.sidebarblock p,.sidebarblock dt,.sidebarblock td.content,p.tableblock{font-size:1em}
.exampleblock>.content{background:#fffef7;border-color:#e0e0dc;box-shadow:0 1px 4px #e0e0dc}
.print-only{display:none!important}
@page{margin:1.25cm .75cm}
@media print{*{box-shadow:none!important;text-shadow:none!important}
html{font-size:80%}
a{color:inherit!important;text-decoration:underline!important}
a.bare,a[href^="#"],a[href^="mailto:"]{text-decoration:none!important}
a[href^="http:"]:not(.bare)::after,a[href^="https:"]:not(.bare)::after{content:"(" attr(href) ")";display:inline-block;font-size:.875em;padding-left:.25em}
abbr[title]{border-bottom:1px dotted}
abbr[title]::after{content:" (" attr(title) ")"}
pre,blockquote,tr,img,object,svg{page-break-inside:avoid}
thead{display:table-header-group}
svg{max-width:100%}
p,blockquote,dt,td.content{font-size:1em;orphans:3;widows:3}
h2,h3,#toctitle,.sidebarblock>.content>.title{page-break-after:avoid}
#header,#content,#footnotes,#footer{max-width:none}
#toc,.sidebarblock,.exampleblock>.content{background:none!important}
#toc{border-bottom:1px solid #dddddf!important;padding-bottom:0!important}
body.book #header{text-align:center}
body.book #header>h1:first-child{border:0!important;margin:2.5em 0 1em}
body.book #header .details{border:0!important;display:block;padding:0!important}
body.book #header .details span:first-child{margin-left:0!important}
body.book #header .details br{display:block}
body.book #header .details br+span::before{content:none!important}
body.book #toc{border:0!important;text-align:left!important;padding:0!important;margin:0!important}
body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-break-before:always}
.listingblock code[data-lang]::before{display:block}
#footer{padding:0 .9375em}
.hide-on-print{display:none!important}
.print-only{display:block!important}
.hide-for-print{display:none!important}
.show-for-print{display:inherit!important}}
@media amzn-kf8,print{#header>h1:first-child{margin-top:1.25rem}
.sect1{padding:0!important}
.sect1+.sect1{border:0}
#footer{background:none}
#footer-text{color:rgba(0,0,0,.6);font-size:.9em}}
@media amzn-kf8{#header,#content,#footnotes,#footer{padding:0}}
</style>
</head>
<body class="article">
<div id="header">
<h1>Multi-Pack-Index (MIDX) Design Notes</h1>
</div>
<div id="content">
<div id="preamble">
<div class="sectionbody">
<div class="paragraph">
<p>The Git object directory contains a <em>pack</em> directory containing
packfiles (with suffix ".pack") and pack-indexes (with suffix
".idx"). The pack-indexes provide a way to lookup objects and
navigate to their offset within the pack, but these must come
in pairs with the packfiles. This pairing depends on the file
names, as the pack-index differs only in suffix with its pack-
file. While the pack-indexes provide fast lookup per packfile,
this performance degrades as the number of packfiles increases,
because abbreviations need to inspect every packfile and we are
more likely to have a miss on our most-recently-used packfile.
For some large repositories, repacking into a single packfile
is not feasible due to storage space or excessive repack times.</p>
</div>
<div class="paragraph">
<p>The multi-pack-index (MIDX for short) stores a list of objects
and their offsets into multiple packfiles. It contains:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>A list of packfile names.</p>
</li>
<li>
<p>A sorted list of object IDs.</p>
</li>
<li>
<p>A list of metadata for the ith object ID including:</p>
<div class="ulist">
<ul>
<li>
<p>A value j referring to the jth packfile.</p>
</li>
<li>
<p>An offset within the jth packfile for the object.</p>
</li>
</ul>
</div>
</li>
<li>
<p>If large offsets are required, we use another list of large
offsets similar to version 2 pack-indexes.</p>
<div class="ulist">
<ul>
<li>
<p>An optional list of objects in pseudo-pack order (used with MIDX bitmaps).</p>
</li>
</ul>
</div>
</li>
</ul>
</div>
<div class="paragraph">
<p>Thus, we can provide O(log N) lookup time for any number
of packfiles.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_design_details">Design Details</h2>
<div class="sectionbody">
<div class="ulist">
<ul>
<li>
<p>The MIDX is stored in a file named <em>multi-pack-index</em> in the
.git/objects/pack directory. This could be stored in the pack
directory of an alternate. It refers only to packfiles in that
same directory.</p>
</li>
<li>
<p>The core.multiPackIndex config setting must be on (which is the
default) to consume MIDX files. Setting it to <code>false</code> prevents
Git from reading a MIDX file, even if one exists.</p>
</li>
<li>
<p>The file format includes parameters for the object ID hash
function, so a future change of hash algorithm does not require
a change in format.</p>
</li>
<li>
<p>The MIDX keeps only one record per object ID. If an object appears
in multiple packfiles, then the MIDX selects the copy in the
preferred packfile, otherwise selecting from the most-recently
modified packfile.</p>
</li>
<li>
<p>If there exist packfiles in the pack directory not registered in
the MIDX, then those packfiles are loaded into the <code>packed_git</code>
list and <code>packed_git_mru</code> cache.</p>
</li>
<li>
<p>The pack-indexes (.idx files) remain in the pack directory so we
can delete the MIDX file, set core.midx to false, or downgrade
without any loss of information.</p>
</li>
<li>
<p>The MIDX file format uses a chunk-based approach (similar to the
commit-graph file) that allows optional data to be added.</p>
</li>
</ul>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_incremental_multi_pack_indexes">Incremental multi-pack indexes</h2>
<div class="sectionbody">
<div class="paragraph">
<p>As repositories grow in size, it becomes more expensive to write a
multi-pack index (MIDX) that includes all packfiles. To accommodate
this, the "incremental multi-pack indexes" feature allows for combining
a "chain" of multi-pack indexes.</p>
</div>
<div class="paragraph">
<p>Each individual component of the chain need only contain a small number
of packfiles. Appending to the chain does not invalidate earlier parts
of the chain, so repositories can control how much time is spent
updating the MIDX chain by determining the number of packs in each layer
of the MIDX chain.</p>
</div>
<div class="sect2">
<h3 id="_design_state">Design state</h3>
<div class="paragraph">
<p>At present, the incremental multi-pack indexes feature is missing two
important components:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>The ability to rewrite earlier portions of the MIDX chain (i.e., to
"compact" some collection of adjacent MIDX layers into a single
MIDX). At present the only supported way of shrinking a MIDX chain
is to rewrite the entire chain from scratch without the <code>--split</code>
flag.</p>
<div class="paragraph">
<p>There are no fundamental limitations that stand in the way of being able
to implement this feature. It is omitted from the initial implementation
in order to reduce the complexity, but will be added later.</p>
</div>
</li>
<li>
<p>Support for reachability bitmaps. The classic single MIDX
implementation does support reachability bitmaps (see the section
titled "multi-pack-index reverse indexes" in
<a href="../gitformat-pack.html">gitformat-pack(5)</a> for more details).</p>
<div class="paragraph">
<p>As above, there are no fundamental limitations that stand in the way of
extending the incremental MIDX format to support reachability bitmaps.
The design below specifically takes this into account, and support for
reachability bitmaps will be added in a future patch series. It is
omitted from the current implementation for the same reason as above.</p>
</div>
<div class="paragraph">
<p>In brief, to support reachability bitmaps with the incremental MIDX
feature, the concept of the pseudo-pack order is extended across each
layer of the incremental MIDX chain to form a concatenated pseudo-pack
order. This concatenation takes place in the same order as the chain
itself (in other words, the concatenated pseudo-pack order for a chain
{$H1, <code>$H2,</code> <code>$H3</code>} would be the pseudo-pack order for <code>$H1</code>, followed by
the pseudo-pack order for <code>$H2</code>, followed by the pseudo-pack order for
<code>$H3</code>).</p>
</div>
<div class="paragraph">
<p>The layout will then be extended so that each layer of the incremental
MIDX chain can write a *.<code>bitmap</code>. The objects in each layer&#8217;s bitmap
are offset by the number of objects in the previous layers of the chain.</p>
</div>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_file_layout">File layout</h3>
<div class="paragraph">
<p>Instead of storing a single <code>multi-pack-index</code> file (with an optional
.<code>rev</code> and .<code>bitmap</code> extension) in <code>$GIT_DIR/objects/pack</code>, incremental
MIDXs are stored in the following layout:</p>
</div>
<div class="listingblock">
<div class="content">
<pre>$GIT_DIR/objects/pack/multi-pack-index.d/
$GIT_DIR/objects/pack/multi-pack-index.d/multi-pack-index-chain
$GIT_DIR/objects/pack/multi-pack-index.d/multi-pack-index-$H1.midx
$GIT_DIR/objects/pack/multi-pack-index.d/multi-pack-index-$H2.midx
$GIT_DIR/objects/pack/multi-pack-index.d/multi-pack-index-$H3.midx</pre>
</div>
</div>
<div class="paragraph">
<p>The <code>multi-pack-index-chain</code> file contains a list of the incremental
MIDX files in the chain, in order. The above example shows a chain whose
<code>multi-pack-index-chain</code> file would contain the following lines:</p>
</div>
<div class="listingblock">
<div class="content">
<pre>$H1
$H2
$H3</pre>
</div>
</div>
<div class="paragraph">
<p>The <code>multi-pack-index-$H1.midx</code> file contains the first layer of the
multi-pack-index chain. The <code>multi-pack-index-$H2.midx</code> file contains
the second layer of the chain, and so on.</p>
</div>
<div class="paragraph">
<p>When both an incremental- and non-incremental MIDX are present, the
non-incremental MIDX is always read first.</p>
</div>
</div>
<div class="sect2">
<h3 id="_object_positions_for_incremental_midxs">Object positions for incremental MIDXs</h3>
<div class="paragraph">
<p>In the original multi-pack-index design, we refer to objects via their
lexicographic position (by object IDs) within the repository&#8217;s singular
multi-pack-index. In the incremental multi-pack-index design, we refer
to objects via their index into a concatenated lexicographic ordering
among each component in the MIDX chain.</p>
</div>
<div class="paragraph">
<p>If <code>objects_nr</code>() is a function that returns the number of objects in a
given MIDX layer, then the index of an object at lexicographic position
<code>i</code> within, say, $H3 is defined as:</p>
</div>
<div class="listingblock">
<div class="content">
<pre>objects_nr($H2) + objects_nr($H1) + i</pre>
</div>
</div>
<div class="paragraph">
<p>(in the C implementation, this is often computed as <code>i</code> <code>+</code>
<code>m-</code>&gt;<code>num_objects_in_base</code>).</p>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_future_work">Future Work</h2>
<div class="sectionbody">
<div class="ulist">
<ul>
<li>
<p>The multi-pack-index allows many packfiles, especially in a context
where repacking is expensive (such as a very large repo), or
unexpected maintenance time is unacceptable (such as a high-demand
build machine). However, the multi-pack-index needs to be rewritten
in full every time. We can extend the format to be incremental, so
writes are fast. By storing a small "tip" multi-pack-index that
points to large "base" MIDX files, we can keep writes fast while
still reducing the number of binary searches required for object
lookups.</p>
</li>
<li>
<p>If the multi-pack-index is extended to store a "stable object order"
(a function Order(hash) = integer that is constant for a given hash,
even as the multi-pack-index is updated) then MIDX bitmaps could be
updated independently of the MIDX.</p>
</li>
<li>
<p>Packfiles can be marked as "special" using empty files that share
the initial name but replace ".pack" with ".keep" or ".promisor".
We can add an optional chunk of data to the multi-pack-index that
records flags of information about the packfiles. This allows new
states, such as <em>repacked</em> or <em>redeltified</em>, that can help with
pack maintenance in a multi-pack environment. It may also be
helpful to organize packfiles by object type (commit, tree, blob,
etc.) and use this metadata to help that maintenance.</p>
</li>
</ul>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_related_links">Related Links</h2>
<div class="sectionbody">
<div class="paragraph">
<p>[0] <a href="https://bugs.chromium.org/p/git/issues/detail?id=6" class="bare">https://bugs.chromium.org/p/git/issues/detail?id=6</a>
Chromium work item for: Multi-Pack Index (MIDX)</p>
</div>
<div class="paragraph">
<p>[1] <a href="https://lore.kernel.org/git/20180107181459.222909-1-dstolee@microsoft.com/" class="bare">https://lore.kernel.org/git/20180107181459.222909-1-dstolee@microsoft.com/</a>
An earlier RFC for the multi-pack-index feature</p>
</div>
<div class="paragraph">
<p>[2] <a href="https://lore.kernel.org/git/alpine.DEB.2.20.1803091557510.23109@alexmv-linux/" class="bare">https://lore.kernel.org/git/alpine.DEB.2.20.1803091557510.23109@alexmv-linux/</a>
Git Merge 2018 Contributor&#8217;s summit notes (includes discussion of MIDX)</p>
</div>
</div>
</div>
</div>
<div id="footer">
<div id="footer-text">
Last updated 2025-03-17 07:09:38 UTC
</div>
</div>
</body>
</html>

View File

@@ -0,0 +1,460 @@
Concerning Git's Packing Heuristics
===================================
Oh, here's a really stupid question:
Where do I go
to learn the details
of Git's packing heuristics?
Be careful what you ask!
Followers of the Git, please open the Git IRC Log and turn to
February 10, 2006.
It's a rare occasion, and we are joined by the King Git Himself,
Linus Torvalds (linus). Nathaniel Smith, (njs`), has the floor
and seeks enlightenment. Others are present, but silent.
Let's listen in!
<njs`> Oh, here's a really stupid question -- where do I go to
learn the details of Git's packing heuristics? google avails
me not, reading the source didn't help a lot, and wading
through the whole mailing list seems less efficient than any
of that.
It is a bold start! A plea for help combined with a simultaneous
tri-part attack on some of the tried and true mainstays in the quest
for enlightenment. Brash accusations of google being useless. Hubris!
Maligning the source. Heresy! Disdain for the mailing list archives.
Woe.
<pasky> yes, the packing-related delta stuff is somewhat
mysterious even for me ;)
Ah! Modesty after all.
<linus> njs, I don't think the docs exist. That's something where
I don't think anybody else than me even really got involved.
Most of the rest of Git others have been busy with (especially
Junio), but packing nobody touched after I did it.
It's cryptic, yet vague. Linus in style for sure. Wise men
interpret this as an apology. A few argue it is merely a
statement of fact.
<njs`> I guess the next step is "read the source again", but I
have to build up a certain level of gumption first :-)
Indeed! On both points.
<linus> The packing heuristic is actually really really simple.
Bait...
<linus> But strange.
And switch. That ought to do it!
<linus> Remember: Git really doesn't follow files. So what it does is
- generate a list of all objects
- sort the list according to magic heuristics
- walk the list, using a sliding window, seeing if an object
can be diffed against another object in the window
- write out the list in recency order
The traditional understatement:
<njs`> I suspect that what I'm missing is the precise definition of
the word "magic"
The traditional insight:
<pasky> yes
And Babel-like confusion flowed.
<njs`> oh, hmm, and I'm not sure what this sliding window means either
<pasky> iirc, it appeared to me to be just the sha1 of the object
when reading the code casually ...
... which simply doesn't sound as a very good heuristics, though ;)
<njs`> .....and recency order. okay, I think it's clear I didn't
even realize how much I wasn't realizing :-)
Ah, grasshopper! And thus the enlightenment begins anew.
<linus> The "magic" is actually in theory totally arbitrary.
ANY order will give you a working pack, but no, it's not
ordered by SHA-1.
Before talking about the ordering for the sliding delta
window, let's talk about the recency order. That's more
important in one way.
<njs`> Right, but if all you want is a working way to pack things
together, you could just use cat and save yourself some
trouble...
Waaait for it....
<linus> The recency ordering (which is basically: put objects
_physically_ into the pack in the order that they are
"reachable" from the head) is important.
<njs`> okay
<linus> It's important because that's the thing that gives packs
good locality. It keeps the objects close to the head (whether
they are old or new, but they are _reachable_ from the head)
at the head of the pack. So packs actually have absolutely
_wonderful_ IO patterns.
Read that again, because it is important.
<linus> But recency ordering is totally useless for deciding how
to actually generate the deltas, so the delta ordering is
something else.
The delta ordering is (wait for it):
- first sort by the "basename" of the object, as defined by
the name the object was _first_ reached through when
generating the object list
- within the same basename, sort by size of the object
- but always sort different types separately (commits first).
That's not exactly it, but it's very close.
<njs`> The "_first_ reached" thing is not too important, just you
need some way to break ties since the same objects may be
reachable many ways, yes?
And as if to clarify:
<linus> The point is that it's all really just any random
heuristic, and the ordering is totally unimportant for
correctness, but it helps a lot if the heuristic gives
"clumping" for things that are likely to delta well against
each other.
It is an important point, so secretly, I did my own research and have
included my results below. To be fair, it has changed some over time.
And through the magic of Revisionistic History, I draw upon this entry
from The Git IRC Logs on my father's birthday, March 1:
<gitster> The quote from the above linus should be rewritten a
bit (wait for it):
- first sort by type. Different objects never delta with
each other.
- then sort by filename/dirname. hash of the basename
occupies the top BITS_PER_INT-DIR_BITS bits, and bottom
DIR_BITS are for the hash of leading path elements.
- then if we are doing "thin" pack, the objects we are _not_
going to pack but we know about are sorted earlier than
other objects.
- and finally sort by size, larger to smaller.
In one swell-foop, clarification and obscurification! Nonetheless,
authoritative. Cryptic, yet concise. It even solicits notions of
quotes from The Source Code. Clearly, more study is needed.
<gitster> That's the sort order. What this means is:
- we do not delta different object types.
- we prefer to delta the objects with the same full path, but
allow files with the same name from different directories.
- we always prefer to delta against objects we are not going
to send, if there are some.
- we prefer to delta against larger objects, so that we have
lots of removals.
The penultimate rule is for "thin" packs. It is used when
the other side is known to have such objects.
There it is again. "Thin" packs. I'm thinking to myself, "What
is a 'thin' pack?" So I ask:
<jdl> What is a "thin" pack?
<gitster> Use of --objects-edge to rev-list as the upstream of
pack-objects. The pack transfer protocol negotiates that.
Woo hoo! Cleared that _right_ up!
<gitster> There are two directions - push and fetch.
There! Did you see it? It is not '"push" and "pull"'! How often the
confusion has started here. So casually mentioned, too!
<gitster> For push, git-send-pack invokes git-receive-pack on the
other end. The receive-pack says "I have up to these commits".
send-pack looks at them, and computes what are missing from
the other end. So "thin" could be the default there.
In the other direction, fetch, git-fetch-pack and
git-clone-pack invokes git-upload-pack on the other end
(via ssh or by talking to the daemon).
There are two cases: fetch-pack with -k and clone-pack is one,
fetch-pack without -k is the other. clone-pack and fetch-pack
with -k will keep the downloaded packfile without expanded, so
we do not use thin pack transfer. Otherwise, the generated
pack will have delta without base object in the same pack.
But fetch-pack without -k will explode the received pack into
individual objects, so we automatically ask upload-pack to
give us a thin pack if upload-pack supports it.
OK then.
Uh.
Let's return to the previous conversation still in progress.
<njs`> and "basename" means something like "the tail of end of
path of file objects and dir objects, as per basename(3), and
we just declare all commit and tag objects to have the same
basename" or something?
Luckily, that too is a point that gitster clarified for us!
If I might add, the trick is to make files that _might_ be similar be
located close to each other in the hash buckets based on their file
names. It used to be that "foo/Makefile", "bar/baz/quux/Makefile" and
"Makefile" all landed in the same bucket due to their common basename,
"Makefile". However, now they land in "close" buckets.
The algorithm allows not just for the _same_ bucket, but for _close_
buckets to be considered delta candidates. The rationale is
essentially that files, like Makefiles, often have very similar
content no matter what directory they live in.
<linus> I played around with different delta algorithms, and with
making the "delta window" bigger, but having too big of a
sliding window makes it very expensive to generate the pack:
you need to compare every object with a _ton_ of other objects.
There are a number of other trivial heuristics too, which
basically boil down to "don't bother even trying to delta this
pair" if we can tell before-hand that the delta isn't worth it
(due to size differences, where we can take a previous delta
result into account to decide that "ok, no point in trying
that one, it will be worse").
End result: packing is actually very size efficient. It's
somewhat CPU-wasteful, but on the other hand, since you're
really only supposed to do it maybe once a month (and you can
do it during the night), nobody really seems to care.
Nice Engineering Touch, there. Find when it doesn't matter, and
proclaim it a non-issue. Good style too!
<njs`> So, just to repeat to see if I'm following, we start by
getting a list of the objects we want to pack, we sort it by
this heuristic (basically lexicographically on the tuple
(type, basename, size)).
Then we walk through this list, and calculate a delta of
each object against the last n (tunable parameter) objects,
and pick the smallest of these deltas.
Vastly simplified, but the essence is there!
<linus> Correct.
<njs`> And then once we have picked a delta or fulltext to
represent each object, we re-sort by recency, and write them
out in that order.
<linus> Yup. Some other small details:
And of course there is the "Other Shoe" Factor too.
<linus> - We limit the delta depth to another magic value (right
now both the window and delta depth magic values are just "10")
<njs`> Hrm, my intuition is that you'd end up with really _bad_ IO
patterns, because the things you want are near by, but to
actually reconstruct them you may have to jump all over in
random ways.
<linus> - When we write out a delta, and we haven't yet written
out the object it is a delta against, we write out the base
object first. And no, when we reconstruct them, we actually
get nice IO patterns, because:
- larger objects tend to be "more recent" (Linus' law: files grow)
- we actively try to generate deltas from a larger object to a
smaller one
- this means that the top-of-tree very seldom has deltas
(i.e. deltas in _practice_ are "backwards deltas")
Again, we should reread that whole paragraph. Not just because
Linus has slipped Linus's Law in there on us, but because it is
important. Let's make sure we clarify some of the points here:
<njs`> So the point is just that in practice, delta order and
recency order match each other quite well.
<linus> Yes. There's another nice side to this (and yes, it was
designed that way ;):
- the reason we generate deltas against the larger object is
actually a big space saver too!
<njs`> Hmm, but your last comment (if "we haven't yet written out
the object it is a delta against, we write out the base object
first"), seems like it would make these facts mostly
irrelevant because even if in practice you would not have to
wander around much, in fact you just brute-force say that in
the cases where you might have to wander, don't do that :-)
<linus> Yes and no. Notice the rule: we only write out the base
object first if the delta against it was more recent. That
means that you can actually have deltas that refer to a base
object that is _not_ close to the delta object, but that only
happens when the delta is needed to generate an _old_ object.
<linus> See?
Yeah, no. I missed that on the first two or three readings myself.
<linus> This keeps the front of the pack dense. The front of the
pack never contains data that isn't relevant to a "recent"
object. The size optimization comes from our use of xdelta
(but is true for many other delta algorithms): removing data
is cheaper (in size) than adding data.
When you remove data, you only need to say "copy bytes n--m".
In contrast, in a delta that _adds_ data, you have to say "add
these bytes: 'actual data goes here'"
*** njs` has quit: Read error: 104 (Connection reset by peer)
<linus> Uhhuh. I hope I didn't blow njs` mind.
*** njs` has joined channel #git
<pasky> :)
The silent observers are amused. Of course.
And as if njs` was expected to be omniscient:
<linus> njs - did you miss anything?
OK, I'll spell it out. That's Geek Humor. If njs` was not actually
connected for a little bit there, how would he know if missed anything
while he was disconnected? He's a benevolent dictator with a sense of
humor! Well noted!
<njs`> Stupid router. Or gremlins, or whatever.
It's a cheap shot at Cisco. Take 'em when you can.
<njs`> Yes and no. Notice the rule: we only write out the base
object first if the delta against it was more recent.
I'm getting lost in all these orders, let me re-read :-)
So the write-out order is from most recent to least recent?
(Conceivably it could be the opposite way too, I'm not sure if
we've said) though my connection back at home is logging, so I
can just read what you said there :-)
And for those of you paying attention, the Omniscient Trick has just
been detailed!
<linus> Yes, we always write out most recent first
<njs`> And, yeah, I got the part about deeper-in-history stuff
having worse IO characteristics, one sort of doesn't care.
<linus> With the caveat that if the "most recent" needs an older
object to delta against (hey, shrinking sometimes does
happen), we write out the old object with the delta.
<njs`> (if only it happened more...)
<linus> Anyway, the pack-file could easily be denser still, but
because it's used both for streaming (the Git protocol) and
for on-disk, it has a few pessimizations.
Actually, it is a made-up word. But it is a made-up word being
used as setup for a later optimization, which is a real word:
<linus> In particular, while the pack-file is then compressed,
it's compressed just one object at a time, so the actual
compression factor is less than it could be in theory. But it
means that it's all nice random-access with a simple index to
do "object name->location in packfile" translation.
<njs`> I'm assuming the real win for delta-ing large->small is
more homogeneous statistics for gzip to run over?
(You have to put the bytes in one place or another, but
putting them in a larger blob wins on compression)
Actually, what is the compression strategy -- each delta
individually gzipped, the whole file gzipped, somewhere in
between, no compression at all, ....?
Right.
Reality IRC sets in. For example:
<pasky> I'll read the rest in the morning, I really have to go
sleep or there's no hope whatsoever for me at the today's
exam... g'nite all.
Heh.
<linus> pasky: g'nite
<njs`> pasky: 'luck
<linus> Right: large->small matters exactly because of compression
behaviour. If it was non-compressed, it probably wouldn't make
any difference.
<njs`> yeah
<linus> Anyway: I'm not even trying to claim that the pack-files
are perfect, but they do tend to have a nice balance of
density vs ease-of use.
Gasp! OK, saved. That's a fair Engineering trade off. Close call!
In fact, Linus reflects on some Basic Engineering Fundamentals,
design options, etc.
<linus> More importantly, they allow Git to still _conceptually_
never deal with deltas at all, and be a "whole object" store.
Which has some problems (we discussed bad huge-file
behaviour on the Git lists the other day), but it does mean
that the basic Git concepts are really really simple and
straightforward.
It's all been quite stable.
Which I think is very much a result of having very simple
basic ideas, so that there's never any confusion about what's
going on.
Bugs happen, but they are "simple" bugs. And bugs that
actually get some object store detail wrong are almost always
so obvious that they never go anywhere.
<njs`> Yeah.
Nuff said.
<linus> Anyway. I'm off for bed. It's not 6AM here, but I've got
three kids, and have to get up early in the morning to send
them off. I need my beauty sleep.
<njs`> :-)
<njs`> appreciate the infodump, I really was failing to find the
details on Git packs :-)
And now you know the rest of the story.

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,82 @@
Packfile URIs
=============
This feature allows servers to serve part of their packfile response as URIs.
This allows server designs that improve scalability in bandwidth and CPU usage
(for example, by serving some data through a CDN), and (in the future) provides
some measure of resumability to clients.
This feature is available only in protocol version 2.
Protocol
--------
The server advertises the `packfile-uris` capability.
If the client then communicates which protocols (HTTPS, etc.) it supports with
a `packfile-uris` argument, the server MAY send a `packfile-uris` section
directly before the `packfile` section (right after `wanted-refs` if it is
sent) containing URIs of any of the given protocols. The URIs point to
packfiles that use only features that the client has declared that it supports
(e.g. ofs-delta and thin-pack). See linkgit:gitprotocol-v2[5] for the documentation of
this section.
Clients should then download and index all the given URIs (in addition to
downloading and indexing the packfile given in the `packfile` section of the
response) before performing the connectivity check.
Server design
-------------
The server can be trivially made compatible with the proposed protocol by
having it advertise `packfile-uris`, tolerating the client sending
`packfile-uris`, and never sending any `packfile-uris` section. But we should
include some sort of non-trivial implementation in the Minimum Viable Product,
at least so that we can test the client.
This is the implementation: a feature, marked experimental, that allows the
server to be configured by one or more `uploadpack.blobPackfileUri=
<object-hash> <pack-hash> <uri>` entries. Whenever the list of objects to be
sent is assembled, all such blobs are excluded, replaced with URIs. As noted
in "Future work" below, the server can evolve in the future to support
excluding other objects (or other implementations of servers could be made
that support excluding other objects) without needing a protocol change, so
clients should not expect that packfiles downloaded in this way only contain
single blobs.
Client design
-------------
The client has a config variable `fetch.uriprotocols` that determines which
protocols the end user is willing to use. By default, this is empty.
When the client downloads the given URIs, it should store them with "keep"
files, just like it does with the packfile in the `packfile` section. These
additional "keep" files can only be removed after the refs have been updated -
just like the "keep" file for the packfile in the `packfile` section.
The division of work (initial fetch + additional URIs) introduces convenient
points for resumption of an interrupted clone - such resumption can be done
after the Minimum Viable Product (see "Future work").
Future work
-----------
The protocol design allows some evolution of the server and client without any
need for protocol changes, so only a small-scoped design is included here to
form the MVP. For example, the following can be done:
* On the server, more sophisticated means of excluding objects (e.g. by
specifying a commit to represent that commit and all objects that it
references).
* On the client, resumption of clone. If a clone is interrupted, information
could be recorded in the repository's config and a "clone-resume" command
can resume the clone in progress. (Resumption of subsequent fetches is more
difficult because that must deal with the user wanting to use the repository
even after the fetch was interrupted.)
There are some possible features that will require a change in protocol:
* Additional HTTP headers (e.g. authentication)
* Byte range support
* Different file formats referenced by URIs (e.g. raw object)

View File

@@ -0,0 +1,270 @@
Parallel Checkout Design Notes
==============================
The "Parallel Checkout" feature attempts to use multiple processes to
parallelize the work of uncompressing the blobs, applying in-core
filters, and writing the resulting contents to the working tree during a
checkout operation. It can be used by all checkout-related commands,
such as `clone`, `checkout`, `reset`, `sparse-checkout`, and others.
These commands share the following basic structure:
* Step 1: Read the current index file into memory.
* Step 2: Modify the in-memory index based upon the command, and
temporarily mark all cache entries that need to be updated.
* Step 3: Populate the working tree to match the new candidate index.
This includes iterating over all of the to-be-updated cache entries
and delete, create, or overwrite the associated files in the working
tree.
* Step 4: Write the new index to disk.
Step 3 is the focus of the "parallel checkout" effort described here.
Sequential Implementation
-------------------------
For the purposes of discussion here, the current sequential
implementation of Step 3 is divided in 3 parts, each one implemented in
its own function:
* Step 3a: `unpack-trees.c:check_updates()` contains a series of
sequential loops iterating over the `cache_entry`'s array. The main
loop in this function calls the Step 3b function for each of the
to-be-updated entries.
* Step 3b: `entry.c:checkout_entry()` examines the existing working tree
for file conflicts, collisions, and unsaved changes. It removes files
and creates leading directories as necessary. It calls the Step 3c
function for each entry to be written.
* Step 3c: `entry.c:write_entry()` loads the blob into memory, smudges
it if necessary, creates the file in the working tree, writes the
smudged contents, calls `fstat()` or `lstat()`, and updates the
associated `cache_entry` struct with the stat information gathered.
It wouldn't be safe to perform Step 3b in parallel, as there could be
race conditions between file creations and removals. Instead, the
parallel checkout framework lets the sequential code handle Step 3b,
and uses parallel workers to replace the sequential
`entry.c:write_entry()` calls from Step 3c.
Rejected Multi-Threaded Solution
--------------------------------
The most "straightforward" implementation would be to spread the set of
to-be-updated cache entries across multiple threads. But due to the
thread-unsafe functions in the object database code, we would have to use locks to
coordinate the parallel operation. An early prototype of this solution
showed that the multi-threaded checkout would bring performance
improvements over the sequential code, but there was still too much lock
contention. A `perf` profiling indicated that around 20% of the runtime
during a local Linux clone (on an SSD) was spent in locking functions.
For this reason this approach was rejected in favor of using multiple
child processes, which led to better performance.
Multi-Process Solution
----------------------
Parallel checkout alters the aforementioned Step 3 to use multiple
`checkout--worker` background processes to distribute the work. The
long-running worker processes are controlled by the foreground Git
command using the existing run-command API.
Overview
~~~~~~~~
Step 3b is only slightly altered; for each entry to be checked out, the
main process performs the following steps:
* M1: Check whether there is any untracked or unclean file in the
working tree which would be overwritten by this entry, and decide
whether to proceed (removing the file(s)) or not.
* M2: Create the leading directories.
* M3: Load the conversion attributes for the entry's path.
* M4: Check, based on the entry's type and conversion attributes,
whether the entry is eligible for parallel checkout (more on this
later). If it is eligible, enqueue the entry and the loaded
attributes to later write the entry in parallel. If not, write the
entry right away, using the default sequential code.
Note: we save the conversion attributes associated with each entry
because the workers don't have access to the main process' index state,
so they can't load the attributes by themselves (and the attributes are
needed to properly smudge the entry). Additionally, this has a positive
impact on performance as (1) we don't need to load the attributes twice
and (2) the attributes machinery is optimized to handle paths in
sequential order.
After all entries have passed through the above steps, the main process
checks if the number of enqueued entries is sufficient to spread among
the workers. If not, it just writes them sequentially. Otherwise, it
spawns the workers and distributes the queued entries uniformly in
continuous chunks. This aims to minimize the chances of two workers
writing to the same directory simultaneously, which could increase lock
contention in the kernel.
Then, for each assigned item, each worker:
* W1: Checks if there is any non-directory file in the leading part of
the entry's path or if there already exists a file at the entry' path.
If so, mark the entry with `PC_ITEM_COLLIDED` and skip it (more on
this later).
* W2: Creates the file (with O_CREAT and O_EXCL).
* W3: Loads the blob into memory (inflating and delta reconstructing
it).
* W4: Applies any required in-process filter, like end-of-line
conversion and re-encoding.
* W5: Writes the result to the file descriptor opened at W2.
* W6: Calls `fstat()` or `lstat()` on the just-written path, and sends
the result back to the main process, together with the end status of
the operation and the item's identification number.
Note that, when possible, steps W3 to W5 are delegated to the streaming
machinery, removing the need to keep the entire blob in memory.
If the worker fails to read the blob or to write it to the working tree,
it removes the created file to avoid leaving empty files behind. This is
the *only* time a worker is allowed to remove a file.
As mentioned earlier, it is the responsibility of the main process to
remove any file that blocks the checkout operation (or abort if the
removal(s) would cause data loss and the user didn't ask to `--force`).
This is crucial to avoid race conditions and also to properly detect
path collisions at Step W1.
After the workers finish writing the items and sending back the required
information, the main process handles the results in two steps:
- First, it updates the in-memory index with the `lstat()` information
sent by the workers. (This must be done first as this information
might be required in the following step.)
- Then it writes the items which collided on disk (i.e. items marked
with `PC_ITEM_COLLIDED`). More on this below.
Path Collisions
---------------
Path collisions happen when two different paths correspond to the same
entry in the file system. E.g. the paths 'a' and 'A' would collide in a
case-insensitive file system.
The sequential checkout deals with collisions in the same way that it
deals with files that were already present in the working tree before
checkout. Basically, it checks if the path that it wants to write
already exists on disk, makes sure the existing file doesn't have
unsaved data, and then overwrites it. (To be more pedantic: it deletes
the existing file and creates the new one.) So, if there are multiple
colliding files to be checked out, the sequential code will write each
one of them but only the last will actually survive on disk.
Parallel checkout aims to reproduce the same behavior. However, we
cannot let the workers racily write to the same file on disk. Instead,
the workers detect when the entry that they want to check out would
collide with an existing file, and mark it with `PC_ITEM_COLLIDED`.
Later, the main process can sequentially feed these entries back to
`checkout_entry()` without the risk of race conditions. On clone, this
also has the effect of marking the colliding entries to later emit a
warning for the user, like the classic sequential checkout does.
The workers are able to detect both collisions among the entries being
concurrently written and collisions between a parallel-eligible entry
and an ineligible entry. The general idea for collision detection is
quite straightforward: for each parallel-eligible entry, the main
process must remove all files that prevent this entry from being written
(before enqueueing it). This includes any non-directory file in the
leading path of the entry. Later, when a worker gets assigned the entry,
it looks again for the non-directory files and for an already existing
file at the entry's path. If any of these checks finds something, the
worker knows that there was a path collision.
Because parallel checkout can distinguish path collisions from the case
where the file was already present in the working tree before checkout,
we could alternatively choose to skip the checkout of colliding entries.
However, each entry that doesn't get written would have NULL `lstat()`
fields on the index. This could cause performance penalties for
subsequent commands that need to refresh the index, as they would have
to go to the file system to see if the entry is dirty. Thus, if we have
N entries in a colliding group and we decide to write and `lstat()` only
one of them, every subsequent `git-status` will have to read, convert,
and hash the written file N - 1 times. By checking out all colliding
entries (like the sequential code does), we only pay the overhead once,
during checkout.
Eligible Entries for Parallel Checkout
--------------------------------------
As previously mentioned, not all entries passed to `checkout_entry()`
will be considered eligible for parallel checkout. More specifically, we
exclude:
- Symbolic links; to avoid race conditions that, in combination with
path collisions, could cause workers to write files at the wrong
place. For example, if we were to concurrently check out a symlink
'a' -> 'b' and a regular file 'A/f' in a case-insensitive file system,
we could potentially end up writing the file 'A/f' at 'a/f', due to a
race condition.
- Regular files that require external filters (either "one shot" filters
or long-running process filters). These filters are black-boxes to Git
and may have their own internal locking or non-concurrent assumptions.
So it might not be safe to run multiple instances in parallel.
+
Besides, long-running filters may use the delayed checkout feature to
postpone the return of some filtered blobs. The delayed checkout queue
and the parallel checkout queue are not compatible and should remain
separate.
+
Note: regular files that only require internal filters, like end-of-line
conversion and re-encoding, are eligible for parallel checkout.
Ineligible entries are checked out by the classic sequential codepath
*before* spawning workers.
Note: submodules' files are also eligible for parallel checkout (as
long as they don't fall into any of the excluding categories mentioned
above). But since each submodule is checked out in its own child
process, we don't mix the superproject's and the submodules' files in
the same parallel checkout process or queue.
The API
-------
The parallel checkout API was designed with the goal of minimizing
changes to the current users of the checkout machinery. This means that
they don't have to call a different function for sequential or parallel
checkout. As already mentioned, `checkout_entry()` will automatically
insert the given entry in the parallel checkout queue when this feature
is enabled and the entry is eligible; otherwise, it will just write the
entry right away, using the sequential code. In general, callers of the
parallel checkout API should look similar to this:
----------------------------------------------
int pc_workers, pc_threshold, err = 0;
struct checkout state;
get_parallel_checkout_configs(&pc_workers, &pc_threshold);
/*
* This check is not strictly required, but it
* should save some time in sequential mode.
*/
if (pc_workers > 1)
init_parallel_checkout();
for (each cache_entry ce to-be-updated)
err |= checkout_entry(ce, &state, NULL, NULL);
err |= run_parallel_checkout(&state, pc_workers, pc_threshold, NULL, NULL);
----------------------------------------------

View File

@@ -0,0 +1,805 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="en">
<head>
<meta charset="UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=edge"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<meta name="generator" content="Asciidoctor 2.0.23"/>
<title>Parallel Checkout Design Notes</title>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"/>
<style>
/*! Asciidoctor default stylesheet | MIT License | https://asciidoctor.org */
/* Uncomment the following line when using as a custom stylesheet */
/* @import "https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"; */
html{font-family:sans-serif;-webkit-text-size-adjust:100%}
a{background:none}
a:focus{outline:thin dotted}
a:active,a:hover{outline:0}
h1{font-size:2em;margin:.67em 0}
b,strong{font-weight:bold}
abbr{font-size:.9em}
abbr[title]{cursor:help;border-bottom:1px dotted #dddddf;text-decoration:none}
dfn{font-style:italic}
hr{height:0}
mark{background:#ff0;color:#000}
code,kbd,pre,samp{font-family:monospace;font-size:1em}
pre{white-space:pre-wrap}
q{quotes:"\201C" "\201D" "\2018" "\2019"}
small{font-size:80%}
sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline}
sup{top:-.5em}
sub{bottom:-.25em}
img{border:0}
svg:not(:root){overflow:hidden}
figure{margin:0}
audio,video{display:inline-block}
audio:not([controls]){display:none;height:0}
fieldset{border:1px solid silver;margin:0 2px;padding:.35em .625em .75em}
legend{border:0;padding:0}
button,input,select,textarea{font-family:inherit;font-size:100%;margin:0}
button,input{line-height:normal}
button,select{text-transform:none}
button,html input[type=button],input[type=reset],input[type=submit]{-webkit-appearance:button;cursor:pointer}
button[disabled],html input[disabled]{cursor:default}
input[type=checkbox],input[type=radio]{padding:0}
button::-moz-focus-inner,input::-moz-focus-inner{border:0;padding:0}
textarea{overflow:auto;vertical-align:top}
table{border-collapse:collapse;border-spacing:0}
*,::before,::after{box-sizing:border-box}
html,body{font-size:100%}
body{background:#fff;color:rgba(0,0,0,.8);padding:0;margin:0;font-family:"Noto Serif","DejaVu Serif",serif;line-height:1;position:relative;cursor:auto;-moz-tab-size:4;-o-tab-size:4;tab-size:4;word-wrap:anywhere;-moz-osx-font-smoothing:grayscale;-webkit-font-smoothing:antialiased}
a:hover{cursor:pointer}
img,object,embed{max-width:100%;height:auto}
object,embed{height:100%}
img{-ms-interpolation-mode:bicubic}
.left{float:left!important}
.right{float:right!important}
.text-left{text-align:left!important}
.text-right{text-align:right!important}
.text-center{text-align:center!important}
.text-justify{text-align:justify!important}
.hide{display:none}
img,object,svg{display:inline-block;vertical-align:middle}
textarea{height:auto;min-height:50px}
select{width:100%}
.subheader,.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{line-height:1.45;color:#7a2518;font-weight:400;margin-top:0;margin-bottom:.25em}
div,dl,dt,dd,ul,ol,li,h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6,pre,form,p,blockquote,th,td{margin:0;padding:0}
a{color:#2156a5;text-decoration:underline;line-height:inherit}
a:hover,a:focus{color:#1d4b8f}
a img{border:0}
p{line-height:1.6;margin-bottom:1.25em;text-rendering:optimizeLegibility}
p aside{font-size:.875em;line-height:1.35;font-style:italic}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{font-family:"Open Sans","DejaVu Sans",sans-serif;font-weight:300;font-style:normal;color:#ba3925;text-rendering:optimizeLegibility;margin-top:1em;margin-bottom:.5em;line-height:1.0125em}
h1 small,h2 small,h3 small,#toctitle small,.sidebarblock>.content>.title small,h4 small,h5 small,h6 small{font-size:60%;color:#e99b8f;line-height:0}
h1{font-size:2.125em}
h2{font-size:1.6875em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.375em}
h4,h5{font-size:1.125em}
h6{font-size:1em}
hr{border:solid #dddddf;border-width:1px 0 0;clear:both;margin:1.25em 0 1.1875em}
em,i{font-style:italic;line-height:inherit}
strong,b{font-weight:bold;line-height:inherit}
small{font-size:60%;line-height:inherit}
code{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:400;color:rgba(0,0,0,.9)}
ul,ol,dl{line-height:1.6;margin-bottom:1.25em;list-style-position:outside;font-family:inherit}
ul,ol{margin-left:1.5em}
ul li ul,ul li ol{margin-left:1.25em;margin-bottom:0}
ul.circle{list-style-type:circle}
ul.disc{list-style-type:disc}
ul.square{list-style-type:square}
ul.circle ul:not([class]),ul.disc ul:not([class]),ul.square ul:not([class]){list-style:inherit}
ol li ul,ol li ol{margin-left:1.25em;margin-bottom:0}
dl dt{margin-bottom:.3125em;font-weight:bold}
dl dd{margin-bottom:1.25em}
blockquote{margin:0 0 1.25em;padding:.5625em 1.25em 0 1.1875em;border-left:1px solid #ddd}
blockquote,blockquote p{line-height:1.6;color:rgba(0,0,0,.85)}
@media screen and (min-width:768px){h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2}
h1{font-size:2.75em}
h2{font-size:2.3125em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.6875em}
h4{font-size:1.4375em}}
table{background:#fff;margin-bottom:1.25em;border:1px solid #dedede;word-wrap:normal}
table thead,table tfoot{background:#f7f8f7}
table thead tr th,table thead tr td,table tfoot tr th,table tfoot tr td{padding:.5em .625em .625em;font-size:inherit;color:rgba(0,0,0,.8);text-align:left}
table tr th,table tr td{padding:.5625em .625em;font-size:inherit;color:rgba(0,0,0,.8)}
table tr.even,table tr.alt{background:#f8f8f7}
table thead tr th,table tfoot tr th,table tbody tr td,table tr td,table tfoot tr td{line-height:1.6}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2;word-spacing:-.05em}
h1 strong,h2 strong,h3 strong,#toctitle strong,.sidebarblock>.content>.title strong,h4 strong,h5 strong,h6 strong{font-weight:400}
.center{margin-left:auto;margin-right:auto}
.stretch{width:100%}
.clearfix::before,.clearfix::after,.float-group::before,.float-group::after{content:" ";display:table}
.clearfix::after,.float-group::after{clear:both}
:not(pre).nobreak{word-wrap:normal}
:not(pre).nowrap{white-space:nowrap}
:not(pre).pre-wrap{white-space:pre-wrap}
:not(pre):not([class^=L])>code{font-size:.9375em;font-style:normal!important;letter-spacing:0;padding:.1em .5ex;word-spacing:-.15em;background:#f7f7f8;border-radius:4px;line-height:1.45;text-rendering:optimizeSpeed}
pre{color:rgba(0,0,0,.9);font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;line-height:1.45;text-rendering:optimizeSpeed}
pre code,pre pre{color:inherit;font-size:inherit;line-height:inherit}
pre>code{display:block}
pre.nowrap,pre.nowrap pre{white-space:pre;word-wrap:normal}
em em{font-style:normal}
strong strong{font-weight:400}
.keyseq{color:rgba(51,51,51,.8)}
kbd{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;display:inline-block;color:rgba(0,0,0,.8);font-size:.65em;line-height:1.45;background:#f7f7f7;border:1px solid #ccc;border-radius:3px;box-shadow:0 1px 0 rgba(0,0,0,.2),inset 0 0 0 .1em #fff;margin:0 .15em;padding:.2em .5em;vertical-align:middle;position:relative;top:-.1em;white-space:nowrap}
.keyseq kbd:first-child{margin-left:0}
.keyseq kbd:last-child{margin-right:0}
.menuseq,.menuref{color:#000}
.menuseq b:not(.caret),.menuref{font-weight:inherit}
.menuseq{word-spacing:-.02em}
.menuseq b.caret{font-size:1.25em;line-height:.8}
.menuseq i.caret{font-weight:bold;text-align:center;width:.45em}
b.button::before,b.button::after{position:relative;top:-1px;font-weight:400}
b.button::before{content:"[";padding:0 3px 0 2px}
b.button::after{content:"]";padding:0 2px 0 3px}
p a>code:hover{color:rgba(0,0,0,.9)}
#header,#content,#footnotes,#footer{width:100%;margin:0 auto;max-width:62.5em;*zoom:1;position:relative;padding-left:.9375em;padding-right:.9375em}
#header::before,#header::after,#content::before,#content::after,#footnotes::before,#footnotes::after,#footer::before,#footer::after{content:" ";display:table}
#header::after,#content::after,#footnotes::after,#footer::after{clear:both}
#content{margin-top:1.25em}
#content::before{content:none}
#header>h1:first-child{color:rgba(0,0,0,.85);margin-top:2.25rem;margin-bottom:0}
#header>h1:first-child+#toc{margin-top:8px;border-top:1px solid #dddddf}
#header>h1:only-child{border-bottom:1px solid #dddddf;padding-bottom:8px}
#header .details{border-bottom:1px solid #dddddf;line-height:1.45;padding-top:.25em;padding-bottom:.25em;padding-left:.25em;color:rgba(0,0,0,.6);display:flex;flex-flow:row wrap}
#header .details span:first-child{margin-left:-.125em}
#header .details span.email a{color:rgba(0,0,0,.85)}
#header .details br{display:none}
#header .details br+span::before{content:"\00a0\2013\00a0"}
#header .details br+span.author::before{content:"\00a0\22c5\00a0";color:rgba(0,0,0,.85)}
#header .details br+span#revremark::before{content:"\00a0|\00a0"}
#header #revnumber{text-transform:capitalize}
#header #revnumber::after{content:"\00a0"}
#content>h1:first-child:not([class]){color:rgba(0,0,0,.85);border-bottom:1px solid #dddddf;padding-bottom:8px;margin-top:0;padding-top:1rem;margin-bottom:1.25rem}
#toc{border-bottom:1px solid #e7e7e9;padding-bottom:.5em}
#toc>ul{margin-left:.125em}
#toc ul.sectlevel0>li>a{font-style:italic}
#toc ul.sectlevel0 ul.sectlevel1{margin:.5em 0}
#toc ul{font-family:"Open Sans","DejaVu Sans",sans-serif;list-style-type:none}
#toc li{line-height:1.3334;margin-top:.3334em}
#toc a{text-decoration:none}
#toc a:active{text-decoration:underline}
#toctitle{color:#7a2518;font-size:1.2em}
@media screen and (min-width:768px){#toctitle{font-size:1.375em}
body.toc2{padding-left:15em;padding-right:0}
body.toc2 #header>h1:nth-last-child(2){border-bottom:1px solid #dddddf;padding-bottom:8px}
#toc.toc2{margin-top:0!important;background:#f8f8f7;position:fixed;width:15em;left:0;top:0;border-right:1px solid #e7e7e9;border-top-width:0!important;border-bottom-width:0!important;z-index:1000;padding:1.25em 1em;height:100%;overflow:auto}
#toc.toc2 #toctitle{margin-top:0;margin-bottom:.8rem;font-size:1.2em}
#toc.toc2>ul{font-size:.9em;margin-bottom:0}
#toc.toc2 ul ul{margin-left:0;padding-left:1em}
#toc.toc2 ul.sectlevel0 ul.sectlevel1{padding-left:0;margin-top:.5em;margin-bottom:.5em}
body.toc2.toc-right{padding-left:0;padding-right:15em}
body.toc2.toc-right #toc.toc2{border-right-width:0;border-left:1px solid #e7e7e9;left:auto;right:0}}
@media screen and (min-width:1280px){body.toc2{padding-left:20em;padding-right:0}
#toc.toc2{width:20em}
#toc.toc2 #toctitle{font-size:1.375em}
#toc.toc2>ul{font-size:.95em}
#toc.toc2 ul ul{padding-left:1.25em}
body.toc2.toc-right{padding-left:0;padding-right:20em}}
#content #toc{border:1px solid #e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;border-radius:4px}
#content #toc>:first-child{margin-top:0}
#content #toc>:last-child{margin-bottom:0}
#footer{max-width:none;background:rgba(0,0,0,.8);padding:1.25em}
#footer-text{color:hsla(0,0%,100%,.8);line-height:1.44}
#content{margin-bottom:.625em}
.sect1{padding-bottom:.625em}
@media screen and (min-width:768px){#content{margin-bottom:1.25em}
.sect1{padding-bottom:1.25em}}
.sect1:last-child{padding-bottom:0}
.sect1+.sect1{border-top:1px solid #e7e7e9}
#content h1>a.anchor,h2>a.anchor,h3>a.anchor,#toctitle>a.anchor,.sidebarblock>.content>.title>a.anchor,h4>a.anchor,h5>a.anchor,h6>a.anchor{position:absolute;z-index:1001;width:1.5ex;margin-left:-1.5ex;display:block;text-decoration:none!important;visibility:hidden;text-align:center;font-weight:400}
#content h1>a.anchor::before,h2>a.anchor::before,h3>a.anchor::before,#toctitle>a.anchor::before,.sidebarblock>.content>.title>a.anchor::before,h4>a.anchor::before,h5>a.anchor::before,h6>a.anchor::before{content:"\00A7";font-size:.85em;display:block;padding-top:.1em}
#content h1:hover>a.anchor,#content h1>a.anchor:hover,h2:hover>a.anchor,h2>a.anchor:hover,h3:hover>a.anchor,#toctitle:hover>a.anchor,.sidebarblock>.content>.title:hover>a.anchor,h3>a.anchor:hover,#toctitle>a.anchor:hover,.sidebarblock>.content>.title>a.anchor:hover,h4:hover>a.anchor,h4>a.anchor:hover,h5:hover>a.anchor,h5>a.anchor:hover,h6:hover>a.anchor,h6>a.anchor:hover{visibility:visible}
#content h1>a.link,h2>a.link,h3>a.link,#toctitle>a.link,.sidebarblock>.content>.title>a.link,h4>a.link,h5>a.link,h6>a.link{color:#ba3925;text-decoration:none}
#content h1>a.link:hover,h2>a.link:hover,h3>a.link:hover,#toctitle>a.link:hover,.sidebarblock>.content>.title>a.link:hover,h4>a.link:hover,h5>a.link:hover,h6>a.link:hover{color:#a53221}
details,.audioblock,.imageblock,.literalblock,.listingblock,.stemblock,.videoblock{margin-bottom:1.25em}
details{margin-left:1.25rem}
details>summary{cursor:pointer;display:block;position:relative;line-height:1.6;margin-bottom:.625rem;outline:none;-webkit-tap-highlight-color:transparent}
details>summary::-webkit-details-marker{display:none}
details>summary::before{content:"";border:solid transparent;border-left:solid;border-width:.3em 0 .3em .5em;position:absolute;top:.5em;left:-1.25rem;transform:translateX(15%)}
details[open]>summary::before{border:solid transparent;border-top:solid;border-width:.5em .3em 0;transform:translateY(15%)}
details>summary::after{content:"";width:1.25rem;height:1em;position:absolute;top:.3em;left:-1.25rem}
.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{text-rendering:optimizeLegibility;text-align:left;font-family:"Noto Serif","DejaVu Serif",serif;font-size:1rem;font-style:italic}
table.tableblock.fit-content>caption.title{white-space:nowrap;width:0}
.paragraph.lead>p,#preamble>.sectionbody>[class=paragraph]:first-of-type p{font-size:1.21875em;line-height:1.6;color:rgba(0,0,0,.85)}
.admonitionblock>table{border-collapse:separate;border:0;background:none;width:100%}
.admonitionblock>table td.icon{text-align:center;width:80px}
.admonitionblock>table td.icon img{max-width:none}
.admonitionblock>table td.icon .title{font-weight:bold;font-family:"Open Sans","DejaVu Sans",sans-serif;text-transform:uppercase}
.admonitionblock>table td.content{padding-left:1.125em;padding-right:1.25em;border-left:1px solid #dddddf;color:rgba(0,0,0,.6);word-wrap:anywhere}
.admonitionblock>table td.content>:last-child>:last-child{margin-bottom:0}
.exampleblock>.content{border:1px solid #e6e6e6;margin-bottom:1.25em;padding:1.25em;background:#fff;border-radius:4px}
.sidebarblock{border:1px solid #dbdbd6;margin-bottom:1.25em;padding:1.25em;background:#f3f3f2;border-radius:4px}
.sidebarblock>.content>.title{color:#7a2518;margin-top:0;text-align:center}
.exampleblock>.content>:first-child,.sidebarblock>.content>:first-child{margin-top:0}
.exampleblock>.content>:last-child,.exampleblock>.content>:last-child>:last-child,.exampleblock>.content .olist>ol>li:last-child>:last-child,.exampleblock>.content .ulist>ul>li:last-child>:last-child,.exampleblock>.content .qlist>ol>li:last-child>:last-child,.sidebarblock>.content>:last-child,.sidebarblock>.content>:last-child>:last-child,.sidebarblock>.content .olist>ol>li:last-child>:last-child,.sidebarblock>.content .ulist>ul>li:last-child>:last-child,.sidebarblock>.content .qlist>ol>li:last-child>:last-child{margin-bottom:0}
.literalblock pre,.listingblock>.content>pre{border-radius:4px;overflow-x:auto;padding:1em;font-size:.8125em}
@media screen and (min-width:768px){.literalblock pre,.listingblock>.content>pre{font-size:.90625em}}
@media screen and (min-width:1280px){.literalblock pre,.listingblock>.content>pre{font-size:1em}}
.literalblock pre,.listingblock>.content>pre:not(.highlight),.listingblock>.content>pre[class=highlight],.listingblock>.content>pre[class^="highlight "]{background:#f7f7f8}
.literalblock.output pre{color:#f7f7f8;background:rgba(0,0,0,.9)}
.listingblock>.content{position:relative}
.listingblock code[data-lang]::before{display:none;content:attr(data-lang);position:absolute;font-size:.75em;top:.425rem;right:.5rem;line-height:1;text-transform:uppercase;color:inherit;opacity:.5}
.listingblock:hover code[data-lang]::before{display:block}
.listingblock.terminal pre .command::before{content:attr(data-prompt);padding-right:.5em;color:inherit;opacity:.5}
.listingblock.terminal pre .command:not([data-prompt])::before{content:"$"}
.listingblock pre.highlightjs{padding:0}
.listingblock pre.highlightjs>code{padding:1em;border-radius:4px}
.listingblock pre.prettyprint{border-width:0}
.prettyprint{background:#f7f7f8}
pre.prettyprint .linenums{line-height:1.45;margin-left:2em}
pre.prettyprint li{background:none;list-style-type:inherit;padding-left:0}
pre.prettyprint li code[data-lang]::before{opacity:1}
pre.prettyprint li:not(:first-child) code[data-lang]::before{display:none}
table.linenotable{border-collapse:separate;border:0;margin-bottom:0;background:none}
table.linenotable td[class]{color:inherit;vertical-align:top;padding:0;line-height:inherit;white-space:normal}
table.linenotable td.code{padding-left:.75em}
table.linenotable td.linenos,pre.pygments .linenos{border-right:1px solid;opacity:.35;padding-right:.5em;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}
pre.pygments span.linenos{display:inline-block;margin-right:.75em}
.quoteblock{margin:0 1em 1.25em 1.5em;display:table}
.quoteblock:not(.excerpt)>.title{margin-left:-1.5em;margin-bottom:.75em}
.quoteblock blockquote,.quoteblock p{color:rgba(0,0,0,.85);font-size:1.15rem;line-height:1.75;word-spacing:.1em;letter-spacing:0;font-style:italic;text-align:justify}
.quoteblock blockquote{margin:0;padding:0;border:0}
.quoteblock blockquote::before{content:"\201c";float:left;font-size:2.75em;font-weight:bold;line-height:.6em;margin-left:-.6em;color:#7a2518;text-shadow:0 1px 2px rgba(0,0,0,.1)}
.quoteblock blockquote>.paragraph:last-child p{margin-bottom:0}
.quoteblock .attribution{margin-top:.75em;margin-right:.5ex;text-align:right}
.verseblock{margin:0 1em 1.25em}
.verseblock pre{font-family:"Open Sans","DejaVu Sans",sans-serif;font-size:1.15rem;color:rgba(0,0,0,.85);font-weight:300;text-rendering:optimizeLegibility}
.verseblock pre strong{font-weight:400}
.verseblock .attribution{margin-top:1.25rem;margin-left:.5ex}
.quoteblock .attribution,.verseblock .attribution{font-size:.9375em;line-height:1.45;font-style:italic}
.quoteblock .attribution br,.verseblock .attribution br{display:none}
.quoteblock .attribution cite,.verseblock .attribution cite{display:block;letter-spacing:-.025em;color:rgba(0,0,0,.6)}
.quoteblock.abstract blockquote::before,.quoteblock.excerpt blockquote::before,.quoteblock .quoteblock blockquote::before{display:none}
.quoteblock.abstract blockquote,.quoteblock.abstract p,.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{line-height:1.6;word-spacing:0}
.quoteblock.abstract{margin:0 1em 1.25em;display:block}
.quoteblock.abstract>.title{margin:0 0 .375em;font-size:1.15em;text-align:center}
.quoteblock.excerpt>blockquote,.quoteblock .quoteblock{padding:0 0 .25em 1em;border-left:.25em solid #dddddf}
.quoteblock.excerpt,.quoteblock .quoteblock{margin-left:0}
.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{color:inherit;font-size:1.0625rem}
.quoteblock.excerpt .attribution,.quoteblock .quoteblock .attribution{color:inherit;font-size:.85rem;text-align:left;margin-right:0}
p.tableblock:last-child{margin-bottom:0}
td.tableblock>.content{margin-bottom:1.25em;word-wrap:anywhere}
td.tableblock>.content>:last-child{margin-bottom:-1.25em}
table.tableblock,th.tableblock,td.tableblock{border:0 solid #dedede}
table.grid-all>*>tr>*{border-width:1px}
table.grid-cols>*>tr>*{border-width:0 1px}
table.grid-rows>*>tr>*{border-width:1px 0}
table.frame-all{border-width:1px}
table.frame-ends{border-width:1px 0}
table.frame-sides{border-width:0 1px}
table.frame-none>colgroup+*>:first-child>*,table.frame-sides>colgroup+*>:first-child>*{border-top-width:0}
table.frame-none>:last-child>:last-child>*,table.frame-sides>:last-child>:last-child>*{border-bottom-width:0}
table.frame-none>*>tr>:first-child,table.frame-ends>*>tr>:first-child{border-left-width:0}
table.frame-none>*>tr>:last-child,table.frame-ends>*>tr>:last-child{border-right-width:0}
table.stripes-all>*>tr,table.stripes-odd>*>tr:nth-of-type(odd),table.stripes-even>*>tr:nth-of-type(even),table.stripes-hover>*>tr:hover{background:#f8f8f7}
th.halign-left,td.halign-left{text-align:left}
th.halign-right,td.halign-right{text-align:right}
th.halign-center,td.halign-center{text-align:center}
th.valign-top,td.valign-top{vertical-align:top}
th.valign-bottom,td.valign-bottom{vertical-align:bottom}
th.valign-middle,td.valign-middle{vertical-align:middle}
table thead th,table tfoot th{font-weight:bold}
tbody tr th{background:#f7f8f7}
tbody tr th,tbody tr th p,tfoot tr th,tfoot tr th p{color:rgba(0,0,0,.8);font-weight:bold}
p.tableblock>code:only-child{background:none;padding:0}
p.tableblock{font-size:1em}
ol{margin-left:1.75em}
ul li ol{margin-left:1.5em}
dl dd{margin-left:1.125em}
dl dd:last-child,dl dd:last-child>:last-child{margin-bottom:0}
li p,ul dd,ol dd,.olist .olist,.ulist .ulist,.ulist .olist,.olist .ulist{margin-bottom:.625em}
ul.checklist,ul.none,ol.none,ul.no-bullet,ol.no-bullet,ol.unnumbered,ul.unstyled,ol.unstyled{list-style-type:none}
ul.no-bullet,ol.no-bullet,ol.unnumbered{margin-left:.625em}
ul.unstyled,ol.unstyled{margin-left:0}
li>p:empty:only-child::before{content:"";display:inline-block}
ul.checklist>li>p:first-child{margin-left:-1em}
ul.checklist>li>p:first-child>.fa-square-o:first-child,ul.checklist>li>p:first-child>.fa-check-square-o:first-child{width:1.25em;font-size:.8em;position:relative;bottom:.125em}
ul.checklist>li>p:first-child>input[type=checkbox]:first-child{margin-right:.25em}
ul.inline{display:flex;flex-flow:row wrap;list-style:none;margin:0 0 .625em -1.25em}
ul.inline>li{margin-left:1.25em}
.unstyled dl dt{font-weight:400;font-style:normal}
ol.arabic{list-style-type:decimal}
ol.decimal{list-style-type:decimal-leading-zero}
ol.loweralpha{list-style-type:lower-alpha}
ol.upperalpha{list-style-type:upper-alpha}
ol.lowerroman{list-style-type:lower-roman}
ol.upperroman{list-style-type:upper-roman}
ol.lowergreek{list-style-type:lower-greek}
.hdlist>table,.colist>table{border:0;background:none}
.hdlist>table>tbody>tr,.colist>table>tbody>tr{background:none}
td.hdlist1,td.hdlist2{vertical-align:top;padding:0 .625em}
td.hdlist1{font-weight:bold;padding-bottom:1.25em}
td.hdlist2{word-wrap:anywhere}
.literalblock+.colist,.listingblock+.colist{margin-top:-.5em}
.colist td:not([class]):first-child{padding:.4em .75em 0;line-height:1;vertical-align:top}
.colist td:not([class]):first-child img{max-width:none}
.colist td:not([class]):last-child{padding:.25em 0}
.thumb,.th{line-height:0;display:inline-block;border:4px solid #fff;box-shadow:0 0 0 1px #ddd}
.imageblock.left{margin:.25em .625em 1.25em 0}
.imageblock.right{margin:.25em 0 1.25em .625em}
.imageblock>.title{margin-bottom:0}
.imageblock.thumb,.imageblock.th{border-width:6px}
.imageblock.thumb>.title,.imageblock.th>.title{padding:0 .125em}
.image.left,.image.right{margin-top:.25em;margin-bottom:.25em;display:inline-block;line-height:0}
.image.left{margin-right:.625em}
.image.right{margin-left:.625em}
a.image{text-decoration:none;display:inline-block}
a.image object{pointer-events:none}
sup.footnote,sup.footnoteref{font-size:.875em;position:static;vertical-align:super}
sup.footnote a,sup.footnoteref a{text-decoration:none}
sup.footnote a:active,sup.footnoteref a:active,#footnotes .footnote a:first-of-type:active{text-decoration:underline}
#footnotes{padding-top:.75em;padding-bottom:.75em;margin-bottom:.625em}
#footnotes hr{width:20%;min-width:6.25em;margin:-.25em 0 .75em;border-width:1px 0 0}
#footnotes .footnote{padding:0 .375em 0 .225em;line-height:1.3334;font-size:.875em;margin-left:1.2em;margin-bottom:.2em}
#footnotes .footnote a:first-of-type{font-weight:bold;text-decoration:none;margin-left:-1.05em}
#footnotes .footnote:last-of-type{margin-bottom:0}
#content #footnotes{margin-top:-.625em;margin-bottom:0;padding:.75em 0}
div.unbreakable{page-break-inside:avoid}
.big{font-size:larger}
.small{font-size:smaller}
.underline{text-decoration:underline}
.overline{text-decoration:overline}
.line-through{text-decoration:line-through}
.aqua{color:#00bfbf}
.aqua-background{background:#00fafa}
.black{color:#000}
.black-background{background:#000}
.blue{color:#0000bf}
.blue-background{background:#0000fa}
.fuchsia{color:#bf00bf}
.fuchsia-background{background:#fa00fa}
.gray{color:#606060}
.gray-background{background:#7d7d7d}
.green{color:#006000}
.green-background{background:#007d00}
.lime{color:#00bf00}
.lime-background{background:#00fa00}
.maroon{color:#600000}
.maroon-background{background:#7d0000}
.navy{color:#000060}
.navy-background{background:#00007d}
.olive{color:#606000}
.olive-background{background:#7d7d00}
.purple{color:#600060}
.purple-background{background:#7d007d}
.red{color:#bf0000}
.red-background{background:#fa0000}
.silver{color:#909090}
.silver-background{background:#bcbcbc}
.teal{color:#006060}
.teal-background{background:#007d7d}
.white{color:#bfbfbf}
.white-background{background:#fafafa}
.yellow{color:#bfbf00}
.yellow-background{background:#fafa00}
span.icon>.fa{cursor:default}
a span.icon>.fa{cursor:inherit}
.admonitionblock td.icon [class^="fa icon-"]{font-size:2.5em;text-shadow:1px 1px 2px rgba(0,0,0,.5);cursor:default}
.admonitionblock td.icon .icon-note::before{content:"\f05a";color:#19407c}
.admonitionblock td.icon .icon-tip::before{content:"\f0eb";text-shadow:1px 1px 2px rgba(155,155,0,.8);color:#111}
.admonitionblock td.icon .icon-warning::before{content:"\f071";color:#bf6900}
.admonitionblock td.icon .icon-caution::before{content:"\f06d";color:#bf3400}
.admonitionblock td.icon .icon-important::before{content:"\f06a";color:#bf0000}
.conum[data-value]{display:inline-block;color:#fff!important;background:rgba(0,0,0,.8);border-radius:50%;text-align:center;font-size:.75em;width:1.67em;height:1.67em;line-height:1.67em;font-family:"Open Sans","DejaVu Sans",sans-serif;font-style:normal;font-weight:bold}
.conum[data-value] *{color:#fff!important}
.conum[data-value]+b{display:none}
.conum[data-value]::after{content:attr(data-value)}
pre .conum[data-value]{position:relative;top:-.125em}
b.conum *{color:inherit!important}
.conum:not([data-value]):empty{display:none}
dt,th.tableblock,td.content,div.footnote{text-rendering:optimizeLegibility}
h1,h2,p,td.content,span.alt,summary{letter-spacing:-.01em}
p strong,td.content strong,div.footnote strong{letter-spacing:-.005em}
p,blockquote,dt,td.content,td.hdlist1,span.alt,summary{font-size:1.0625rem}
p{margin-bottom:1.25rem}
.sidebarblock p,.sidebarblock dt,.sidebarblock td.content,p.tableblock{font-size:1em}
.exampleblock>.content{background:#fffef7;border-color:#e0e0dc;box-shadow:0 1px 4px #e0e0dc}
.print-only{display:none!important}
@page{margin:1.25cm .75cm}
@media print{*{box-shadow:none!important;text-shadow:none!important}
html{font-size:80%}
a{color:inherit!important;text-decoration:underline!important}
a.bare,a[href^="#"],a[href^="mailto:"]{text-decoration:none!important}
a[href^="http:"]:not(.bare)::after,a[href^="https:"]:not(.bare)::after{content:"(" attr(href) ")";display:inline-block;font-size:.875em;padding-left:.25em}
abbr[title]{border-bottom:1px dotted}
abbr[title]::after{content:" (" attr(title) ")"}
pre,blockquote,tr,img,object,svg{page-break-inside:avoid}
thead{display:table-header-group}
svg{max-width:100%}
p,blockquote,dt,td.content{font-size:1em;orphans:3;widows:3}
h2,h3,#toctitle,.sidebarblock>.content>.title{page-break-after:avoid}
#header,#content,#footnotes,#footer{max-width:none}
#toc,.sidebarblock,.exampleblock>.content{background:none!important}
#toc{border-bottom:1px solid #dddddf!important;padding-bottom:0!important}
body.book #header{text-align:center}
body.book #header>h1:first-child{border:0!important;margin:2.5em 0 1em}
body.book #header .details{border:0!important;display:block;padding:0!important}
body.book #header .details span:first-child{margin-left:0!important}
body.book #header .details br{display:block}
body.book #header .details br+span::before{content:none!important}
body.book #toc{border:0!important;text-align:left!important;padding:0!important;margin:0!important}
body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-break-before:always}
.listingblock code[data-lang]::before{display:block}
#footer{padding:0 .9375em}
.hide-on-print{display:none!important}
.print-only{display:block!important}
.hide-for-print{display:none!important}
.show-for-print{display:inherit!important}}
@media amzn-kf8,print{#header>h1:first-child{margin-top:1.25rem}
.sect1{padding:0!important}
.sect1+.sect1{border:0}
#footer{background:none}
#footer-text{color:rgba(0,0,0,.6);font-size:.9em}}
@media amzn-kf8{#header,#content,#footnotes,#footer{padding:0}}
</style>
</head>
<body class="article">
<div id="header">
<h1>Parallel Checkout Design Notes</h1>
</div>
<div id="content">
<div id="preamble">
<div class="sectionbody">
<div class="paragraph">
<p>The "Parallel Checkout" feature attempts to use multiple processes to
parallelize the work of uncompressing the blobs, applying in-core
filters, and writing the resulting contents to the working tree during a
checkout operation. It can be used by all checkout-related commands,
such as <code>clone</code>, <code>checkout</code>, <code>reset</code>, <code>sparse-checkout</code>, and others.</p>
</div>
<div class="paragraph">
<p>These commands share the following basic structure:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Step 1: Read the current index file into memory.</p>
</li>
<li>
<p>Step 2: Modify the in-memory index based upon the command, and
temporarily mark all cache entries that need to be updated.</p>
</li>
<li>
<p>Step 3: Populate the working tree to match the new candidate index.
This includes iterating over all of the to-be-updated cache entries
and delete, create, or overwrite the associated files in the working
tree.</p>
</li>
<li>
<p>Step 4: Write the new index to disk.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Step 3 is the focus of the "parallel checkout" effort described here.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_sequential_implementation">Sequential Implementation</h2>
<div class="sectionbody">
<div class="paragraph">
<p>For the purposes of discussion here, the current sequential
implementation of Step 3 is divided in 3 parts, each one implemented in
its own function:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Step 3a: <code>unpack-trees.c:check_updates</code>() contains a series of
sequential loops iterating over the <code>cache_entry</code>'s array. The main
loop in this function calls the Step 3b function for each of the
to-be-updated entries.</p>
</li>
<li>
<p>Step 3b: <code>entry.c:checkout_entry</code>() examines the existing working tree
for file conflicts, collisions, and unsaved changes. It removes files
and creates leading directories as necessary. It calls the Step 3c
function for each entry to be written.</p>
</li>
<li>
<p>Step 3c: <code>entry.c:write_entry</code>() loads the blob into memory, smudges
it if necessary, creates the file in the working tree, writes the
smudged contents, calls <code>fstat</code>() or <code>lstat</code>(), and updates the
associated <code>cache_entry</code> struct with the stat information gathered.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>It wouldn&#8217;t be safe to perform Step 3b in parallel, as there could be
race conditions between file creations and removals. Instead, the
parallel checkout framework lets the sequential code handle Step 3b,
and uses parallel workers to replace the sequential
<code>entry.c:write_entry</code>() calls from Step 3c.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_rejected_multi_threaded_solution">Rejected Multi-Threaded Solution</h2>
<div class="sectionbody">
<div class="paragraph">
<p>The most "straightforward" implementation would be to spread the set of
to-be-updated cache entries across multiple threads. But due to the
thread-unsafe functions in the object database code, we would have to use locks to
coordinate the parallel operation. An early prototype of this solution
showed that the multi-threaded checkout would bring performance
improvements over the sequential code, but there was still too much lock
contention. A <code>perf</code> profiling indicated that around 20% of the runtime
during a local Linux clone (on an SSD) was spent in locking functions.
For this reason this approach was rejected in favor of using multiple
child processes, which led to better performance.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_multi_process_solution">Multi-Process Solution</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Parallel checkout alters the aforementioned Step 3 to use multiple
<code>checkout--worker</code> background processes to distribute the work. The
long-running worker processes are controlled by the foreground Git
command using the existing run-command API.</p>
</div>
<div class="sect2">
<h3 id="_overview">Overview</h3>
<div class="paragraph">
<p>Step 3b is only slightly altered; for each entry to be checked out, the
main process performs the following steps:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>M1: Check whether there is any untracked or unclean file in the
working tree which would be overwritten by this entry, and decide
whether to proceed (removing the file(s)) or not.</p>
</li>
<li>
<p>M2: Create the leading directories.</p>
</li>
<li>
<p>M3: Load the conversion attributes for the entry&#8217;s path.</p>
</li>
<li>
<p>M4: Check, based on the entry&#8217;s type and conversion attributes,
whether the entry is eligible for parallel checkout (more on this
later). If it is eligible, enqueue the entry and the loaded
attributes to later write the entry in parallel. If not, write the
entry right away, using the default sequential code.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Note: we save the conversion attributes associated with each entry
because the workers don&#8217;t have access to the main process' index state,
so they can&#8217;t load the attributes by themselves (and the attributes are
needed to properly smudge the entry). Additionally, this has a positive
impact on performance as (1) we don&#8217;t need to load the attributes twice
and (2) the attributes machinery is optimized to handle paths in
sequential order.</p>
</div>
<div class="paragraph">
<p>After all entries have passed through the above steps, the main process
checks if the number of enqueued entries is sufficient to spread among
the workers. If not, it just writes them sequentially. Otherwise, it
spawns the workers and distributes the queued entries uniformly in
continuous chunks. This aims to minimize the chances of two workers
writing to the same directory simultaneously, which could increase lock
contention in the kernel.</p>
</div>
<div class="paragraph">
<p>Then, for each assigned item, each worker:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>W1: Checks if there is any non-directory file in the leading part of
the entry&#8217;s path or if there already exists a file at the entry' path.
If so, mark the entry with <code>PC_ITEM_COLLIDED</code> and skip it (more on
this later).</p>
</li>
<li>
<p>W2: Creates the file (with O_CREAT and O_EXCL).</p>
</li>
<li>
<p>W3: Loads the blob into memory (inflating and delta reconstructing
it).</p>
</li>
<li>
<p>W4: Applies any required in-process filter, like end-of-line
conversion and re-encoding.</p>
</li>
<li>
<p>W5: Writes the result to the file descriptor opened at W2.</p>
</li>
<li>
<p>W6: Calls <code>fstat</code>() or <code>lstat</code>() on the just-written path, and sends
the result back to the main process, together with the end status of
the operation and the item&#8217;s identification number.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Note that, when possible, steps W3 to W5 are delegated to the streaming
machinery, removing the need to keep the entire blob in memory.</p>
</div>
<div class="paragraph">
<p>If the worker fails to read the blob or to write it to the working tree,
it removes the created file to avoid leaving empty files behind. This is
the <strong>only</strong> time a worker is allowed to remove a file.</p>
</div>
<div class="paragraph">
<p>As mentioned earlier, it is the responsibility of the main process to
remove any file that blocks the checkout operation (or abort if the
removal(s) would cause data loss and the user didn&#8217;t ask to <code>--force</code>).
This is crucial to avoid race conditions and also to properly detect
path collisions at Step W1.</p>
</div>
<div class="paragraph">
<p>After the workers finish writing the items and sending back the required
information, the main process handles the results in two steps:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>First, it updates the in-memory index with the <code>lstat</code>() information
sent by the workers. (This must be done first as this information
might be required in the following step.)</p>
</li>
<li>
<p>Then it writes the items which collided on disk (i.e. items marked
with <code>PC_ITEM_COLLIDED</code>). More on this below.</p>
</li>
</ul>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_path_collisions">Path Collisions</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Path collisions happen when two different paths correspond to the same
entry in the file system. E.g. the paths <em>a</em> and <em>A</em> would collide in a
case-insensitive file system.</p>
</div>
<div class="paragraph">
<p>The sequential checkout deals with collisions in the same way that it
deals with files that were already present in the working tree before
checkout. Basically, it checks if the path that it wants to write
already exists on disk, makes sure the existing file doesn&#8217;t have
unsaved data, and then overwrites it. (To be more pedantic: it deletes
the existing file and creates the new one.) So, if there are multiple
colliding files to be checked out, the sequential code will write each
one of them but only the last will actually survive on disk.</p>
</div>
<div class="paragraph">
<p>Parallel checkout aims to reproduce the same behavior. However, we
cannot let the workers racily write to the same file on disk. Instead,
the workers detect when the entry that they want to check out would
collide with an existing file, and mark it with <code>PC_ITEM_COLLIDED</code>.
Later, the main process can sequentially feed these entries back to
<code>checkout_entry</code>() without the risk of race conditions. On clone, this
also has the effect of marking the colliding entries to later emit a
warning for the user, like the classic sequential checkout does.</p>
</div>
<div class="paragraph">
<p>The workers are able to detect both collisions among the entries being
concurrently written and collisions between a parallel-eligible entry
and an ineligible entry. The general idea for collision detection is
quite straightforward: for each parallel-eligible entry, the main
process must remove all files that prevent this entry from being written
(before enqueueing it). This includes any non-directory file in the
leading path of the entry. Later, when a worker gets assigned the entry,
it looks again for the non-directory files and for an already existing
file at the entry&#8217;s path. If any of these checks finds something, the
worker knows that there was a path collision.</p>
</div>
<div class="paragraph">
<p>Because parallel checkout can distinguish path collisions from the case
where the file was already present in the working tree before checkout,
we could alternatively choose to skip the checkout of colliding entries.
However, each entry that doesn&#8217;t get written would have NULL <code>lstat</code>()
fields on the index. This could cause performance penalties for
subsequent commands that need to refresh the index, as they would have
to go to the file system to see if the entry is dirty. Thus, if we have
N entries in a colliding group and we decide to write and <code>lstat</code>() only
one of them, every subsequent <code>git-status</code> will have to read, convert,
and hash the written file N - 1 times. By checking out all colliding
entries (like the sequential code does), we only pay the overhead once,
during checkout.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_eligible_entries_for_parallel_checkout">Eligible Entries for Parallel Checkout</h2>
<div class="sectionbody">
<div class="paragraph">
<p>As previously mentioned, not all entries passed to <code>checkout_entry</code>()
will be considered eligible for parallel checkout. More specifically, we
exclude:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Symbolic links; to avoid race conditions that, in combination with
path collisions, could cause workers to write files at the wrong
place. For example, if we were to concurrently check out a symlink
<em>a</em> &#8594; <em>b</em> and a regular file <em>A/f</em> in a case-insensitive file system,
we could potentially end up writing the file <em>A/f</em> at <em>a/f</em>, due to a
race condition.</p>
</li>
<li>
<p>Regular files that require external filters (either "one shot" filters
or long-running process filters). These filters are black-boxes to Git
and may have their own internal locking or non-concurrent assumptions.
So it might not be safe to run multiple instances in parallel.</p>
<div class="paragraph">
<p>Besides, long-running filters may use the delayed checkout feature to
postpone the return of some filtered blobs. The delayed checkout queue
and the parallel checkout queue are not compatible and should remain
separate.</p>
</div>
<div class="paragraph">
<p>Note: regular files that only require internal filters, like end-of-line
conversion and re-encoding, are eligible for parallel checkout.</p>
</div>
</li>
</ul>
</div>
<div class="paragraph">
<p>Ineligible entries are checked out by the classic sequential codepath
<strong>before</strong> spawning workers.</p>
</div>
<div class="paragraph">
<p>Note: submodules' files are also eligible for parallel checkout (as
long as they don&#8217;t fall into any of the excluding categories mentioned
above). But since each submodule is checked out in its own child
process, we don&#8217;t mix the superproject&#8217;s and the submodules' files in
the same parallel checkout process or queue.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_the_api">The API</h2>
<div class="sectionbody">
<div class="paragraph">
<p>The parallel checkout API was designed with the goal of minimizing
changes to the current users of the checkout machinery. This means that
they don&#8217;t have to call a different function for sequential or parallel
checkout. As already mentioned, <code>checkout_entry</code>() will automatically
insert the given entry in the parallel checkout queue when this feature
is enabled and the entry is eligible; otherwise, it will just write the
entry right away, using the sequential code. In general, callers of the
parallel checkout API should look similar to this:</p>
</div>
<div class="listingblock">
<div class="content">
<pre>int pc_workers, pc_threshold, err = 0;
struct checkout state;
get_parallel_checkout_configs(&amp;pc_workers, &amp;pc_threshold);
/*
* This check is not strictly required, but it
* should save some time in sequential mode.
*/
if (pc_workers &gt; 1)
init_parallel_checkout();
for (each cache_entry ce to-be-updated)
err |= checkout_entry(ce, &amp;state, NULL, NULL);
err |= run_parallel_checkout(&amp;state, pc_workers, pc_threshold, NULL, NULL);</pre>
</div>
</div>
</div>
</div>
</div>
<div id="footer">
<div id="footer-text">
Last updated 2025-03-17 07:09:38 UTC
</div>
</div>
</body>
</html>

View File

@@ -0,0 +1,367 @@
Partial Clone Design Notes
==========================
The "Partial Clone" feature is a performance optimization for Git that
allows Git to function without having a complete copy of the repository.
The goal of this work is to allow Git to better handle extremely large
repositories.
During clone and fetch operations, Git downloads the complete contents
and history of the repository. This includes all commits, trees, and
blobs for the complete life of the repository. For extremely large
repositories, clones can take hours (or days) and consume 100+GiB of disk
space.
Often in these repositories there are many blobs and trees that the user
does not need such as:
1. files outside of the user's work area in the tree. For example, in
a repository with 500K directories and 3.5M files in every commit,
we can avoid downloading many objects if the user only needs a
narrow "cone" of the source tree.
2. large binary assets. For example, in a repository where large build
artifacts are checked into the tree, we can avoid downloading all
previous versions of these non-mergeable binary assets and only
download versions that are actually referenced.
Partial clone allows us to avoid downloading such unneeded objects *in
advance* during clone and fetch operations and thereby reduce download
times and disk usage. Missing objects can later be "demand fetched"
if/when needed.
A remote that can later provide the missing objects is called a
promisor remote, as it promises to send the objects when
requested. Initially Git supported only one promisor remote, the origin
remote from which the user cloned and that was configured in the
"extensions.partialClone" config option. Later support for more than
one promisor remote has been implemented.
Use of partial clone requires that the user be online and the origin
remote or other promisor remotes be available for on-demand fetching
of missing objects. This may or may not be problematic for the user.
For example, if the user can stay within the pre-selected subset of
the source tree, they may not encounter any missing objects.
Alternatively, the user could try to pre-fetch various objects if they
know that they are going offline.
Non-Goals
---------
Partial clone is a mechanism to limit the number of blobs and trees downloaded
*within* a given range of commits -- and is therefore independent of and not
intended to conflict with existing DAG-level mechanisms to limit the set of
requested commits (i.e. shallow clone, single branch, or fetch '<refspec>').
Design Overview
---------------
Partial clone logically consists of the following parts:
- A mechanism for the client to describe unneeded or unwanted objects to
the server.
- A mechanism for the server to omit such unwanted objects from packfiles
sent to the client.
- A mechanism for the client to gracefully handle missing objects (that
were previously omitted by the server).
- A mechanism for the client to backfill missing objects as needed.
Design Details
--------------
- A new pack-protocol capability "filter" is added to the fetch-pack and
upload-pack negotiation.
+
This uses the existing capability discovery mechanism.
See "filter" in linkgit:gitprotocol-pack[5].
- Clients pass a "filter-spec" to clone and fetch which is passed to the
server to request filtering during packfile construction.
+
There are various filters available to accommodate different situations.
See "--filter=<filter-spec>" in Documentation/rev-list-options.adoc.
- On the server pack-objects applies the requested filter-spec as it
creates "filtered" packfiles for the client.
+
These filtered packfiles are *incomplete* in the traditional sense because
they may contain objects that reference objects not contained in the
packfile and that the client doesn't already have. For example, the
filtered packfile may contain trees or tags that reference missing blobs
or commits that reference missing trees.
- On the client these incomplete packfiles are marked as "promisor packfiles"
and treated differently by various commands.
- On the client a repository extension is added to the local config to
prevent older versions of git from failing mid-operation because of
missing objects that they cannot handle.
See `extensions.partialClone` in linkgit:git-config[1].
Handling Missing Objects
------------------------
- An object may be missing due to a partial clone or fetch, or missing
due to repository corruption. To differentiate these cases, the
local repository specially indicates such filtered packfiles
obtained from promisor remotes as "promisor packfiles".
+
These promisor packfiles consist of a "<name>.promisor" file with
arbitrary contents (like the "<name>.keep" files), in addition to
their "<name>.pack" and "<name>.idx" files.
- The local repository considers a "promisor object" to be an object that
it knows (to the best of its ability) that promisor remotes have promised
that they have, either because the local repository has that object in one of
its promisor packfiles, or because another promisor object refers to it.
+
When Git encounters a missing object, Git can see if it is a promisor object
and handle it appropriately. If not, Git can report a corruption.
+
This means that there is no need for the client to explicitly maintain an
expensive-to-modify list of missing objects.[a]
- Since almost all Git code currently expects any referenced object to be
present locally and because we do not want to force every command to do
a dry-run first, a fallback mechanism is added to allow Git to attempt
to dynamically fetch missing objects from promisor remotes.
+
When the normal object lookup fails to find an object, Git invokes
promisor_remote_get_direct() to try to get the object from a promisor
remote and then retry the object lookup. This allows objects to be
"faulted in" without complicated prediction algorithms.
+
For efficiency reasons, no check as to whether the missing object is
actually a promisor object is performed.
+
Dynamic object fetching tends to be slow as objects are fetched one at
a time.
- `checkout` (and any other command using `unpack-trees`) has been taught
to bulk pre-fetch all required missing blobs in a single batch.
- `rev-list` has been taught to print missing objects.
+
This can be used by other commands to bulk prefetch objects.
For example, a "git log -p A..B" may internally want to first do
something like "git rev-list --objects --quiet --missing=print A..B"
and prefetch those objects in bulk.
- `fsck` has been updated to be fully aware of promisor objects.
- `repack` in GC has been updated to not touch promisor packfiles at all,
and to only repack other objects.
- The global variable "fetch_if_missing" is used to control whether an
object lookup will attempt to dynamically fetch a missing object or
report an error.
+
We are not happy with this global variable and would like to remove it,
but that requires significant refactoring of the object code to pass an
additional flag.
Fetching Missing Objects
------------------------
- Fetching of objects is done by invoking a "git fetch" subprocess.
- The local repository sends a request with the hashes of all requested
objects, and does not perform any packfile negotiation.
It then receives a packfile.
- Because we are reusing the existing fetch mechanism, fetching
currently fetches all objects referred to by the requested objects, even
though they are not necessary.
- Fetching with `--refetch` will request a complete new filtered packfile from
the remote, which can be used to change a filter without needing to
dynamically fetch missing objects.
Using many promisor remotes
---------------------------
Many promisor remotes can be configured and used.
This allows for example a user to have multiple geographically-close
cache servers for fetching missing blobs while continuing to do
filtered `git-fetch` commands from the central server.
When fetching objects, promisor remotes are tried one after the other
until all the objects have been fetched.
Remotes that are considered "promisor" remotes are those specified by
the following configuration variables:
- `extensions.partialClone = <name>`
- `remote.<name>.promisor = true`
- `remote.<name>.partialCloneFilter = ...`
Only one promisor remote can be configured using the
`extensions.partialClone` config variable. This promisor remote will
be the last one tried when fetching objects.
We decided to make it the last one we try, because it is likely that
someone using many promisor remotes is doing so because the other
promisor remotes are better for some reason (maybe they are closer or
faster for some kind of objects) than the origin, and the origin is
likely to be the remote specified by extensions.partialClone.
This justification is not very strong, but one choice had to be made,
and anyway the long term plan should be to make the order somehow
fully configurable.
For now though the other promisor remotes will be tried in the order
they appear in the config file.
Current Limitations
-------------------
- It is not possible to specify the order in which the promisor
remotes are tried in other ways than the order in which they appear
in the config file.
+
It is also not possible to specify an order to be used when fetching
from one remote and a different order when fetching from another
remote.
- It is not possible to push only specific objects to a promisor
remote.
+
It is not possible to push at the same time to multiple promisor
remote in a specific order.
- Dynamic object fetching will only ask promisor remotes for missing
objects. We assume that promisor remotes have a complete view of the
repository and can satisfy all such requests.
- Repack essentially treats promisor and non-promisor packfiles as 2
distinct partitions and does not mix them.
- Dynamic object fetching invokes fetch-pack once *for each item*
because most algorithms stumble upon a missing object and need to have
it resolved before continuing their work. This may incur significant
overhead -- and multiple authentication requests -- if many objects are
needed.
- Dynamic object fetching currently uses the existing pack protocol V0
which means that each object is requested via fetch-pack. The server
will send a full set of info/refs when the connection is established.
If there are a large number of refs, this may incur significant overhead.
Future Work
-----------
- Improve the way to specify the order in which promisor remotes are
tried.
+
For example this could allow specifying explicitly something like:
"When fetching from this remote, I want to use these promisor remotes
in this order, though, when pushing or fetching to that remote, I want
to use those promisor remotes in that order."
- Allow pushing to promisor remotes.
+
The user might want to work in a triangular work flow with multiple
promisor remotes that each have an incomplete view of the repository.
- Allow non-pathname-based filters to make use of packfile bitmaps (when
present). This was just an omission during the initial implementation.
- Investigate use of a long-running process to dynamically fetch a series
of objects, such as proposed in [5,6] to reduce process startup and
overhead costs.
+
It would be nice if pack protocol V2 could allow that long-running
process to make a series of requests over a single long-running
connection.
- Investigate pack protocol V2 to avoid the info/refs broadcast on
each connection with the server to dynamically fetch missing objects.
- Investigate the need to handle loose promisor objects.
+
Objects in promisor packfiles are allowed to reference missing objects
that can be dynamically fetched from the server. An assumption was
made that loose objects are only created locally and therefore should
not reference a missing object. We may need to revisit that assumption
if, for example, we dynamically fetch a missing tree and store it as a
loose object rather than a single object packfile.
+
This does not necessarily mean we need to mark loose objects as promisor;
it may be sufficient to relax the object lookup or is-promisor functions.
Non-Tasks
---------
- Every time the subject of "demand loading blobs" comes up it seems
that someone suggests that the server be allowed to "guess" and send
additional objects that may be related to the requested objects.
+
No work has gone into actually doing that; we're just documenting that
it is a common suggestion. We're not sure how it would work and have
no plans to work on it.
+
It is valid for the server to send more objects than requested (even
for a dynamic object fetch), but we are not building on that.
Footnotes
---------
[a] expensive-to-modify list of missing objects: Earlier in the design of
partial clone we discussed the need for a single list of missing objects.
This would essentially be a sorted linear list of OIDs that were
omitted by the server during a clone or subsequent fetches.
This file would need to be loaded into memory on every object lookup.
It would need to be read, updated, and re-written (like the .git/index)
on every explicit "git fetch" command *and* on any dynamic object fetch.
The cost to read, update, and write this file could add significant
overhead to every command if there are many missing objects. For example,
if there are 100M missing blobs, this file would be at least 2GiB on disk.
With the "promisor" concept, we *infer* a missing object based upon the
type of packfile that references it.
Related Links
-------------
[0] https://crbug.com/git/2
Bug#2: Partial Clone
[1] https://lore.kernel.org/git/20170113155253.1644-1-benpeart@microsoft.com/ +
Subject: [RFC] Add support for downloading blobs on demand +
Date: Fri, 13 Jan 2017 10:52:53 -0500
[2] https://lore.kernel.org/git/cover.1506714999.git.jonathantanmy@google.com/ +
Subject: [PATCH 00/18] Partial clone (from clone to lazy fetch in 18 patches) +
Date: Fri, 29 Sep 2017 13:11:36 -0700
[3] https://lore.kernel.org/git/20170426221346.25337-1-jonathantanmy@google.com/ +
Subject: Proposal for missing blob support in Git repos +
Date: Wed, 26 Apr 2017 15:13:46 -0700
[4] https://lore.kernel.org/git/1488999039-37631-1-git-send-email-git@jeffhostetler.com/ +
Subject: [PATCH 00/10] RFC Partial Clone and Fetch +
Date: Wed, 8 Mar 2017 18:50:29 +0000
[5] https://lore.kernel.org/git/20170505152802.6724-1-benpeart@microsoft.com/ +
Subject: [PATCH v7 00/10] refactor the filter process code into a reusable module +
Date: Fri, 5 May 2017 11:27:52 -0400
[6] https://lore.kernel.org/git/20170714132651.170708-1-benpeart@microsoft.com/ +
Subject: [RFC/PATCH v2 0/1] Add support for downloading blobs on demand +
Date: Fri, 14 Jul 2017 09:26:50 -0400

View File

@@ -0,0 +1,954 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="en">
<head>
<meta charset="UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=edge"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<meta name="generator" content="Asciidoctor 2.0.23"/>
<title>Partial Clone Design Notes</title>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"/>
<style>
/*! Asciidoctor default stylesheet | MIT License | https://asciidoctor.org */
/* Uncomment the following line when using as a custom stylesheet */
/* @import "https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"; */
html{font-family:sans-serif;-webkit-text-size-adjust:100%}
a{background:none}
a:focus{outline:thin dotted}
a:active,a:hover{outline:0}
h1{font-size:2em;margin:.67em 0}
b,strong{font-weight:bold}
abbr{font-size:.9em}
abbr[title]{cursor:help;border-bottom:1px dotted #dddddf;text-decoration:none}
dfn{font-style:italic}
hr{height:0}
mark{background:#ff0;color:#000}
code,kbd,pre,samp{font-family:monospace;font-size:1em}
pre{white-space:pre-wrap}
q{quotes:"\201C" "\201D" "\2018" "\2019"}
small{font-size:80%}
sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline}
sup{top:-.5em}
sub{bottom:-.25em}
img{border:0}
svg:not(:root){overflow:hidden}
figure{margin:0}
audio,video{display:inline-block}
audio:not([controls]){display:none;height:0}
fieldset{border:1px solid silver;margin:0 2px;padding:.35em .625em .75em}
legend{border:0;padding:0}
button,input,select,textarea{font-family:inherit;font-size:100%;margin:0}
button,input{line-height:normal}
button,select{text-transform:none}
button,html input[type=button],input[type=reset],input[type=submit]{-webkit-appearance:button;cursor:pointer}
button[disabled],html input[disabled]{cursor:default}
input[type=checkbox],input[type=radio]{padding:0}
button::-moz-focus-inner,input::-moz-focus-inner{border:0;padding:0}
textarea{overflow:auto;vertical-align:top}
table{border-collapse:collapse;border-spacing:0}
*,::before,::after{box-sizing:border-box}
html,body{font-size:100%}
body{background:#fff;color:rgba(0,0,0,.8);padding:0;margin:0;font-family:"Noto Serif","DejaVu Serif",serif;line-height:1;position:relative;cursor:auto;-moz-tab-size:4;-o-tab-size:4;tab-size:4;word-wrap:anywhere;-moz-osx-font-smoothing:grayscale;-webkit-font-smoothing:antialiased}
a:hover{cursor:pointer}
img,object,embed{max-width:100%;height:auto}
object,embed{height:100%}
img{-ms-interpolation-mode:bicubic}
.left{float:left!important}
.right{float:right!important}
.text-left{text-align:left!important}
.text-right{text-align:right!important}
.text-center{text-align:center!important}
.text-justify{text-align:justify!important}
.hide{display:none}
img,object,svg{display:inline-block;vertical-align:middle}
textarea{height:auto;min-height:50px}
select{width:100%}
.subheader,.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{line-height:1.45;color:#7a2518;font-weight:400;margin-top:0;margin-bottom:.25em}
div,dl,dt,dd,ul,ol,li,h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6,pre,form,p,blockquote,th,td{margin:0;padding:0}
a{color:#2156a5;text-decoration:underline;line-height:inherit}
a:hover,a:focus{color:#1d4b8f}
a img{border:0}
p{line-height:1.6;margin-bottom:1.25em;text-rendering:optimizeLegibility}
p aside{font-size:.875em;line-height:1.35;font-style:italic}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{font-family:"Open Sans","DejaVu Sans",sans-serif;font-weight:300;font-style:normal;color:#ba3925;text-rendering:optimizeLegibility;margin-top:1em;margin-bottom:.5em;line-height:1.0125em}
h1 small,h2 small,h3 small,#toctitle small,.sidebarblock>.content>.title small,h4 small,h5 small,h6 small{font-size:60%;color:#e99b8f;line-height:0}
h1{font-size:2.125em}
h2{font-size:1.6875em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.375em}
h4,h5{font-size:1.125em}
h6{font-size:1em}
hr{border:solid #dddddf;border-width:1px 0 0;clear:both;margin:1.25em 0 1.1875em}
em,i{font-style:italic;line-height:inherit}
strong,b{font-weight:bold;line-height:inherit}
small{font-size:60%;line-height:inherit}
code{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:400;color:rgba(0,0,0,.9)}
ul,ol,dl{line-height:1.6;margin-bottom:1.25em;list-style-position:outside;font-family:inherit}
ul,ol{margin-left:1.5em}
ul li ul,ul li ol{margin-left:1.25em;margin-bottom:0}
ul.circle{list-style-type:circle}
ul.disc{list-style-type:disc}
ul.square{list-style-type:square}
ul.circle ul:not([class]),ul.disc ul:not([class]),ul.square ul:not([class]){list-style:inherit}
ol li ul,ol li ol{margin-left:1.25em;margin-bottom:0}
dl dt{margin-bottom:.3125em;font-weight:bold}
dl dd{margin-bottom:1.25em}
blockquote{margin:0 0 1.25em;padding:.5625em 1.25em 0 1.1875em;border-left:1px solid #ddd}
blockquote,blockquote p{line-height:1.6;color:rgba(0,0,0,.85)}
@media screen and (min-width:768px){h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2}
h1{font-size:2.75em}
h2{font-size:2.3125em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.6875em}
h4{font-size:1.4375em}}
table{background:#fff;margin-bottom:1.25em;border:1px solid #dedede;word-wrap:normal}
table thead,table tfoot{background:#f7f8f7}
table thead tr th,table thead tr td,table tfoot tr th,table tfoot tr td{padding:.5em .625em .625em;font-size:inherit;color:rgba(0,0,0,.8);text-align:left}
table tr th,table tr td{padding:.5625em .625em;font-size:inherit;color:rgba(0,0,0,.8)}
table tr.even,table tr.alt{background:#f8f8f7}
table thead tr th,table tfoot tr th,table tbody tr td,table tr td,table tfoot tr td{line-height:1.6}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2;word-spacing:-.05em}
h1 strong,h2 strong,h3 strong,#toctitle strong,.sidebarblock>.content>.title strong,h4 strong,h5 strong,h6 strong{font-weight:400}
.center{margin-left:auto;margin-right:auto}
.stretch{width:100%}
.clearfix::before,.clearfix::after,.float-group::before,.float-group::after{content:" ";display:table}
.clearfix::after,.float-group::after{clear:both}
:not(pre).nobreak{word-wrap:normal}
:not(pre).nowrap{white-space:nowrap}
:not(pre).pre-wrap{white-space:pre-wrap}
:not(pre):not([class^=L])>code{font-size:.9375em;font-style:normal!important;letter-spacing:0;padding:.1em .5ex;word-spacing:-.15em;background:#f7f7f8;border-radius:4px;line-height:1.45;text-rendering:optimizeSpeed}
pre{color:rgba(0,0,0,.9);font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;line-height:1.45;text-rendering:optimizeSpeed}
pre code,pre pre{color:inherit;font-size:inherit;line-height:inherit}
pre>code{display:block}
pre.nowrap,pre.nowrap pre{white-space:pre;word-wrap:normal}
em em{font-style:normal}
strong strong{font-weight:400}
.keyseq{color:rgba(51,51,51,.8)}
kbd{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;display:inline-block;color:rgba(0,0,0,.8);font-size:.65em;line-height:1.45;background:#f7f7f7;border:1px solid #ccc;border-radius:3px;box-shadow:0 1px 0 rgba(0,0,0,.2),inset 0 0 0 .1em #fff;margin:0 .15em;padding:.2em .5em;vertical-align:middle;position:relative;top:-.1em;white-space:nowrap}
.keyseq kbd:first-child{margin-left:0}
.keyseq kbd:last-child{margin-right:0}
.menuseq,.menuref{color:#000}
.menuseq b:not(.caret),.menuref{font-weight:inherit}
.menuseq{word-spacing:-.02em}
.menuseq b.caret{font-size:1.25em;line-height:.8}
.menuseq i.caret{font-weight:bold;text-align:center;width:.45em}
b.button::before,b.button::after{position:relative;top:-1px;font-weight:400}
b.button::before{content:"[";padding:0 3px 0 2px}
b.button::after{content:"]";padding:0 2px 0 3px}
p a>code:hover{color:rgba(0,0,0,.9)}
#header,#content,#footnotes,#footer{width:100%;margin:0 auto;max-width:62.5em;*zoom:1;position:relative;padding-left:.9375em;padding-right:.9375em}
#header::before,#header::after,#content::before,#content::after,#footnotes::before,#footnotes::after,#footer::before,#footer::after{content:" ";display:table}
#header::after,#content::after,#footnotes::after,#footer::after{clear:both}
#content{margin-top:1.25em}
#content::before{content:none}
#header>h1:first-child{color:rgba(0,0,0,.85);margin-top:2.25rem;margin-bottom:0}
#header>h1:first-child+#toc{margin-top:8px;border-top:1px solid #dddddf}
#header>h1:only-child{border-bottom:1px solid #dddddf;padding-bottom:8px}
#header .details{border-bottom:1px solid #dddddf;line-height:1.45;padding-top:.25em;padding-bottom:.25em;padding-left:.25em;color:rgba(0,0,0,.6);display:flex;flex-flow:row wrap}
#header .details span:first-child{margin-left:-.125em}
#header .details span.email a{color:rgba(0,0,0,.85)}
#header .details br{display:none}
#header .details br+span::before{content:"\00a0\2013\00a0"}
#header .details br+span.author::before{content:"\00a0\22c5\00a0";color:rgba(0,0,0,.85)}
#header .details br+span#revremark::before{content:"\00a0|\00a0"}
#header #revnumber{text-transform:capitalize}
#header #revnumber::after{content:"\00a0"}
#content>h1:first-child:not([class]){color:rgba(0,0,0,.85);border-bottom:1px solid #dddddf;padding-bottom:8px;margin-top:0;padding-top:1rem;margin-bottom:1.25rem}
#toc{border-bottom:1px solid #e7e7e9;padding-bottom:.5em}
#toc>ul{margin-left:.125em}
#toc ul.sectlevel0>li>a{font-style:italic}
#toc ul.sectlevel0 ul.sectlevel1{margin:.5em 0}
#toc ul{font-family:"Open Sans","DejaVu Sans",sans-serif;list-style-type:none}
#toc li{line-height:1.3334;margin-top:.3334em}
#toc a{text-decoration:none}
#toc a:active{text-decoration:underline}
#toctitle{color:#7a2518;font-size:1.2em}
@media screen and (min-width:768px){#toctitle{font-size:1.375em}
body.toc2{padding-left:15em;padding-right:0}
body.toc2 #header>h1:nth-last-child(2){border-bottom:1px solid #dddddf;padding-bottom:8px}
#toc.toc2{margin-top:0!important;background:#f8f8f7;position:fixed;width:15em;left:0;top:0;border-right:1px solid #e7e7e9;border-top-width:0!important;border-bottom-width:0!important;z-index:1000;padding:1.25em 1em;height:100%;overflow:auto}
#toc.toc2 #toctitle{margin-top:0;margin-bottom:.8rem;font-size:1.2em}
#toc.toc2>ul{font-size:.9em;margin-bottom:0}
#toc.toc2 ul ul{margin-left:0;padding-left:1em}
#toc.toc2 ul.sectlevel0 ul.sectlevel1{padding-left:0;margin-top:.5em;margin-bottom:.5em}
body.toc2.toc-right{padding-left:0;padding-right:15em}
body.toc2.toc-right #toc.toc2{border-right-width:0;border-left:1px solid #e7e7e9;left:auto;right:0}}
@media screen and (min-width:1280px){body.toc2{padding-left:20em;padding-right:0}
#toc.toc2{width:20em}
#toc.toc2 #toctitle{font-size:1.375em}
#toc.toc2>ul{font-size:.95em}
#toc.toc2 ul ul{padding-left:1.25em}
body.toc2.toc-right{padding-left:0;padding-right:20em}}
#content #toc{border:1px solid #e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;border-radius:4px}
#content #toc>:first-child{margin-top:0}
#content #toc>:last-child{margin-bottom:0}
#footer{max-width:none;background:rgba(0,0,0,.8);padding:1.25em}
#footer-text{color:hsla(0,0%,100%,.8);line-height:1.44}
#content{margin-bottom:.625em}
.sect1{padding-bottom:.625em}
@media screen and (min-width:768px){#content{margin-bottom:1.25em}
.sect1{padding-bottom:1.25em}}
.sect1:last-child{padding-bottom:0}
.sect1+.sect1{border-top:1px solid #e7e7e9}
#content h1>a.anchor,h2>a.anchor,h3>a.anchor,#toctitle>a.anchor,.sidebarblock>.content>.title>a.anchor,h4>a.anchor,h5>a.anchor,h6>a.anchor{position:absolute;z-index:1001;width:1.5ex;margin-left:-1.5ex;display:block;text-decoration:none!important;visibility:hidden;text-align:center;font-weight:400}
#content h1>a.anchor::before,h2>a.anchor::before,h3>a.anchor::before,#toctitle>a.anchor::before,.sidebarblock>.content>.title>a.anchor::before,h4>a.anchor::before,h5>a.anchor::before,h6>a.anchor::before{content:"\00A7";font-size:.85em;display:block;padding-top:.1em}
#content h1:hover>a.anchor,#content h1>a.anchor:hover,h2:hover>a.anchor,h2>a.anchor:hover,h3:hover>a.anchor,#toctitle:hover>a.anchor,.sidebarblock>.content>.title:hover>a.anchor,h3>a.anchor:hover,#toctitle>a.anchor:hover,.sidebarblock>.content>.title>a.anchor:hover,h4:hover>a.anchor,h4>a.anchor:hover,h5:hover>a.anchor,h5>a.anchor:hover,h6:hover>a.anchor,h6>a.anchor:hover{visibility:visible}
#content h1>a.link,h2>a.link,h3>a.link,#toctitle>a.link,.sidebarblock>.content>.title>a.link,h4>a.link,h5>a.link,h6>a.link{color:#ba3925;text-decoration:none}
#content h1>a.link:hover,h2>a.link:hover,h3>a.link:hover,#toctitle>a.link:hover,.sidebarblock>.content>.title>a.link:hover,h4>a.link:hover,h5>a.link:hover,h6>a.link:hover{color:#a53221}
details,.audioblock,.imageblock,.literalblock,.listingblock,.stemblock,.videoblock{margin-bottom:1.25em}
details{margin-left:1.25rem}
details>summary{cursor:pointer;display:block;position:relative;line-height:1.6;margin-bottom:.625rem;outline:none;-webkit-tap-highlight-color:transparent}
details>summary::-webkit-details-marker{display:none}
details>summary::before{content:"";border:solid transparent;border-left:solid;border-width:.3em 0 .3em .5em;position:absolute;top:.5em;left:-1.25rem;transform:translateX(15%)}
details[open]>summary::before{border:solid transparent;border-top:solid;border-width:.5em .3em 0;transform:translateY(15%)}
details>summary::after{content:"";width:1.25rem;height:1em;position:absolute;top:.3em;left:-1.25rem}
.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{text-rendering:optimizeLegibility;text-align:left;font-family:"Noto Serif","DejaVu Serif",serif;font-size:1rem;font-style:italic}
table.tableblock.fit-content>caption.title{white-space:nowrap;width:0}
.paragraph.lead>p,#preamble>.sectionbody>[class=paragraph]:first-of-type p{font-size:1.21875em;line-height:1.6;color:rgba(0,0,0,.85)}
.admonitionblock>table{border-collapse:separate;border:0;background:none;width:100%}
.admonitionblock>table td.icon{text-align:center;width:80px}
.admonitionblock>table td.icon img{max-width:none}
.admonitionblock>table td.icon .title{font-weight:bold;font-family:"Open Sans","DejaVu Sans",sans-serif;text-transform:uppercase}
.admonitionblock>table td.content{padding-left:1.125em;padding-right:1.25em;border-left:1px solid #dddddf;color:rgba(0,0,0,.6);word-wrap:anywhere}
.admonitionblock>table td.content>:last-child>:last-child{margin-bottom:0}
.exampleblock>.content{border:1px solid #e6e6e6;margin-bottom:1.25em;padding:1.25em;background:#fff;border-radius:4px}
.sidebarblock{border:1px solid #dbdbd6;margin-bottom:1.25em;padding:1.25em;background:#f3f3f2;border-radius:4px}
.sidebarblock>.content>.title{color:#7a2518;margin-top:0;text-align:center}
.exampleblock>.content>:first-child,.sidebarblock>.content>:first-child{margin-top:0}
.exampleblock>.content>:last-child,.exampleblock>.content>:last-child>:last-child,.exampleblock>.content .olist>ol>li:last-child>:last-child,.exampleblock>.content .ulist>ul>li:last-child>:last-child,.exampleblock>.content .qlist>ol>li:last-child>:last-child,.sidebarblock>.content>:last-child,.sidebarblock>.content>:last-child>:last-child,.sidebarblock>.content .olist>ol>li:last-child>:last-child,.sidebarblock>.content .ulist>ul>li:last-child>:last-child,.sidebarblock>.content .qlist>ol>li:last-child>:last-child{margin-bottom:0}
.literalblock pre,.listingblock>.content>pre{border-radius:4px;overflow-x:auto;padding:1em;font-size:.8125em}
@media screen and (min-width:768px){.literalblock pre,.listingblock>.content>pre{font-size:.90625em}}
@media screen and (min-width:1280px){.literalblock pre,.listingblock>.content>pre{font-size:1em}}
.literalblock pre,.listingblock>.content>pre:not(.highlight),.listingblock>.content>pre[class=highlight],.listingblock>.content>pre[class^="highlight "]{background:#f7f7f8}
.literalblock.output pre{color:#f7f7f8;background:rgba(0,0,0,.9)}
.listingblock>.content{position:relative}
.listingblock code[data-lang]::before{display:none;content:attr(data-lang);position:absolute;font-size:.75em;top:.425rem;right:.5rem;line-height:1;text-transform:uppercase;color:inherit;opacity:.5}
.listingblock:hover code[data-lang]::before{display:block}
.listingblock.terminal pre .command::before{content:attr(data-prompt);padding-right:.5em;color:inherit;opacity:.5}
.listingblock.terminal pre .command:not([data-prompt])::before{content:"$"}
.listingblock pre.highlightjs{padding:0}
.listingblock pre.highlightjs>code{padding:1em;border-radius:4px}
.listingblock pre.prettyprint{border-width:0}
.prettyprint{background:#f7f7f8}
pre.prettyprint .linenums{line-height:1.45;margin-left:2em}
pre.prettyprint li{background:none;list-style-type:inherit;padding-left:0}
pre.prettyprint li code[data-lang]::before{opacity:1}
pre.prettyprint li:not(:first-child) code[data-lang]::before{display:none}
table.linenotable{border-collapse:separate;border:0;margin-bottom:0;background:none}
table.linenotable td[class]{color:inherit;vertical-align:top;padding:0;line-height:inherit;white-space:normal}
table.linenotable td.code{padding-left:.75em}
table.linenotable td.linenos,pre.pygments .linenos{border-right:1px solid;opacity:.35;padding-right:.5em;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}
pre.pygments span.linenos{display:inline-block;margin-right:.75em}
.quoteblock{margin:0 1em 1.25em 1.5em;display:table}
.quoteblock:not(.excerpt)>.title{margin-left:-1.5em;margin-bottom:.75em}
.quoteblock blockquote,.quoteblock p{color:rgba(0,0,0,.85);font-size:1.15rem;line-height:1.75;word-spacing:.1em;letter-spacing:0;font-style:italic;text-align:justify}
.quoteblock blockquote{margin:0;padding:0;border:0}
.quoteblock blockquote::before{content:"\201c";float:left;font-size:2.75em;font-weight:bold;line-height:.6em;margin-left:-.6em;color:#7a2518;text-shadow:0 1px 2px rgba(0,0,0,.1)}
.quoteblock blockquote>.paragraph:last-child p{margin-bottom:0}
.quoteblock .attribution{margin-top:.75em;margin-right:.5ex;text-align:right}
.verseblock{margin:0 1em 1.25em}
.verseblock pre{font-family:"Open Sans","DejaVu Sans",sans-serif;font-size:1.15rem;color:rgba(0,0,0,.85);font-weight:300;text-rendering:optimizeLegibility}
.verseblock pre strong{font-weight:400}
.verseblock .attribution{margin-top:1.25rem;margin-left:.5ex}
.quoteblock .attribution,.verseblock .attribution{font-size:.9375em;line-height:1.45;font-style:italic}
.quoteblock .attribution br,.verseblock .attribution br{display:none}
.quoteblock .attribution cite,.verseblock .attribution cite{display:block;letter-spacing:-.025em;color:rgba(0,0,0,.6)}
.quoteblock.abstract blockquote::before,.quoteblock.excerpt blockquote::before,.quoteblock .quoteblock blockquote::before{display:none}
.quoteblock.abstract blockquote,.quoteblock.abstract p,.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{line-height:1.6;word-spacing:0}
.quoteblock.abstract{margin:0 1em 1.25em;display:block}
.quoteblock.abstract>.title{margin:0 0 .375em;font-size:1.15em;text-align:center}
.quoteblock.excerpt>blockquote,.quoteblock .quoteblock{padding:0 0 .25em 1em;border-left:.25em solid #dddddf}
.quoteblock.excerpt,.quoteblock .quoteblock{margin-left:0}
.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{color:inherit;font-size:1.0625rem}
.quoteblock.excerpt .attribution,.quoteblock .quoteblock .attribution{color:inherit;font-size:.85rem;text-align:left;margin-right:0}
p.tableblock:last-child{margin-bottom:0}
td.tableblock>.content{margin-bottom:1.25em;word-wrap:anywhere}
td.tableblock>.content>:last-child{margin-bottom:-1.25em}
table.tableblock,th.tableblock,td.tableblock{border:0 solid #dedede}
table.grid-all>*>tr>*{border-width:1px}
table.grid-cols>*>tr>*{border-width:0 1px}
table.grid-rows>*>tr>*{border-width:1px 0}
table.frame-all{border-width:1px}
table.frame-ends{border-width:1px 0}
table.frame-sides{border-width:0 1px}
table.frame-none>colgroup+*>:first-child>*,table.frame-sides>colgroup+*>:first-child>*{border-top-width:0}
table.frame-none>:last-child>:last-child>*,table.frame-sides>:last-child>:last-child>*{border-bottom-width:0}
table.frame-none>*>tr>:first-child,table.frame-ends>*>tr>:first-child{border-left-width:0}
table.frame-none>*>tr>:last-child,table.frame-ends>*>tr>:last-child{border-right-width:0}
table.stripes-all>*>tr,table.stripes-odd>*>tr:nth-of-type(odd),table.stripes-even>*>tr:nth-of-type(even),table.stripes-hover>*>tr:hover{background:#f8f8f7}
th.halign-left,td.halign-left{text-align:left}
th.halign-right,td.halign-right{text-align:right}
th.halign-center,td.halign-center{text-align:center}
th.valign-top,td.valign-top{vertical-align:top}
th.valign-bottom,td.valign-bottom{vertical-align:bottom}
th.valign-middle,td.valign-middle{vertical-align:middle}
table thead th,table tfoot th{font-weight:bold}
tbody tr th{background:#f7f8f7}
tbody tr th,tbody tr th p,tfoot tr th,tfoot tr th p{color:rgba(0,0,0,.8);font-weight:bold}
p.tableblock>code:only-child{background:none;padding:0}
p.tableblock{font-size:1em}
ol{margin-left:1.75em}
ul li ol{margin-left:1.5em}
dl dd{margin-left:1.125em}
dl dd:last-child,dl dd:last-child>:last-child{margin-bottom:0}
li p,ul dd,ol dd,.olist .olist,.ulist .ulist,.ulist .olist,.olist .ulist{margin-bottom:.625em}
ul.checklist,ul.none,ol.none,ul.no-bullet,ol.no-bullet,ol.unnumbered,ul.unstyled,ol.unstyled{list-style-type:none}
ul.no-bullet,ol.no-bullet,ol.unnumbered{margin-left:.625em}
ul.unstyled,ol.unstyled{margin-left:0}
li>p:empty:only-child::before{content:"";display:inline-block}
ul.checklist>li>p:first-child{margin-left:-1em}
ul.checklist>li>p:first-child>.fa-square-o:first-child,ul.checklist>li>p:first-child>.fa-check-square-o:first-child{width:1.25em;font-size:.8em;position:relative;bottom:.125em}
ul.checklist>li>p:first-child>input[type=checkbox]:first-child{margin-right:.25em}
ul.inline{display:flex;flex-flow:row wrap;list-style:none;margin:0 0 .625em -1.25em}
ul.inline>li{margin-left:1.25em}
.unstyled dl dt{font-weight:400;font-style:normal}
ol.arabic{list-style-type:decimal}
ol.decimal{list-style-type:decimal-leading-zero}
ol.loweralpha{list-style-type:lower-alpha}
ol.upperalpha{list-style-type:upper-alpha}
ol.lowerroman{list-style-type:lower-roman}
ol.upperroman{list-style-type:upper-roman}
ol.lowergreek{list-style-type:lower-greek}
.hdlist>table,.colist>table{border:0;background:none}
.hdlist>table>tbody>tr,.colist>table>tbody>tr{background:none}
td.hdlist1,td.hdlist2{vertical-align:top;padding:0 .625em}
td.hdlist1{font-weight:bold;padding-bottom:1.25em}
td.hdlist2{word-wrap:anywhere}
.literalblock+.colist,.listingblock+.colist{margin-top:-.5em}
.colist td:not([class]):first-child{padding:.4em .75em 0;line-height:1;vertical-align:top}
.colist td:not([class]):first-child img{max-width:none}
.colist td:not([class]):last-child{padding:.25em 0}
.thumb,.th{line-height:0;display:inline-block;border:4px solid #fff;box-shadow:0 0 0 1px #ddd}
.imageblock.left{margin:.25em .625em 1.25em 0}
.imageblock.right{margin:.25em 0 1.25em .625em}
.imageblock>.title{margin-bottom:0}
.imageblock.thumb,.imageblock.th{border-width:6px}
.imageblock.thumb>.title,.imageblock.th>.title{padding:0 .125em}
.image.left,.image.right{margin-top:.25em;margin-bottom:.25em;display:inline-block;line-height:0}
.image.left{margin-right:.625em}
.image.right{margin-left:.625em}
a.image{text-decoration:none;display:inline-block}
a.image object{pointer-events:none}
sup.footnote,sup.footnoteref{font-size:.875em;position:static;vertical-align:super}
sup.footnote a,sup.footnoteref a{text-decoration:none}
sup.footnote a:active,sup.footnoteref a:active,#footnotes .footnote a:first-of-type:active{text-decoration:underline}
#footnotes{padding-top:.75em;padding-bottom:.75em;margin-bottom:.625em}
#footnotes hr{width:20%;min-width:6.25em;margin:-.25em 0 .75em;border-width:1px 0 0}
#footnotes .footnote{padding:0 .375em 0 .225em;line-height:1.3334;font-size:.875em;margin-left:1.2em;margin-bottom:.2em}
#footnotes .footnote a:first-of-type{font-weight:bold;text-decoration:none;margin-left:-1.05em}
#footnotes .footnote:last-of-type{margin-bottom:0}
#content #footnotes{margin-top:-.625em;margin-bottom:0;padding:.75em 0}
div.unbreakable{page-break-inside:avoid}
.big{font-size:larger}
.small{font-size:smaller}
.underline{text-decoration:underline}
.overline{text-decoration:overline}
.line-through{text-decoration:line-through}
.aqua{color:#00bfbf}
.aqua-background{background:#00fafa}
.black{color:#000}
.black-background{background:#000}
.blue{color:#0000bf}
.blue-background{background:#0000fa}
.fuchsia{color:#bf00bf}
.fuchsia-background{background:#fa00fa}
.gray{color:#606060}
.gray-background{background:#7d7d7d}
.green{color:#006000}
.green-background{background:#007d00}
.lime{color:#00bf00}
.lime-background{background:#00fa00}
.maroon{color:#600000}
.maroon-background{background:#7d0000}
.navy{color:#000060}
.navy-background{background:#00007d}
.olive{color:#606000}
.olive-background{background:#7d7d00}
.purple{color:#600060}
.purple-background{background:#7d007d}
.red{color:#bf0000}
.red-background{background:#fa0000}
.silver{color:#909090}
.silver-background{background:#bcbcbc}
.teal{color:#006060}
.teal-background{background:#007d7d}
.white{color:#bfbfbf}
.white-background{background:#fafafa}
.yellow{color:#bfbf00}
.yellow-background{background:#fafa00}
span.icon>.fa{cursor:default}
a span.icon>.fa{cursor:inherit}
.admonitionblock td.icon [class^="fa icon-"]{font-size:2.5em;text-shadow:1px 1px 2px rgba(0,0,0,.5);cursor:default}
.admonitionblock td.icon .icon-note::before{content:"\f05a";color:#19407c}
.admonitionblock td.icon .icon-tip::before{content:"\f0eb";text-shadow:1px 1px 2px rgba(155,155,0,.8);color:#111}
.admonitionblock td.icon .icon-warning::before{content:"\f071";color:#bf6900}
.admonitionblock td.icon .icon-caution::before{content:"\f06d";color:#bf3400}
.admonitionblock td.icon .icon-important::before{content:"\f06a";color:#bf0000}
.conum[data-value]{display:inline-block;color:#fff!important;background:rgba(0,0,0,.8);border-radius:50%;text-align:center;font-size:.75em;width:1.67em;height:1.67em;line-height:1.67em;font-family:"Open Sans","DejaVu Sans",sans-serif;font-style:normal;font-weight:bold}
.conum[data-value] *{color:#fff!important}
.conum[data-value]+b{display:none}
.conum[data-value]::after{content:attr(data-value)}
pre .conum[data-value]{position:relative;top:-.125em}
b.conum *{color:inherit!important}
.conum:not([data-value]):empty{display:none}
dt,th.tableblock,td.content,div.footnote{text-rendering:optimizeLegibility}
h1,h2,p,td.content,span.alt,summary{letter-spacing:-.01em}
p strong,td.content strong,div.footnote strong{letter-spacing:-.005em}
p,blockquote,dt,td.content,td.hdlist1,span.alt,summary{font-size:1.0625rem}
p{margin-bottom:1.25rem}
.sidebarblock p,.sidebarblock dt,.sidebarblock td.content,p.tableblock{font-size:1em}
.exampleblock>.content{background:#fffef7;border-color:#e0e0dc;box-shadow:0 1px 4px #e0e0dc}
.print-only{display:none!important}
@page{margin:1.25cm .75cm}
@media print{*{box-shadow:none!important;text-shadow:none!important}
html{font-size:80%}
a{color:inherit!important;text-decoration:underline!important}
a.bare,a[href^="#"],a[href^="mailto:"]{text-decoration:none!important}
a[href^="http:"]:not(.bare)::after,a[href^="https:"]:not(.bare)::after{content:"(" attr(href) ")";display:inline-block;font-size:.875em;padding-left:.25em}
abbr[title]{border-bottom:1px dotted}
abbr[title]::after{content:" (" attr(title) ")"}
pre,blockquote,tr,img,object,svg{page-break-inside:avoid}
thead{display:table-header-group}
svg{max-width:100%}
p,blockquote,dt,td.content{font-size:1em;orphans:3;widows:3}
h2,h3,#toctitle,.sidebarblock>.content>.title{page-break-after:avoid}
#header,#content,#footnotes,#footer{max-width:none}
#toc,.sidebarblock,.exampleblock>.content{background:none!important}
#toc{border-bottom:1px solid #dddddf!important;padding-bottom:0!important}
body.book #header{text-align:center}
body.book #header>h1:first-child{border:0!important;margin:2.5em 0 1em}
body.book #header .details{border:0!important;display:block;padding:0!important}
body.book #header .details span:first-child{margin-left:0!important}
body.book #header .details br{display:block}
body.book #header .details br+span::before{content:none!important}
body.book #toc{border:0!important;text-align:left!important;padding:0!important;margin:0!important}
body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-break-before:always}
.listingblock code[data-lang]::before{display:block}
#footer{padding:0 .9375em}
.hide-on-print{display:none!important}
.print-only{display:block!important}
.hide-for-print{display:none!important}
.show-for-print{display:inherit!important}}
@media amzn-kf8,print{#header>h1:first-child{margin-top:1.25rem}
.sect1{padding:0!important}
.sect1+.sect1{border:0}
#footer{background:none}
#footer-text{color:rgba(0,0,0,.6);font-size:.9em}}
@media amzn-kf8{#header,#content,#footnotes,#footer{padding:0}}
</style>
</head>
<body class="article">
<div id="header">
<h1>Partial Clone Design Notes</h1>
</div>
<div id="content">
<div id="preamble">
<div class="sectionbody">
<div class="paragraph">
<p>The "Partial Clone" feature is a performance optimization for Git that
allows Git to function without having a complete copy of the repository.
The goal of this work is to allow Git to better handle extremely large
repositories.</p>
</div>
<div class="paragraph">
<p>During clone and fetch operations, Git downloads the complete contents
and history of the repository. This includes all commits, trees, and
blobs for the complete life of the repository. For extremely large
repositories, clones can take hours (or days) and consume 100+GiB of disk
space.</p>
</div>
<div class="paragraph">
<p>Often in these repositories there are many blobs and trees that the user
does not need such as:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>files outside of the user&#8217;s work area in the tree. For example, in
a repository with 500K directories and 3.5M files in every commit,
we can avoid downloading many objects if the user only needs a
narrow "cone" of the source tree.</p>
</li>
<li>
<p>large binary assets. For example, in a repository where large build
artifacts are checked into the tree, we can avoid downloading all
previous versions of these non-mergeable binary assets and only
download versions that are actually referenced.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>Partial clone allows us to avoid downloading such unneeded objects <strong>in
advance</strong> during clone and fetch operations and thereby reduce download
times and disk usage. Missing objects can later be "demand fetched"
if/when needed.</p>
</div>
<div class="paragraph">
<p>A remote that can later provide the missing objects is called a
promisor remote, as it promises to send the objects when
requested. Initially Git supported only one promisor remote, the origin
remote from which the user cloned and that was configured in the
"extensions.partialClone" config option. Later support for more than
one promisor remote has been implemented.</p>
</div>
<div class="paragraph">
<p>Use of partial clone requires that the user be online and the origin
remote or other promisor remotes be available for on-demand fetching
of missing objects. This may or may not be problematic for the user.
For example, if the user can stay within the pre-selected subset of
the source tree, they may not encounter any missing objects.
Alternatively, the user could try to pre-fetch various objects if they
know that they are going offline.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_non_goals">Non-Goals</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Partial clone is a mechanism to limit the number of blobs and trees downloaded
<strong>within</strong> a given range of commits&#8201;&#8212;&#8201;and is therefore independent of and not
intended to conflict with existing DAG-level mechanisms to limit the set of
requested commits (i.e. shallow clone, single branch, or fetch <em>&lt;refspec&gt;</em>).</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_design_overview">Design Overview</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Partial clone logically consists of the following parts:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>A mechanism for the client to describe unneeded or unwanted objects to
the server.</p>
</li>
<li>
<p>A mechanism for the server to omit such unwanted objects from packfiles
sent to the client.</p>
</li>
<li>
<p>A mechanism for the client to gracefully handle missing objects (that
were previously omitted by the server).</p>
</li>
<li>
<p>A mechanism for the client to backfill missing objects as needed.</p>
</li>
</ul>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_design_details">Design Details</h2>
<div class="sectionbody">
<div class="ulist">
<ul>
<li>
<p>A new pack-protocol capability "filter" is added to the fetch-pack and
upload-pack negotiation.</p>
<div class="paragraph">
<p>This uses the existing capability discovery mechanism.
See "filter" in <a href="../gitprotocol-pack.html">gitprotocol-pack(5)</a>.</p>
</div>
</li>
<li>
<p>Clients pass a "filter-spec" to clone and fetch which is passed to the
server to request filtering during packfile construction.</p>
<div class="paragraph">
<p>There are various filters available to accommodate different situations.
See "--filter=&lt;filter-spec&gt;" in Documentation/rev-list-options.adoc.</p>
</div>
</li>
<li>
<p>On the server pack-objects applies the requested filter-spec as it
creates "filtered" packfiles for the client.</p>
<div class="paragraph">
<p>These filtered packfiles are <strong>incomplete</strong> in the traditional sense because
they may contain objects that reference objects not contained in the
packfile and that the client doesn&#8217;t already have. For example, the
filtered packfile may contain trees or tags that reference missing blobs
or commits that reference missing trees.</p>
</div>
</li>
<li>
<p>On the client these incomplete packfiles are marked as "promisor packfiles"
and treated differently by various commands.</p>
</li>
<li>
<p>On the client a repository extension is added to the local config to
prevent older versions of git from failing mid-operation because of
missing objects that they cannot handle.
See <code>extensions.partialClone</code> in <a href="../git-config.html">git-config(1)</a>.</p>
</li>
</ul>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_handling_missing_objects">Handling Missing Objects</h2>
<div class="sectionbody">
<div class="ulist">
<ul>
<li>
<p>An object may be missing due to a partial clone or fetch, or missing
due to repository corruption. To differentiate these cases, the
local repository specially indicates such filtered packfiles
obtained from promisor remotes as "promisor packfiles".</p>
<div class="paragraph">
<p>These promisor packfiles consist of a "&lt;name&gt;.promisor" file with
arbitrary contents (like the "&lt;name&gt;.keep" files), in addition to
their "&lt;name&gt;.pack" and "&lt;name&gt;.idx" files.</p>
</div>
</li>
<li>
<p>The local repository considers a "promisor object" to be an object that
it knows (to the best of its ability) that promisor remotes have promised
that they have, either because the local repository has that object in one of
its promisor packfiles, or because another promisor object refers to it.</p>
<div class="paragraph">
<p>When Git encounters a missing object, Git can see if it is a promisor object
and handle it appropriately. If not, Git can report a corruption.</p>
</div>
<div class="paragraph">
<p>This means that there is no need for the client to explicitly maintain an
expensive-to-modify list of missing objects.[a]</p>
</div>
</li>
<li>
<p>Since almost all Git code currently expects any referenced object to be
present locally and because we do not want to force every command to do
a dry-run first, a fallback mechanism is added to allow Git to attempt
to dynamically fetch missing objects from promisor remotes.</p>
<div class="paragraph">
<p>When the normal object lookup fails to find an object, Git invokes
promisor_remote_get_direct() to try to get the object from a promisor
remote and then retry the object lookup. This allows objects to be
"faulted in" without complicated prediction algorithms.</p>
</div>
<div class="paragraph">
<p>For efficiency reasons, no check as to whether the missing object is
actually a promisor object is performed.</p>
</div>
<div class="paragraph">
<p>Dynamic object fetching tends to be slow as objects are fetched one at
a time.</p>
</div>
</li>
<li>
<p><code>checkout</code> (and any other command using <code>unpack-trees</code>) has been taught
to bulk pre-fetch all required missing blobs in a single batch.</p>
</li>
<li>
<p><code>rev-list</code> has been taught to print missing objects.</p>
<div class="paragraph">
<p>This can be used by other commands to bulk prefetch objects.
For example, a "git log -p A..B" may internally want to first do
something like "git rev-list --objects --quiet --missing=print A..B"
and prefetch those objects in bulk.</p>
</div>
</li>
<li>
<p><code>fsck</code> has been updated to be fully aware of promisor objects.</p>
</li>
<li>
<p><code>repack</code> in GC has been updated to not touch promisor packfiles at all,
and to only repack other objects.</p>
</li>
<li>
<p>The global variable "fetch_if_missing" is used to control whether an
object lookup will attempt to dynamically fetch a missing object or
report an error.</p>
<div class="paragraph">
<p>We are not happy with this global variable and would like to remove it,
but that requires significant refactoring of the object code to pass an
additional flag.</p>
</div>
</li>
</ul>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_fetching_missing_objects">Fetching Missing Objects</h2>
<div class="sectionbody">
<div class="ulist">
<ul>
<li>
<p>Fetching of objects is done by invoking a "git fetch" subprocess.</p>
</li>
<li>
<p>The local repository sends a request with the hashes of all requested
objects, and does not perform any packfile negotiation.
It then receives a packfile.</p>
</li>
<li>
<p>Because we are reusing the existing fetch mechanism, fetching
currently fetches all objects referred to by the requested objects, even
though they are not necessary.</p>
</li>
<li>
<p>Fetching with <code>--refetch</code> will request a complete new filtered packfile from
the remote, which can be used to change a filter without needing to
dynamically fetch missing objects.</p>
</li>
</ul>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_using_many_promisor_remotes">Using many promisor remotes</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Many promisor remotes can be configured and used.</p>
</div>
<div class="paragraph">
<p>This allows for example a user to have multiple geographically-close
cache servers for fetching missing blobs while continuing to do
filtered <code>git-fetch</code> commands from the central server.</p>
</div>
<div class="paragraph">
<p>When fetching objects, promisor remotes are tried one after the other
until all the objects have been fetched.</p>
</div>
<div class="paragraph">
<p>Remotes that are considered "promisor" remotes are those specified by
the following configuration variables:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>extensions.partialClone</code> <code>=</code> <em>&lt;name&gt;</em></p>
</li>
<li>
<p><code>remote.</code><em>&lt;name&gt;</em><code>.promisor</code> <code>=</code> <code>true</code></p>
</li>
<li>
<p><code>remote.</code><em>&lt;name&gt;</em><code>.partialCloneFilter</code> <code>=</code> ...</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Only one promisor remote can be configured using the
<code>extensions.partialClone</code> config variable. This promisor remote will
be the last one tried when fetching objects.</p>
</div>
<div class="paragraph">
<p>We decided to make it the last one we try, because it is likely that
someone using many promisor remotes is doing so because the other
promisor remotes are better for some reason (maybe they are closer or
faster for some kind of objects) than the origin, and the origin is
likely to be the remote specified by extensions.partialClone.</p>
</div>
<div class="paragraph">
<p>This justification is not very strong, but one choice had to be made,
and anyway the long term plan should be to make the order somehow
fully configurable.</p>
</div>
<div class="paragraph">
<p>For now though the other promisor remotes will be tried in the order
they appear in the config file.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_current_limitations">Current Limitations</h2>
<div class="sectionbody">
<div class="ulist">
<ul>
<li>
<p>It is not possible to specify the order in which the promisor
remotes are tried in other ways than the order in which they appear
in the config file.</p>
<div class="paragraph">
<p>It is also not possible to specify an order to be used when fetching
from one remote and a different order when fetching from another
remote.</p>
</div>
</li>
<li>
<p>It is not possible to push only specific objects to a promisor
remote.</p>
<div class="paragraph">
<p>It is not possible to push at the same time to multiple promisor
remote in a specific order.</p>
</div>
</li>
<li>
<p>Dynamic object fetching will only ask promisor remotes for missing
objects. We assume that promisor remotes have a complete view of the
repository and can satisfy all such requests.</p>
</li>
<li>
<p>Repack essentially treats promisor and non-promisor packfiles as 2
distinct partitions and does not mix them.</p>
</li>
<li>
<p>Dynamic object fetching invokes fetch-pack once <strong>for each item</strong>
because most algorithms stumble upon a missing object and need to have
it resolved before continuing their work. This may incur significant
overhead&#8201;&#8212;&#8201;and multiple authentication requests&#8201;&#8212;&#8201;if many objects are
needed.</p>
</li>
<li>
<p>Dynamic object fetching currently uses the existing pack protocol V0
which means that each object is requested via fetch-pack. The server
will send a full set of info/refs when the connection is established.
If there are a large number of refs, this may incur significant overhead.</p>
</li>
</ul>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_future_work">Future Work</h2>
<div class="sectionbody">
<div class="ulist">
<ul>
<li>
<p>Improve the way to specify the order in which promisor remotes are
tried.</p>
<div class="paragraph">
<p>For example this could allow specifying explicitly something like:
"When fetching from this remote, I want to use these promisor remotes
in this order, though, when pushing or fetching to that remote, I want
to use those promisor remotes in that order."</p>
</div>
</li>
<li>
<p>Allow pushing to promisor remotes.</p>
<div class="paragraph">
<p>The user might want to work in a triangular work flow with multiple
promisor remotes that each have an incomplete view of the repository.</p>
</div>
</li>
<li>
<p>Allow non-pathname-based filters to make use of packfile bitmaps (when
present). This was just an omission during the initial implementation.</p>
</li>
<li>
<p>Investigate use of a long-running process to dynamically fetch a series
of objects, such as proposed in [5,6] to reduce process startup and
overhead costs.</p>
<div class="paragraph">
<p>It would be nice if pack protocol V2 could allow that long-running
process to make a series of requests over a single long-running
connection.</p>
</div>
</li>
<li>
<p>Investigate pack protocol V2 to avoid the info/refs broadcast on
each connection with the server to dynamically fetch missing objects.</p>
</li>
<li>
<p>Investigate the need to handle loose promisor objects.</p>
<div class="paragraph">
<p>Objects in promisor packfiles are allowed to reference missing objects
that can be dynamically fetched from the server. An assumption was
made that loose objects are only created locally and therefore should
not reference a missing object. We may need to revisit that assumption
if, for example, we dynamically fetch a missing tree and store it as a
loose object rather than a single object packfile.</p>
</div>
<div class="paragraph">
<p>This does not necessarily mean we need to mark loose objects as promisor;
it may be sufficient to relax the object lookup or is-promisor functions.</p>
</div>
</li>
</ul>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_non_tasks">Non-Tasks</h2>
<div class="sectionbody">
<div class="ulist">
<ul>
<li>
<p>Every time the subject of "demand loading blobs" comes up it seems
that someone suggests that the server be allowed to "guess" and send
additional objects that may be related to the requested objects.</p>
<div class="paragraph">
<p>No work has gone into actually doing that; we&#8217;re just documenting that
it is a common suggestion. We&#8217;re not sure how it would work and have
no plans to work on it.</p>
</div>
<div class="paragraph">
<p>It is valid for the server to send more objects than requested (even
for a dynamic object fetch), but we are not building on that.</p>
</div>
</li>
</ul>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_footnotes">Footnotes</h2>
<div class="sectionbody">
<div class="paragraph">
<p>[a] expensive-to-modify list of missing objects: Earlier in the design of
partial clone we discussed the need for a single list of missing objects.
This would essentially be a sorted linear list of OIDs that were
omitted by the server during a clone or subsequent fetches.</p>
</div>
<div class="paragraph">
<p>This file would need to be loaded into memory on every object lookup.
It would need to be read, updated, and re-written (like the .git/index)
on every explicit "git fetch" command <strong>and</strong> on any dynamic object fetch.</p>
</div>
<div class="paragraph">
<p>The cost to read, update, and write this file could add significant
overhead to every command if there are many missing objects. For example,
if there are 100M missing blobs, this file would be at least 2GiB on disk.</p>
</div>
<div class="paragraph">
<p>With the "promisor" concept, we <strong>infer</strong> a missing object based upon the
type of packfile that references it.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_related_links">Related Links</h2>
<div class="sectionbody">
<div class="paragraph">
<p>[0] <a href="https://crbug.com/git/2" class="bare">https://crbug.com/git/2</a>
Bug#2: Partial Clone</p>
</div>
<div class="paragraph">
<p>[1] <a href="https://lore.kernel.org/git/20170113155253.1644-1-benpeart@microsoft.com/" class="bare">https://lore.kernel.org/git/20170113155253.1644-1-benpeart@microsoft.com/</a><br/>
Subject: [RFC] Add support for downloading blobs on demand<br/>
Date: Fri, 13 Jan 2017 10:52:53 -0500</p>
</div>
<div class="paragraph">
<p>[2] <a href="https://lore.kernel.org/git/cover.1506714999.git.jonathantanmy@google.com/" class="bare">https://lore.kernel.org/git/cover.1506714999.git.jonathantanmy@google.com/</a><br/>
Subject: [PATCH 00/18] Partial clone (from clone to lazy fetch in 18 patches)<br/>
Date: Fri, 29 Sep 2017 13:11:36 -0700</p>
</div>
<div class="paragraph">
<p>[3] <a href="https://lore.kernel.org/git/20170426221346.25337-1-jonathantanmy@google.com/" class="bare">https://lore.kernel.org/git/20170426221346.25337-1-jonathantanmy@google.com/</a><br/>
Subject: Proposal for missing blob support in Git repos<br/>
Date: Wed, 26 Apr 2017 15:13:46 -0700</p>
</div>
<div class="paragraph">
<p>[4] <a href="https://lore.kernel.org/git/1488999039-37631-1-git-send-email-git@jeffhostetler.com/" class="bare">https://lore.kernel.org/git/1488999039-37631-1-git-send-email-git@jeffhostetler.com/</a><br/>
Subject: [PATCH 00/10] RFC Partial Clone and Fetch<br/>
Date: Wed, 8 Mar 2017 18:50:29 +0000</p>
</div>
<div class="paragraph">
<p>[5] <a href="https://lore.kernel.org/git/20170505152802.6724-1-benpeart@microsoft.com/" class="bare">https://lore.kernel.org/git/20170505152802.6724-1-benpeart@microsoft.com/</a><br/>
Subject: [PATCH v7 00/10] refactor the filter process code into a reusable module<br/>
Date: Fri, 5 May 2017 11:27:52 -0400</p>
</div>
<div class="paragraph">
<p>[6] <a href="https://lore.kernel.org/git/20170714132651.170708-1-benpeart@microsoft.com/" class="bare">https://lore.kernel.org/git/20170714132651.170708-1-benpeart@microsoft.com/</a><br/>
Subject: [RFC/PATCH v2 0/1] Add support for downloading blobs on demand<br/>
Date: Fri, 14 Jul 2017 09:26:50 -0400</p>
</div>
</div>
</div>
</div>
<div id="footer">
<div id="footer-text">
Last updated 2025-03-17 07:09:38 UTC
</div>
</div>
</body>
</html>

View File

@@ -0,0 +1,190 @@
Platform Support Policy
=======================
Git has a history of providing broad "support" for exotic platforms and older
platforms, without an explicit commitment. Stakeholders of these platforms may
want a more predictable support commitment. This is only possible when platform
stakeholders supply Git developers with adequate tooling, so we can test for
compatibility or develop workarounds for platform-specific quirks on our own.
Various levels of platform-specific tooling will allow us to make more solid
commitments around Git's compatibility with that platform.
Note that this document is about maintaining existing support for a platform
that has generally worked in the past; for adding support to a platform which
doesn't generally work with Git, the stakeholders for that platform are expected
to do the bulk of that work themselves. We will consider such patches if they
don't make life harder for other supported platforms or for Git contributors.
Some contributors may volunteer to help with the initial or continued support,
but that's not a given. Support work which is too intrusive or difficult for the
project to maintain may still not be accepted.
Minimum Requirements
--------------------
The rest of this doc describes best practices for platforms to make themselves
easy to support. However, before considering support at all, platforms need to
meet the following minimum requirements:
* Has C99 or C11
* Uses versions of dependencies which are generally accepted as stable and
supportable, e.g., in line with the version used by other long-term-support
distributions
* Has active security support (taking security releases of dependencies, etc)
These requirements are a starting point, and not sufficient on their own for the
Git community to be enthusiastic about supporting your platform. Maintainers of
platforms which do meet these requirements can follow the steps below to make it
more likely that Git updates will respect the platform's needs.
Compatible by next release
--------------------------
To increase probability that compatibility issues introduced in a release
will be fixed in a later release:
* You should send a bug report as soon as you notice the breakage on your
platform. The sooner you notice, the better; watching `seen` means you can
notice problems before they are considered "done with review"; whereas
watching `master` means the stable branch could break for your platform, but
you have a decent chance of avoiding a tagged release breaking you. See "The
Policy" in link:../howto/maintain-git.html["How to maintain Git"] for an
overview of which branches are used in the Git project, and how.
* The bug report should include information about what platform you are using.
* You should also use linkgit:git-bisect[1] and determine which commit
introduced the breakage.
* Please include any information you have about the nature of the breakage: is
it a memory alignment issue? Is an underlying library missing or broken for
your platform? Is there some quirk about your platform which means typical
practices (like malloc) behave strangely?
* If possible, build Git from the exact same source both for your platform and
for a mainstream platform, to see if the problem you noticed appears only
on your platform. If the problem appears in both, then it's not a
compatibility issue, but we of course appreciate hearing about it in a bug
report anyway, to benefit users of every platform. If it appears only on your
platform, mention clearly that it is a compatibility issue in your report.
* Once we begin to fix the issue, please work closely with the contributor
working on it to test the proposed fix against your platform.
Example: NonStop
https://lore.kernel.org/git/01bd01da681a$b8d70a70$2a851f50$@nexbridge.com/[reports
problems] when they're noticed.
Compatible on `master` and releases
-----------------------------------
To make sure all stable builds and regular releases work for your platform the
first time, help us avoid breaking `master` for your platform:
* You should run regular tests against the `next` branch and
publish breakage reports to the mailing list immediately when they happen.
** Ideally, these tests should run daily. They must run more often than
weekly, as topics generally spend at least 7 days in `next` before graduating
to `master`, and it takes time to put the brakes on a patch once it lands in
`next`.
** You may want to ask to join the mailto:git-security@googlegroups.com[security
mailing list] in order to run tests against the fixes proposed there, too.
* It may make sense to automate these; if you do, make sure they are not noisy
(you don't need to send a report when everything works, only when something
breaks; you don't need to send repeated reports for the same breakage night
after night).
* Breakage reports should be actionable - include clear error messages that can
help developers who may not have access to test directly on your platform.
* You should use git-bisect and determine which commit introduced the breakage;
if you can't do this with automation, you should do this yourself manually as
soon as you notice a breakage report was sent.
* You should either:
** Provide on-demand access to your platform to a trusted developer working to
fix the issue, so they can test their fix, OR
** Work closely with the developer fixing the issue; the turnaround to check
that their proposed fix works for your platform should be fast enough that it
doesn't hinder the developer working on that fix. Slow testing turnarounds
may cause the fix to miss the next release, or the developer may lose
interest in working on the fix at all.
Example:
https://lore.kernel.org/git/CAHd-oW6X4cwD_yLNFONPnXXUAFPxgDoccv2SOdpeLrqmHCJB4Q@mail.gmail.com/[AIX]
provides a build farm and runs tests against release candidates.
Compatible on `next`
--------------------
To avoid reactive debugging and fixing when changes hit a release or stable, you
can aim to ensure `next` always works for your platform. (See "The Policy" in
link:../howto/maintain-git.html["How to maintain Git"] for an overview of how
`next` is used in the Git project.) To do that:
* You should add a runner for your platform to the GitHub Actions or GitLab CI
suite. This suite is run when any Git developer proposes a new patch, and
having a runner for your platform/configuration means every developer will
know if they break you, immediately.
** If adding it to an existing CI suite is infeasible (due to architecture
constraints or for performance reasons), any other method which runs as
automatically and quickly as possible works, too. For example, a service
which snoops on the mailing list and automatically runs tests on new [PATCH]
emails, replying to the author with the results, would also be within the
spirit of this requirement.
* If you rely on Git avoiding a specific pattern that doesn't work well with
your platform (like a certain malloc pattern), raise it on the mailing list.
We'll work case-by-case to look for a solution that doesn't unnecessarily
constrain other platforms to keep compatibility with yours.
* If you rely on some configuration or behavior, add a test for it. Untested
behavior is subject to breakage at any time.
** Clearly label these tests as necessary for platform compatibility. Add them
to an isolated compatibility-related test suite, like a new t* file or unit
test suite, so that they're easy to remove when compatibility is no longer
required. If the specific compatibility need is gated behind an issue with
another project, link to documentation of that issue (like a bug or email
thread) to make it easier to tell when that compatibility need goes away.
** Include a comment with an expiration date for these tests no more than 1 year
from now. You can update the expiration date if your platform still needs
that assurance down the road, but we need to know you still care about that
compatibility case and are working to make it unnecessary.
Example: We run our
https://git.kernel.org/pub/scm/git/git.git/tree/.github/workflows/main.yml[CI
suite] on Windows, Ubuntu, Mac, and others.
Getting help writing platform support patches
---------------------------------------------
In general, when sending patches to fix platform support problems, follow
these guidelines to make sure the patch is reviewed with the appropriate level
of urgency:
* Clearly state in the commit message that you are fixing a platform breakage,
and for which platform.
* Use the CI and test suite to ensure that the fix for your platform doesn't
break other platforms.
* If possible, add a test ensuring this regression doesn't happen again. If
it's not possible to add a test, explain why in the commit message.
Platform Maintainers
--------------------
If you maintain a platform, or Git for that platform, and intend to work with
the Git project to ensure compatibility, please send a patch to add yourself to
this list.
NonStop: Randall S. Becker <rsbecker@nexbridge.com>

View File

@@ -0,0 +1,729 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="en">
<head>
<meta charset="UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=edge"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<meta name="generator" content="Asciidoctor 2.0.23"/>
<title>Platform Support Policy</title>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"/>
<style>
/*! Asciidoctor default stylesheet | MIT License | https://asciidoctor.org */
/* Uncomment the following line when using as a custom stylesheet */
/* @import "https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"; */
html{font-family:sans-serif;-webkit-text-size-adjust:100%}
a{background:none}
a:focus{outline:thin dotted}
a:active,a:hover{outline:0}
h1{font-size:2em;margin:.67em 0}
b,strong{font-weight:bold}
abbr{font-size:.9em}
abbr[title]{cursor:help;border-bottom:1px dotted #dddddf;text-decoration:none}
dfn{font-style:italic}
hr{height:0}
mark{background:#ff0;color:#000}
code,kbd,pre,samp{font-family:monospace;font-size:1em}
pre{white-space:pre-wrap}
q{quotes:"\201C" "\201D" "\2018" "\2019"}
small{font-size:80%}
sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline}
sup{top:-.5em}
sub{bottom:-.25em}
img{border:0}
svg:not(:root){overflow:hidden}
figure{margin:0}
audio,video{display:inline-block}
audio:not([controls]){display:none;height:0}
fieldset{border:1px solid silver;margin:0 2px;padding:.35em .625em .75em}
legend{border:0;padding:0}
button,input,select,textarea{font-family:inherit;font-size:100%;margin:0}
button,input{line-height:normal}
button,select{text-transform:none}
button,html input[type=button],input[type=reset],input[type=submit]{-webkit-appearance:button;cursor:pointer}
button[disabled],html input[disabled]{cursor:default}
input[type=checkbox],input[type=radio]{padding:0}
button::-moz-focus-inner,input::-moz-focus-inner{border:0;padding:0}
textarea{overflow:auto;vertical-align:top}
table{border-collapse:collapse;border-spacing:0}
*,::before,::after{box-sizing:border-box}
html,body{font-size:100%}
body{background:#fff;color:rgba(0,0,0,.8);padding:0;margin:0;font-family:"Noto Serif","DejaVu Serif",serif;line-height:1;position:relative;cursor:auto;-moz-tab-size:4;-o-tab-size:4;tab-size:4;word-wrap:anywhere;-moz-osx-font-smoothing:grayscale;-webkit-font-smoothing:antialiased}
a:hover{cursor:pointer}
img,object,embed{max-width:100%;height:auto}
object,embed{height:100%}
img{-ms-interpolation-mode:bicubic}
.left{float:left!important}
.right{float:right!important}
.text-left{text-align:left!important}
.text-right{text-align:right!important}
.text-center{text-align:center!important}
.text-justify{text-align:justify!important}
.hide{display:none}
img,object,svg{display:inline-block;vertical-align:middle}
textarea{height:auto;min-height:50px}
select{width:100%}
.subheader,.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{line-height:1.45;color:#7a2518;font-weight:400;margin-top:0;margin-bottom:.25em}
div,dl,dt,dd,ul,ol,li,h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6,pre,form,p,blockquote,th,td{margin:0;padding:0}
a{color:#2156a5;text-decoration:underline;line-height:inherit}
a:hover,a:focus{color:#1d4b8f}
a img{border:0}
p{line-height:1.6;margin-bottom:1.25em;text-rendering:optimizeLegibility}
p aside{font-size:.875em;line-height:1.35;font-style:italic}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{font-family:"Open Sans","DejaVu Sans",sans-serif;font-weight:300;font-style:normal;color:#ba3925;text-rendering:optimizeLegibility;margin-top:1em;margin-bottom:.5em;line-height:1.0125em}
h1 small,h2 small,h3 small,#toctitle small,.sidebarblock>.content>.title small,h4 small,h5 small,h6 small{font-size:60%;color:#e99b8f;line-height:0}
h1{font-size:2.125em}
h2{font-size:1.6875em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.375em}
h4,h5{font-size:1.125em}
h6{font-size:1em}
hr{border:solid #dddddf;border-width:1px 0 0;clear:both;margin:1.25em 0 1.1875em}
em,i{font-style:italic;line-height:inherit}
strong,b{font-weight:bold;line-height:inherit}
small{font-size:60%;line-height:inherit}
code{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:400;color:rgba(0,0,0,.9)}
ul,ol,dl{line-height:1.6;margin-bottom:1.25em;list-style-position:outside;font-family:inherit}
ul,ol{margin-left:1.5em}
ul li ul,ul li ol{margin-left:1.25em;margin-bottom:0}
ul.circle{list-style-type:circle}
ul.disc{list-style-type:disc}
ul.square{list-style-type:square}
ul.circle ul:not([class]),ul.disc ul:not([class]),ul.square ul:not([class]){list-style:inherit}
ol li ul,ol li ol{margin-left:1.25em;margin-bottom:0}
dl dt{margin-bottom:.3125em;font-weight:bold}
dl dd{margin-bottom:1.25em}
blockquote{margin:0 0 1.25em;padding:.5625em 1.25em 0 1.1875em;border-left:1px solid #ddd}
blockquote,blockquote p{line-height:1.6;color:rgba(0,0,0,.85)}
@media screen and (min-width:768px){h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2}
h1{font-size:2.75em}
h2{font-size:2.3125em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.6875em}
h4{font-size:1.4375em}}
table{background:#fff;margin-bottom:1.25em;border:1px solid #dedede;word-wrap:normal}
table thead,table tfoot{background:#f7f8f7}
table thead tr th,table thead tr td,table tfoot tr th,table tfoot tr td{padding:.5em .625em .625em;font-size:inherit;color:rgba(0,0,0,.8);text-align:left}
table tr th,table tr td{padding:.5625em .625em;font-size:inherit;color:rgba(0,0,0,.8)}
table tr.even,table tr.alt{background:#f8f8f7}
table thead tr th,table tfoot tr th,table tbody tr td,table tr td,table tfoot tr td{line-height:1.6}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2;word-spacing:-.05em}
h1 strong,h2 strong,h3 strong,#toctitle strong,.sidebarblock>.content>.title strong,h4 strong,h5 strong,h6 strong{font-weight:400}
.center{margin-left:auto;margin-right:auto}
.stretch{width:100%}
.clearfix::before,.clearfix::after,.float-group::before,.float-group::after{content:" ";display:table}
.clearfix::after,.float-group::after{clear:both}
:not(pre).nobreak{word-wrap:normal}
:not(pre).nowrap{white-space:nowrap}
:not(pre).pre-wrap{white-space:pre-wrap}
:not(pre):not([class^=L])>code{font-size:.9375em;font-style:normal!important;letter-spacing:0;padding:.1em .5ex;word-spacing:-.15em;background:#f7f7f8;border-radius:4px;line-height:1.45;text-rendering:optimizeSpeed}
pre{color:rgba(0,0,0,.9);font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;line-height:1.45;text-rendering:optimizeSpeed}
pre code,pre pre{color:inherit;font-size:inherit;line-height:inherit}
pre>code{display:block}
pre.nowrap,pre.nowrap pre{white-space:pre;word-wrap:normal}
em em{font-style:normal}
strong strong{font-weight:400}
.keyseq{color:rgba(51,51,51,.8)}
kbd{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;display:inline-block;color:rgba(0,0,0,.8);font-size:.65em;line-height:1.45;background:#f7f7f7;border:1px solid #ccc;border-radius:3px;box-shadow:0 1px 0 rgba(0,0,0,.2),inset 0 0 0 .1em #fff;margin:0 .15em;padding:.2em .5em;vertical-align:middle;position:relative;top:-.1em;white-space:nowrap}
.keyseq kbd:first-child{margin-left:0}
.keyseq kbd:last-child{margin-right:0}
.menuseq,.menuref{color:#000}
.menuseq b:not(.caret),.menuref{font-weight:inherit}
.menuseq{word-spacing:-.02em}
.menuseq b.caret{font-size:1.25em;line-height:.8}
.menuseq i.caret{font-weight:bold;text-align:center;width:.45em}
b.button::before,b.button::after{position:relative;top:-1px;font-weight:400}
b.button::before{content:"[";padding:0 3px 0 2px}
b.button::after{content:"]";padding:0 2px 0 3px}
p a>code:hover{color:rgba(0,0,0,.9)}
#header,#content,#footnotes,#footer{width:100%;margin:0 auto;max-width:62.5em;*zoom:1;position:relative;padding-left:.9375em;padding-right:.9375em}
#header::before,#header::after,#content::before,#content::after,#footnotes::before,#footnotes::after,#footer::before,#footer::after{content:" ";display:table}
#header::after,#content::after,#footnotes::after,#footer::after{clear:both}
#content{margin-top:1.25em}
#content::before{content:none}
#header>h1:first-child{color:rgba(0,0,0,.85);margin-top:2.25rem;margin-bottom:0}
#header>h1:first-child+#toc{margin-top:8px;border-top:1px solid #dddddf}
#header>h1:only-child{border-bottom:1px solid #dddddf;padding-bottom:8px}
#header .details{border-bottom:1px solid #dddddf;line-height:1.45;padding-top:.25em;padding-bottom:.25em;padding-left:.25em;color:rgba(0,0,0,.6);display:flex;flex-flow:row wrap}
#header .details span:first-child{margin-left:-.125em}
#header .details span.email a{color:rgba(0,0,0,.85)}
#header .details br{display:none}
#header .details br+span::before{content:"\00a0\2013\00a0"}
#header .details br+span.author::before{content:"\00a0\22c5\00a0";color:rgba(0,0,0,.85)}
#header .details br+span#revremark::before{content:"\00a0|\00a0"}
#header #revnumber{text-transform:capitalize}
#header #revnumber::after{content:"\00a0"}
#content>h1:first-child:not([class]){color:rgba(0,0,0,.85);border-bottom:1px solid #dddddf;padding-bottom:8px;margin-top:0;padding-top:1rem;margin-bottom:1.25rem}
#toc{border-bottom:1px solid #e7e7e9;padding-bottom:.5em}
#toc>ul{margin-left:.125em}
#toc ul.sectlevel0>li>a{font-style:italic}
#toc ul.sectlevel0 ul.sectlevel1{margin:.5em 0}
#toc ul{font-family:"Open Sans","DejaVu Sans",sans-serif;list-style-type:none}
#toc li{line-height:1.3334;margin-top:.3334em}
#toc a{text-decoration:none}
#toc a:active{text-decoration:underline}
#toctitle{color:#7a2518;font-size:1.2em}
@media screen and (min-width:768px){#toctitle{font-size:1.375em}
body.toc2{padding-left:15em;padding-right:0}
body.toc2 #header>h1:nth-last-child(2){border-bottom:1px solid #dddddf;padding-bottom:8px}
#toc.toc2{margin-top:0!important;background:#f8f8f7;position:fixed;width:15em;left:0;top:0;border-right:1px solid #e7e7e9;border-top-width:0!important;border-bottom-width:0!important;z-index:1000;padding:1.25em 1em;height:100%;overflow:auto}
#toc.toc2 #toctitle{margin-top:0;margin-bottom:.8rem;font-size:1.2em}
#toc.toc2>ul{font-size:.9em;margin-bottom:0}
#toc.toc2 ul ul{margin-left:0;padding-left:1em}
#toc.toc2 ul.sectlevel0 ul.sectlevel1{padding-left:0;margin-top:.5em;margin-bottom:.5em}
body.toc2.toc-right{padding-left:0;padding-right:15em}
body.toc2.toc-right #toc.toc2{border-right-width:0;border-left:1px solid #e7e7e9;left:auto;right:0}}
@media screen and (min-width:1280px){body.toc2{padding-left:20em;padding-right:0}
#toc.toc2{width:20em}
#toc.toc2 #toctitle{font-size:1.375em}
#toc.toc2>ul{font-size:.95em}
#toc.toc2 ul ul{padding-left:1.25em}
body.toc2.toc-right{padding-left:0;padding-right:20em}}
#content #toc{border:1px solid #e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;border-radius:4px}
#content #toc>:first-child{margin-top:0}
#content #toc>:last-child{margin-bottom:0}
#footer{max-width:none;background:rgba(0,0,0,.8);padding:1.25em}
#footer-text{color:hsla(0,0%,100%,.8);line-height:1.44}
#content{margin-bottom:.625em}
.sect1{padding-bottom:.625em}
@media screen and (min-width:768px){#content{margin-bottom:1.25em}
.sect1{padding-bottom:1.25em}}
.sect1:last-child{padding-bottom:0}
.sect1+.sect1{border-top:1px solid #e7e7e9}
#content h1>a.anchor,h2>a.anchor,h3>a.anchor,#toctitle>a.anchor,.sidebarblock>.content>.title>a.anchor,h4>a.anchor,h5>a.anchor,h6>a.anchor{position:absolute;z-index:1001;width:1.5ex;margin-left:-1.5ex;display:block;text-decoration:none!important;visibility:hidden;text-align:center;font-weight:400}
#content h1>a.anchor::before,h2>a.anchor::before,h3>a.anchor::before,#toctitle>a.anchor::before,.sidebarblock>.content>.title>a.anchor::before,h4>a.anchor::before,h5>a.anchor::before,h6>a.anchor::before{content:"\00A7";font-size:.85em;display:block;padding-top:.1em}
#content h1:hover>a.anchor,#content h1>a.anchor:hover,h2:hover>a.anchor,h2>a.anchor:hover,h3:hover>a.anchor,#toctitle:hover>a.anchor,.sidebarblock>.content>.title:hover>a.anchor,h3>a.anchor:hover,#toctitle>a.anchor:hover,.sidebarblock>.content>.title>a.anchor:hover,h4:hover>a.anchor,h4>a.anchor:hover,h5:hover>a.anchor,h5>a.anchor:hover,h6:hover>a.anchor,h6>a.anchor:hover{visibility:visible}
#content h1>a.link,h2>a.link,h3>a.link,#toctitle>a.link,.sidebarblock>.content>.title>a.link,h4>a.link,h5>a.link,h6>a.link{color:#ba3925;text-decoration:none}
#content h1>a.link:hover,h2>a.link:hover,h3>a.link:hover,#toctitle>a.link:hover,.sidebarblock>.content>.title>a.link:hover,h4>a.link:hover,h5>a.link:hover,h6>a.link:hover{color:#a53221}
details,.audioblock,.imageblock,.literalblock,.listingblock,.stemblock,.videoblock{margin-bottom:1.25em}
details{margin-left:1.25rem}
details>summary{cursor:pointer;display:block;position:relative;line-height:1.6;margin-bottom:.625rem;outline:none;-webkit-tap-highlight-color:transparent}
details>summary::-webkit-details-marker{display:none}
details>summary::before{content:"";border:solid transparent;border-left:solid;border-width:.3em 0 .3em .5em;position:absolute;top:.5em;left:-1.25rem;transform:translateX(15%)}
details[open]>summary::before{border:solid transparent;border-top:solid;border-width:.5em .3em 0;transform:translateY(15%)}
details>summary::after{content:"";width:1.25rem;height:1em;position:absolute;top:.3em;left:-1.25rem}
.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{text-rendering:optimizeLegibility;text-align:left;font-family:"Noto Serif","DejaVu Serif",serif;font-size:1rem;font-style:italic}
table.tableblock.fit-content>caption.title{white-space:nowrap;width:0}
.paragraph.lead>p,#preamble>.sectionbody>[class=paragraph]:first-of-type p{font-size:1.21875em;line-height:1.6;color:rgba(0,0,0,.85)}
.admonitionblock>table{border-collapse:separate;border:0;background:none;width:100%}
.admonitionblock>table td.icon{text-align:center;width:80px}
.admonitionblock>table td.icon img{max-width:none}
.admonitionblock>table td.icon .title{font-weight:bold;font-family:"Open Sans","DejaVu Sans",sans-serif;text-transform:uppercase}
.admonitionblock>table td.content{padding-left:1.125em;padding-right:1.25em;border-left:1px solid #dddddf;color:rgba(0,0,0,.6);word-wrap:anywhere}
.admonitionblock>table td.content>:last-child>:last-child{margin-bottom:0}
.exampleblock>.content{border:1px solid #e6e6e6;margin-bottom:1.25em;padding:1.25em;background:#fff;border-radius:4px}
.sidebarblock{border:1px solid #dbdbd6;margin-bottom:1.25em;padding:1.25em;background:#f3f3f2;border-radius:4px}
.sidebarblock>.content>.title{color:#7a2518;margin-top:0;text-align:center}
.exampleblock>.content>:first-child,.sidebarblock>.content>:first-child{margin-top:0}
.exampleblock>.content>:last-child,.exampleblock>.content>:last-child>:last-child,.exampleblock>.content .olist>ol>li:last-child>:last-child,.exampleblock>.content .ulist>ul>li:last-child>:last-child,.exampleblock>.content .qlist>ol>li:last-child>:last-child,.sidebarblock>.content>:last-child,.sidebarblock>.content>:last-child>:last-child,.sidebarblock>.content .olist>ol>li:last-child>:last-child,.sidebarblock>.content .ulist>ul>li:last-child>:last-child,.sidebarblock>.content .qlist>ol>li:last-child>:last-child{margin-bottom:0}
.literalblock pre,.listingblock>.content>pre{border-radius:4px;overflow-x:auto;padding:1em;font-size:.8125em}
@media screen and (min-width:768px){.literalblock pre,.listingblock>.content>pre{font-size:.90625em}}
@media screen and (min-width:1280px){.literalblock pre,.listingblock>.content>pre{font-size:1em}}
.literalblock pre,.listingblock>.content>pre:not(.highlight),.listingblock>.content>pre[class=highlight],.listingblock>.content>pre[class^="highlight "]{background:#f7f7f8}
.literalblock.output pre{color:#f7f7f8;background:rgba(0,0,0,.9)}
.listingblock>.content{position:relative}
.listingblock code[data-lang]::before{display:none;content:attr(data-lang);position:absolute;font-size:.75em;top:.425rem;right:.5rem;line-height:1;text-transform:uppercase;color:inherit;opacity:.5}
.listingblock:hover code[data-lang]::before{display:block}
.listingblock.terminal pre .command::before{content:attr(data-prompt);padding-right:.5em;color:inherit;opacity:.5}
.listingblock.terminal pre .command:not([data-prompt])::before{content:"$"}
.listingblock pre.highlightjs{padding:0}
.listingblock pre.highlightjs>code{padding:1em;border-radius:4px}
.listingblock pre.prettyprint{border-width:0}
.prettyprint{background:#f7f7f8}
pre.prettyprint .linenums{line-height:1.45;margin-left:2em}
pre.prettyprint li{background:none;list-style-type:inherit;padding-left:0}
pre.prettyprint li code[data-lang]::before{opacity:1}
pre.prettyprint li:not(:first-child) code[data-lang]::before{display:none}
table.linenotable{border-collapse:separate;border:0;margin-bottom:0;background:none}
table.linenotable td[class]{color:inherit;vertical-align:top;padding:0;line-height:inherit;white-space:normal}
table.linenotable td.code{padding-left:.75em}
table.linenotable td.linenos,pre.pygments .linenos{border-right:1px solid;opacity:.35;padding-right:.5em;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}
pre.pygments span.linenos{display:inline-block;margin-right:.75em}
.quoteblock{margin:0 1em 1.25em 1.5em;display:table}
.quoteblock:not(.excerpt)>.title{margin-left:-1.5em;margin-bottom:.75em}
.quoteblock blockquote,.quoteblock p{color:rgba(0,0,0,.85);font-size:1.15rem;line-height:1.75;word-spacing:.1em;letter-spacing:0;font-style:italic;text-align:justify}
.quoteblock blockquote{margin:0;padding:0;border:0}
.quoteblock blockquote::before{content:"\201c";float:left;font-size:2.75em;font-weight:bold;line-height:.6em;margin-left:-.6em;color:#7a2518;text-shadow:0 1px 2px rgba(0,0,0,.1)}
.quoteblock blockquote>.paragraph:last-child p{margin-bottom:0}
.quoteblock .attribution{margin-top:.75em;margin-right:.5ex;text-align:right}
.verseblock{margin:0 1em 1.25em}
.verseblock pre{font-family:"Open Sans","DejaVu Sans",sans-serif;font-size:1.15rem;color:rgba(0,0,0,.85);font-weight:300;text-rendering:optimizeLegibility}
.verseblock pre strong{font-weight:400}
.verseblock .attribution{margin-top:1.25rem;margin-left:.5ex}
.quoteblock .attribution,.verseblock .attribution{font-size:.9375em;line-height:1.45;font-style:italic}
.quoteblock .attribution br,.verseblock .attribution br{display:none}
.quoteblock .attribution cite,.verseblock .attribution cite{display:block;letter-spacing:-.025em;color:rgba(0,0,0,.6)}
.quoteblock.abstract blockquote::before,.quoteblock.excerpt blockquote::before,.quoteblock .quoteblock blockquote::before{display:none}
.quoteblock.abstract blockquote,.quoteblock.abstract p,.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{line-height:1.6;word-spacing:0}
.quoteblock.abstract{margin:0 1em 1.25em;display:block}
.quoteblock.abstract>.title{margin:0 0 .375em;font-size:1.15em;text-align:center}
.quoteblock.excerpt>blockquote,.quoteblock .quoteblock{padding:0 0 .25em 1em;border-left:.25em solid #dddddf}
.quoteblock.excerpt,.quoteblock .quoteblock{margin-left:0}
.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{color:inherit;font-size:1.0625rem}
.quoteblock.excerpt .attribution,.quoteblock .quoteblock .attribution{color:inherit;font-size:.85rem;text-align:left;margin-right:0}
p.tableblock:last-child{margin-bottom:0}
td.tableblock>.content{margin-bottom:1.25em;word-wrap:anywhere}
td.tableblock>.content>:last-child{margin-bottom:-1.25em}
table.tableblock,th.tableblock,td.tableblock{border:0 solid #dedede}
table.grid-all>*>tr>*{border-width:1px}
table.grid-cols>*>tr>*{border-width:0 1px}
table.grid-rows>*>tr>*{border-width:1px 0}
table.frame-all{border-width:1px}
table.frame-ends{border-width:1px 0}
table.frame-sides{border-width:0 1px}
table.frame-none>colgroup+*>:first-child>*,table.frame-sides>colgroup+*>:first-child>*{border-top-width:0}
table.frame-none>:last-child>:last-child>*,table.frame-sides>:last-child>:last-child>*{border-bottom-width:0}
table.frame-none>*>tr>:first-child,table.frame-ends>*>tr>:first-child{border-left-width:0}
table.frame-none>*>tr>:last-child,table.frame-ends>*>tr>:last-child{border-right-width:0}
table.stripes-all>*>tr,table.stripes-odd>*>tr:nth-of-type(odd),table.stripes-even>*>tr:nth-of-type(even),table.stripes-hover>*>tr:hover{background:#f8f8f7}
th.halign-left,td.halign-left{text-align:left}
th.halign-right,td.halign-right{text-align:right}
th.halign-center,td.halign-center{text-align:center}
th.valign-top,td.valign-top{vertical-align:top}
th.valign-bottom,td.valign-bottom{vertical-align:bottom}
th.valign-middle,td.valign-middle{vertical-align:middle}
table thead th,table tfoot th{font-weight:bold}
tbody tr th{background:#f7f8f7}
tbody tr th,tbody tr th p,tfoot tr th,tfoot tr th p{color:rgba(0,0,0,.8);font-weight:bold}
p.tableblock>code:only-child{background:none;padding:0}
p.tableblock{font-size:1em}
ol{margin-left:1.75em}
ul li ol{margin-left:1.5em}
dl dd{margin-left:1.125em}
dl dd:last-child,dl dd:last-child>:last-child{margin-bottom:0}
li p,ul dd,ol dd,.olist .olist,.ulist .ulist,.ulist .olist,.olist .ulist{margin-bottom:.625em}
ul.checklist,ul.none,ol.none,ul.no-bullet,ol.no-bullet,ol.unnumbered,ul.unstyled,ol.unstyled{list-style-type:none}
ul.no-bullet,ol.no-bullet,ol.unnumbered{margin-left:.625em}
ul.unstyled,ol.unstyled{margin-left:0}
li>p:empty:only-child::before{content:"";display:inline-block}
ul.checklist>li>p:first-child{margin-left:-1em}
ul.checklist>li>p:first-child>.fa-square-o:first-child,ul.checklist>li>p:first-child>.fa-check-square-o:first-child{width:1.25em;font-size:.8em;position:relative;bottom:.125em}
ul.checklist>li>p:first-child>input[type=checkbox]:first-child{margin-right:.25em}
ul.inline{display:flex;flex-flow:row wrap;list-style:none;margin:0 0 .625em -1.25em}
ul.inline>li{margin-left:1.25em}
.unstyled dl dt{font-weight:400;font-style:normal}
ol.arabic{list-style-type:decimal}
ol.decimal{list-style-type:decimal-leading-zero}
ol.loweralpha{list-style-type:lower-alpha}
ol.upperalpha{list-style-type:upper-alpha}
ol.lowerroman{list-style-type:lower-roman}
ol.upperroman{list-style-type:upper-roman}
ol.lowergreek{list-style-type:lower-greek}
.hdlist>table,.colist>table{border:0;background:none}
.hdlist>table>tbody>tr,.colist>table>tbody>tr{background:none}
td.hdlist1,td.hdlist2{vertical-align:top;padding:0 .625em}
td.hdlist1{font-weight:bold;padding-bottom:1.25em}
td.hdlist2{word-wrap:anywhere}
.literalblock+.colist,.listingblock+.colist{margin-top:-.5em}
.colist td:not([class]):first-child{padding:.4em .75em 0;line-height:1;vertical-align:top}
.colist td:not([class]):first-child img{max-width:none}
.colist td:not([class]):last-child{padding:.25em 0}
.thumb,.th{line-height:0;display:inline-block;border:4px solid #fff;box-shadow:0 0 0 1px #ddd}
.imageblock.left{margin:.25em .625em 1.25em 0}
.imageblock.right{margin:.25em 0 1.25em .625em}
.imageblock>.title{margin-bottom:0}
.imageblock.thumb,.imageblock.th{border-width:6px}
.imageblock.thumb>.title,.imageblock.th>.title{padding:0 .125em}
.image.left,.image.right{margin-top:.25em;margin-bottom:.25em;display:inline-block;line-height:0}
.image.left{margin-right:.625em}
.image.right{margin-left:.625em}
a.image{text-decoration:none;display:inline-block}
a.image object{pointer-events:none}
sup.footnote,sup.footnoteref{font-size:.875em;position:static;vertical-align:super}
sup.footnote a,sup.footnoteref a{text-decoration:none}
sup.footnote a:active,sup.footnoteref a:active,#footnotes .footnote a:first-of-type:active{text-decoration:underline}
#footnotes{padding-top:.75em;padding-bottom:.75em;margin-bottom:.625em}
#footnotes hr{width:20%;min-width:6.25em;margin:-.25em 0 .75em;border-width:1px 0 0}
#footnotes .footnote{padding:0 .375em 0 .225em;line-height:1.3334;font-size:.875em;margin-left:1.2em;margin-bottom:.2em}
#footnotes .footnote a:first-of-type{font-weight:bold;text-decoration:none;margin-left:-1.05em}
#footnotes .footnote:last-of-type{margin-bottom:0}
#content #footnotes{margin-top:-.625em;margin-bottom:0;padding:.75em 0}
div.unbreakable{page-break-inside:avoid}
.big{font-size:larger}
.small{font-size:smaller}
.underline{text-decoration:underline}
.overline{text-decoration:overline}
.line-through{text-decoration:line-through}
.aqua{color:#00bfbf}
.aqua-background{background:#00fafa}
.black{color:#000}
.black-background{background:#000}
.blue{color:#0000bf}
.blue-background{background:#0000fa}
.fuchsia{color:#bf00bf}
.fuchsia-background{background:#fa00fa}
.gray{color:#606060}
.gray-background{background:#7d7d7d}
.green{color:#006000}
.green-background{background:#007d00}
.lime{color:#00bf00}
.lime-background{background:#00fa00}
.maroon{color:#600000}
.maroon-background{background:#7d0000}
.navy{color:#000060}
.navy-background{background:#00007d}
.olive{color:#606000}
.olive-background{background:#7d7d00}
.purple{color:#600060}
.purple-background{background:#7d007d}
.red{color:#bf0000}
.red-background{background:#fa0000}
.silver{color:#909090}
.silver-background{background:#bcbcbc}
.teal{color:#006060}
.teal-background{background:#007d7d}
.white{color:#bfbfbf}
.white-background{background:#fafafa}
.yellow{color:#bfbf00}
.yellow-background{background:#fafa00}
span.icon>.fa{cursor:default}
a span.icon>.fa{cursor:inherit}
.admonitionblock td.icon [class^="fa icon-"]{font-size:2.5em;text-shadow:1px 1px 2px rgba(0,0,0,.5);cursor:default}
.admonitionblock td.icon .icon-note::before{content:"\f05a";color:#19407c}
.admonitionblock td.icon .icon-tip::before{content:"\f0eb";text-shadow:1px 1px 2px rgba(155,155,0,.8);color:#111}
.admonitionblock td.icon .icon-warning::before{content:"\f071";color:#bf6900}
.admonitionblock td.icon .icon-caution::before{content:"\f06d";color:#bf3400}
.admonitionblock td.icon .icon-important::before{content:"\f06a";color:#bf0000}
.conum[data-value]{display:inline-block;color:#fff!important;background:rgba(0,0,0,.8);border-radius:50%;text-align:center;font-size:.75em;width:1.67em;height:1.67em;line-height:1.67em;font-family:"Open Sans","DejaVu Sans",sans-serif;font-style:normal;font-weight:bold}
.conum[data-value] *{color:#fff!important}
.conum[data-value]+b{display:none}
.conum[data-value]::after{content:attr(data-value)}
pre .conum[data-value]{position:relative;top:-.125em}
b.conum *{color:inherit!important}
.conum:not([data-value]):empty{display:none}
dt,th.tableblock,td.content,div.footnote{text-rendering:optimizeLegibility}
h1,h2,p,td.content,span.alt,summary{letter-spacing:-.01em}
p strong,td.content strong,div.footnote strong{letter-spacing:-.005em}
p,blockquote,dt,td.content,td.hdlist1,span.alt,summary{font-size:1.0625rem}
p{margin-bottom:1.25rem}
.sidebarblock p,.sidebarblock dt,.sidebarblock td.content,p.tableblock{font-size:1em}
.exampleblock>.content{background:#fffef7;border-color:#e0e0dc;box-shadow:0 1px 4px #e0e0dc}
.print-only{display:none!important}
@page{margin:1.25cm .75cm}
@media print{*{box-shadow:none!important;text-shadow:none!important}
html{font-size:80%}
a{color:inherit!important;text-decoration:underline!important}
a.bare,a[href^="#"],a[href^="mailto:"]{text-decoration:none!important}
a[href^="http:"]:not(.bare)::after,a[href^="https:"]:not(.bare)::after{content:"(" attr(href) ")";display:inline-block;font-size:.875em;padding-left:.25em}
abbr[title]{border-bottom:1px dotted}
abbr[title]::after{content:" (" attr(title) ")"}
pre,blockquote,tr,img,object,svg{page-break-inside:avoid}
thead{display:table-header-group}
svg{max-width:100%}
p,blockquote,dt,td.content{font-size:1em;orphans:3;widows:3}
h2,h3,#toctitle,.sidebarblock>.content>.title{page-break-after:avoid}
#header,#content,#footnotes,#footer{max-width:none}
#toc,.sidebarblock,.exampleblock>.content{background:none!important}
#toc{border-bottom:1px solid #dddddf!important;padding-bottom:0!important}
body.book #header{text-align:center}
body.book #header>h1:first-child{border:0!important;margin:2.5em 0 1em}
body.book #header .details{border:0!important;display:block;padding:0!important}
body.book #header .details span:first-child{margin-left:0!important}
body.book #header .details br{display:block}
body.book #header .details br+span::before{content:none!important}
body.book #toc{border:0!important;text-align:left!important;padding:0!important;margin:0!important}
body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-break-before:always}
.listingblock code[data-lang]::before{display:block}
#footer{padding:0 .9375em}
.hide-on-print{display:none!important}
.print-only{display:block!important}
.hide-for-print{display:none!important}
.show-for-print{display:inherit!important}}
@media amzn-kf8,print{#header>h1:first-child{margin-top:1.25rem}
.sect1{padding:0!important}
.sect1+.sect1{border:0}
#footer{background:none}
#footer-text{color:rgba(0,0,0,.6);font-size:.9em}}
@media amzn-kf8{#header,#content,#footnotes,#footer{padding:0}}
</style>
</head>
<body class="article">
<div id="header">
<h1>Platform Support Policy</h1>
</div>
<div id="content">
<div id="preamble">
<div class="sectionbody">
<div class="paragraph">
<p>Git has a history of providing broad "support" for exotic platforms and older
platforms, without an explicit commitment. Stakeholders of these platforms may
want a more predictable support commitment. This is only possible when platform
stakeholders supply Git developers with adequate tooling, so we can test for
compatibility or develop workarounds for platform-specific quirks on our own.
Various levels of platform-specific tooling will allow us to make more solid
commitments around Git&#8217;s compatibility with that platform.</p>
</div>
<div class="paragraph">
<p>Note that this document is about maintaining existing support for a platform
that has generally worked in the past; for adding support to a platform which
doesn&#8217;t generally work with Git, the stakeholders for that platform are expected
to do the bulk of that work themselves. We will consider such patches if they
don&#8217;t make life harder for other supported platforms or for Git contributors.
Some contributors may volunteer to help with the initial or continued support,
but that&#8217;s not a given. Support work which is too intrusive or difficult for the
project to maintain may still not be accepted.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_minimum_requirements">Minimum Requirements</h2>
<div class="sectionbody">
<div class="paragraph">
<p>The rest of this doc describes best practices for platforms to make themselves
easy to support. However, before considering support at all, platforms need to
meet the following minimum requirements:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Has C99 or C11</p>
</li>
<li>
<p>Uses versions of dependencies which are generally accepted as stable and
supportable, e.g., in line with the version used by other long-term-support
distributions</p>
</li>
<li>
<p>Has active security support (taking security releases of dependencies, etc)</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>These requirements are a starting point, and not sufficient on their own for the
Git community to be enthusiastic about supporting your platform. Maintainers of
platforms which do meet these requirements can follow the steps below to make it
more likely that Git updates will respect the platform&#8217;s needs.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_compatible_by_next_release">Compatible by next release</h2>
<div class="sectionbody">
<div class="paragraph">
<p>To increase probability that compatibility issues introduced in a release
will be fixed in a later release:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>You should send a bug report as soon as you notice the breakage on your
platform. The sooner you notice, the better; watching <code>seen</code> means you can
notice problems before they are considered "done with review"; whereas
watching <code>master</code> means the stable branch could break for your platform, but
you have a decent chance of avoiding a tagged release breaking you. See "The
Policy" in <a href="../howto/maintain-git.html">"How to maintain Git"</a> for an
overview of which branches are used in the Git project, and how.</p>
</li>
<li>
<p>The bug report should include information about what platform you are using.</p>
</li>
<li>
<p>You should also use <a href="../git-bisect.html">git-bisect(1)</a> and determine which commit
introduced the breakage.</p>
</li>
<li>
<p>Please include any information you have about the nature of the breakage: is
it a memory alignment issue? Is an underlying library missing or broken for
your platform? Is there some quirk about your platform which means typical
practices (like malloc) behave strangely?</p>
</li>
<li>
<p>If possible, build Git from the exact same source both for your platform and
for a mainstream platform, to see if the problem you noticed appears only
on your platform. If the problem appears in both, then it&#8217;s not a
compatibility issue, but we of course appreciate hearing about it in a bug
report anyway, to benefit users of every platform. If it appears only on your
platform, mention clearly that it is a compatibility issue in your report.</p>
</li>
<li>
<p>Once we begin to fix the issue, please work closely with the contributor
working on it to test the proposed fix against your platform.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Example: NonStop
<a href="https://lore.kernel.org/git/01bd01da681a$b8d70a70$2a851f50$@nexbridge.com/">reports
problems</a> when they&#8217;re noticed.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_compatible_on_master_and_releases">Compatible on <code>master</code> and releases</h2>
<div class="sectionbody">
<div class="paragraph">
<p>To make sure all stable builds and regular releases work for your platform the
first time, help us avoid breaking <code>master</code> for your platform:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>You should run regular tests against the <code>next</code> branch and
publish breakage reports to the mailing list immediately when they happen.</p>
<div class="ulist">
<ul>
<li>
<p>Ideally, these tests should run daily. They must run more often than
weekly, as topics generally spend at least 7 days in <code>next</code> before graduating
to <code>master</code>, and it takes time to put the brakes on a patch once it lands in
<code>next</code>.</p>
</li>
<li>
<p>You may want to ask to join the <a href="mailto:git-security@googlegroups.com">security
mailing list</a> in order to run tests against the fixes proposed there, too.</p>
</li>
</ul>
</div>
</li>
<li>
<p>It may make sense to automate these; if you do, make sure they are not noisy
(you don&#8217;t need to send a report when everything works, only when something
breaks; you don&#8217;t need to send repeated reports for the same breakage night
after night).</p>
</li>
<li>
<p>Breakage reports should be actionable - include clear error messages that can
help developers who may not have access to test directly on your platform.</p>
</li>
<li>
<p>You should use git-bisect and determine which commit introduced the breakage;
if you can&#8217;t do this with automation, you should do this yourself manually as
soon as you notice a breakage report was sent.</p>
</li>
<li>
<p>You should either:</p>
<div class="ulist">
<ul>
<li>
<p>Provide on-demand access to your platform to a trusted developer working to
fix the issue, so they can test their fix, OR</p>
</li>
<li>
<p>Work closely with the developer fixing the issue; the turnaround to check
that their proposed fix works for your platform should be fast enough that it
doesn&#8217;t hinder the developer working on that fix. Slow testing turnarounds
may cause the fix to miss the next release, or the developer may lose
interest in working on the fix at all.</p>
</li>
</ul>
</div>
</li>
</ul>
</div>
<div class="paragraph">
<p>Example:
<a href="https://lore.kernel.org/git/CAHd-oW6X4cwD_yLNFONPnXXUAFPxgDoccv2SOdpeLrqmHCJB4Q@mail.gmail.com/">AIX</a>
provides a build farm and runs tests against release candidates.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_compatible_on_next">Compatible on <code>next</code></h2>
<div class="sectionbody">
<div class="paragraph">
<p>To avoid reactive debugging and fixing when changes hit a release or stable, you
can aim to ensure <code>next</code> always works for your platform. (See "The Policy" in
<a href="../howto/maintain-git.html">"How to maintain Git"</a> for an overview of how
<code>next</code> is used in the Git project.) To do that:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>You should add a runner for your platform to the GitHub Actions or GitLab CI
suite. This suite is run when any Git developer proposes a new patch, and
having a runner for your platform/configuration means every developer will
know if they break you, immediately.</p>
<div class="ulist">
<ul>
<li>
<p>If adding it to an existing CI suite is infeasible (due to architecture
constraints or for performance reasons), any other method which runs as
automatically and quickly as possible works, too. For example, a service
which snoops on the mailing list and automatically runs tests on new [PATCH]
emails, replying to the author with the results, would also be within the
spirit of this requirement.</p>
</li>
</ul>
</div>
</li>
<li>
<p>If you rely on Git avoiding a specific pattern that doesn&#8217;t work well with
your platform (like a certain malloc pattern), raise it on the mailing list.
We&#8217;ll work case-by-case to look for a solution that doesn&#8217;t unnecessarily
constrain other platforms to keep compatibility with yours.</p>
</li>
<li>
<p>If you rely on some configuration or behavior, add a test for it. Untested
behavior is subject to breakage at any time.</p>
<div class="ulist">
<ul>
<li>
<p>Clearly label these tests as necessary for platform compatibility. Add them
to an isolated compatibility-related test suite, like a new t* file or unit
test suite, so that they&#8217;re easy to remove when compatibility is no longer
required. If the specific compatibility need is gated behind an issue with
another project, link to documentation of that issue (like a bug or email
thread) to make it easier to tell when that compatibility need goes away.</p>
</li>
<li>
<p>Include a comment with an expiration date for these tests no more than 1 year
from now. You can update the expiration date if your platform still needs
that assurance down the road, but we need to know you still care about that
compatibility case and are working to make it unnecessary.</p>
</li>
</ul>
</div>
</li>
</ul>
</div>
<div class="paragraph">
<p>Example: We run our
<a href="https://git.kernel.org/pub/scm/git/git.git/tree/.github/workflows/main.yml">CI
suite</a> on Windows, Ubuntu, Mac, and others.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_getting_help_writing_platform_support_patches">Getting help writing platform support patches</h2>
<div class="sectionbody">
<div class="paragraph">
<p>In general, when sending patches to fix platform support problems, follow
these guidelines to make sure the patch is reviewed with the appropriate level
of urgency:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Clearly state in the commit message that you are fixing a platform breakage,
and for which platform.</p>
</li>
<li>
<p>Use the CI and test suite to ensure that the fix for your platform doesn&#8217;t
break other platforms.</p>
</li>
<li>
<p>If possible, add a test ensuring this regression doesn&#8217;t happen again. If
it&#8217;s not possible to add a test, explain why in the commit message.</p>
</li>
</ul>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_platform_maintainers">Platform Maintainers</h2>
<div class="sectionbody">
<div class="paragraph">
<p>If you maintain a platform, or Git for that platform, and intend to work with
the Git project to ensure compatibility, please send a patch to add yourself to
this list.</p>
</div>
<div class="paragraph">
<p>NonStop: Randall S. Becker &lt;<a href="mailto:rsbecker@nexbridge.com">rsbecker@nexbridge.com</a>&gt;</p>
</div>
</div>
</div>
</div>
<div id="footer">
<div id="footer-text">
Last updated 2025-03-17 07:09:38 UTC
</div>
</div>
</body>
</html>

View File

@@ -0,0 +1,201 @@
Use of index and Racy Git problem
=================================
Background
----------
The index is one of the most important data structures in Git.
It represents a virtual working tree state by recording list of
paths and their object names and serves as a staging area to
write out the next tree object to be committed. The state is
"virtual" in the sense that it does not necessarily have to, and
often does not, match the files in the working tree.
There are cases where Git needs to examine the differences between the
virtual working tree state in the index and the files in the
working tree. The most obvious case is when the user asks `git
diff` (or its low level implementation, `git diff-files`) or
`git-ls-files --modified`. In addition, Git internally checks
if the files in the working tree are different from what are
recorded in the index to avoid stomping on local changes in them
during patch application, switching branches, and merging.
In order to speed up this comparison between the files in the
working tree and the index entries, the index entries record the
information obtained from the filesystem via `lstat(2)` system
call when they were last updated. When checking if they differ,
Git first runs `lstat(2)` on the files and compares the result
with this information (this is what was originally done by the
`ce_match_stat()` function, but the current code does it in
`ce_match_stat_basic()` function). If some of these "cached
stat information" fields do not match, Git can tell that the
files are modified without even looking at their contents.
Note: not all members in `struct stat` obtained via `lstat(2)`
are used for this comparison. For example, `st_atime` obviously
is not useful. Currently, Git compares the file type (regular
files vs symbolic links) and executable bits (only for regular
files) from `st_mode` member, `st_mtime` and `st_ctime`
timestamps, `st_uid`, `st_gid`, `st_ino`, and `st_size` members.
With a `USE_STDEV` compile-time option, `st_dev` is also
compared, but this is not enabled by default because this member
is not stable on network filesystems. With `USE_NSEC`
compile-time option, `st_mtim.tv_nsec` and `st_ctim.tv_nsec`
members are also compared. On Linux, this is not enabled by default
because in-core timestamps can have finer granularity than
on-disk timestamps, resulting in meaningless changes when an
inode is evicted from the inode cache. See commit 8ce13b0
of git://git.kernel.org/pub/scm/linux/kernel/git/tglx/history.git
([PATCH] Sync in core time granularity with filesystems,
2005-01-04). This patch is included in kernel 2.6.11 and newer, but
only fixes the issue for file systems with exactly 1 ns or 1 s
resolution. Other file systems are still broken in current Linux
kernels (e.g. CEPH, CIFS, NTFS, UDF), see
https://lore.kernel.org/lkml/5577240D.7020309@gmail.com/
Racy Git
--------
There is one slight problem with the optimization based on the
cached stat information. Consider this sequence:
: modify 'foo'
$ git update-index 'foo'
: modify 'foo' again, in-place, without changing its size
The first `update-index` computes the object name of the
contents of file `foo` and updates the index entry for `foo`
along with the `struct stat` information. If the modification
that follows it happens very fast so that the file's `st_mtime`
timestamp does not change, after this sequence, the cached stat
information the index entry records still exactly match what you
would see in the filesystem, even though the file `foo` is now
different.
This way, Git can incorrectly think files in the working tree
are unmodified even though they actually are. This is called
the "racy Git" problem (discovered by Pasky), and the entries
that appear clean when they may not be because of this problem
are called "racily clean".
To avoid this problem, Git does two things:
. When the cached stat information says the file has not been
modified, and the `st_mtime` is the same as (or newer than)
the timestamp of the index file itself (which is the time `git
update-index foo` finished running in the above example), it
also compares the contents with the object registered in the
index entry to make sure they match.
. When the index file is updated that contains racily clean
entries, cached `st_size` information is truncated to zero
before writing a new version of the index file.
Because the index file itself is written after collecting all
the stat information from updated paths, `st_mtime` timestamp of
it is usually the same as or newer than any of the paths the
index contains. And no matter how quick the modification that
follows `git update-index foo` finishes, the resulting
`st_mtime` timestamp on `foo` cannot get a value earlier
than the index file. Therefore, index entries that can be
racily clean are limited to the ones that have the same
timestamp as the index file itself.
The callers that want to check if an index entry matches the
corresponding file in the working tree continue to call
`ce_match_stat()`, but with this change, `ce_match_stat()` uses
`ce_modified_check_fs()` to see if racily clean ones are
actually clean after comparing the cached stat information using
`ce_match_stat_basic()`.
The problem the latter solves is this sequence:
$ git update-index 'foo'
: modify 'foo' in-place without changing its size
: wait for enough time
$ git update-index 'bar'
Without the latter, the timestamp of the index file gets a newer
value, and falsely clean entry `foo` would not be caught by the
timestamp comparison check done with the former logic anymore.
The latter makes sure that the cached stat information for `foo`
would never match with the file in the working tree, so later
checks by `ce_match_stat_basic()` would report that the index entry
does not match the file and Git does not have to fall back on more
expensive `ce_modified_check_fs()`.
Runtime penalty
---------------
The runtime penalty of falling back to `ce_modified_check_fs()`
from `ce_match_stat()` can be very expensive when there are many
racily clean entries. An obvious way to artificially create
this situation is to give the same timestamp to all the files in
the working tree in a large project, run `git update-index` on
them, and give the same timestamp to the index file:
$ date >.datestamp
$ git ls-files | xargs touch -r .datestamp
$ git ls-files | git update-index --stdin
$ touch -r .datestamp .git/index
This will make all index entries racily clean. The linux project, for
example, there are over 20,000 files in the working tree. On my
Athlon 64 X2 3800+, after the above:
$ /usr/bin/time git diff-files
1.68user 0.54system 0:02.22elapsed 100%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+67111minor)pagefaults 0swaps
$ git update-index MAINTAINERS
$ /usr/bin/time git diff-files
0.02user 0.12system 0:00.14elapsed 100%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+935minor)pagefaults 0swaps
Running `git update-index` in the middle checked the racily
clean entries, and left the cached `st_mtime` for all the paths
intact because they were actually clean (so this step took about
the same amount of time as the first `git diff-files`). After
that, they are not racily clean anymore but are truly clean, so
the second invocation of `git diff-files` fully took advantage
of the cached stat information.
Avoiding runtime penalty
------------------------
In order to avoid the above runtime penalty, post 1.4.2 Git used
to have a code that made sure the index file
got a timestamp newer than the youngest files in the index when
there were many young files with the same timestamp as the
resulting index file otherwise would have by waiting
before finishing writing the index file out.
I suspected that in practice the situation where many paths in the
index are all racily clean was quite rare. The only code paths
that can record recent timestamp for large number of paths are:
. Initial `git add .` of a large project.
. `git checkout` of a large project from an empty index into an
unpopulated working tree.
Note: switching branches with `git checkout` keeps the cached
stat information of existing working tree files that are the
same between the current branch and the new branch, which are
all older than the resulting index file, and they will not
become racily clean. Only the files that are actually checked
out can become racily clean.
In a large project where raciness avoidance cost really matters,
however, the initial computation of all object names in the
index takes more than one second, and the index file is written
out after all that happens. Therefore the timestamp of the
index file will be more than one second later than the
youngest file in the working tree. This means that in these
cases there actually will not be any racily clean entry in
the resulting index.
Based on this discussion, the current code does not use the
"workaround" to avoid the runtime penalty that does not exist in
practice anymore. This was done with commit 0fc82cff on Aug 15,
2006.

View File

@@ -0,0 +1,697 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="en">
<head>
<meta charset="UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=edge"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<meta name="generator" content="Asciidoctor 2.0.23"/>
<title>Use of index and Racy Git problem</title>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"/>
<style>
/*! Asciidoctor default stylesheet | MIT License | https://asciidoctor.org */
/* Uncomment the following line when using as a custom stylesheet */
/* @import "https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"; */
html{font-family:sans-serif;-webkit-text-size-adjust:100%}
a{background:none}
a:focus{outline:thin dotted}
a:active,a:hover{outline:0}
h1{font-size:2em;margin:.67em 0}
b,strong{font-weight:bold}
abbr{font-size:.9em}
abbr[title]{cursor:help;border-bottom:1px dotted #dddddf;text-decoration:none}
dfn{font-style:italic}
hr{height:0}
mark{background:#ff0;color:#000}
code,kbd,pre,samp{font-family:monospace;font-size:1em}
pre{white-space:pre-wrap}
q{quotes:"\201C" "\201D" "\2018" "\2019"}
small{font-size:80%}
sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline}
sup{top:-.5em}
sub{bottom:-.25em}
img{border:0}
svg:not(:root){overflow:hidden}
figure{margin:0}
audio,video{display:inline-block}
audio:not([controls]){display:none;height:0}
fieldset{border:1px solid silver;margin:0 2px;padding:.35em .625em .75em}
legend{border:0;padding:0}
button,input,select,textarea{font-family:inherit;font-size:100%;margin:0}
button,input{line-height:normal}
button,select{text-transform:none}
button,html input[type=button],input[type=reset],input[type=submit]{-webkit-appearance:button;cursor:pointer}
button[disabled],html input[disabled]{cursor:default}
input[type=checkbox],input[type=radio]{padding:0}
button::-moz-focus-inner,input::-moz-focus-inner{border:0;padding:0}
textarea{overflow:auto;vertical-align:top}
table{border-collapse:collapse;border-spacing:0}
*,::before,::after{box-sizing:border-box}
html,body{font-size:100%}
body{background:#fff;color:rgba(0,0,0,.8);padding:0;margin:0;font-family:"Noto Serif","DejaVu Serif",serif;line-height:1;position:relative;cursor:auto;-moz-tab-size:4;-o-tab-size:4;tab-size:4;word-wrap:anywhere;-moz-osx-font-smoothing:grayscale;-webkit-font-smoothing:antialiased}
a:hover{cursor:pointer}
img,object,embed{max-width:100%;height:auto}
object,embed{height:100%}
img{-ms-interpolation-mode:bicubic}
.left{float:left!important}
.right{float:right!important}
.text-left{text-align:left!important}
.text-right{text-align:right!important}
.text-center{text-align:center!important}
.text-justify{text-align:justify!important}
.hide{display:none}
img,object,svg{display:inline-block;vertical-align:middle}
textarea{height:auto;min-height:50px}
select{width:100%}
.subheader,.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{line-height:1.45;color:#7a2518;font-weight:400;margin-top:0;margin-bottom:.25em}
div,dl,dt,dd,ul,ol,li,h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6,pre,form,p,blockquote,th,td{margin:0;padding:0}
a{color:#2156a5;text-decoration:underline;line-height:inherit}
a:hover,a:focus{color:#1d4b8f}
a img{border:0}
p{line-height:1.6;margin-bottom:1.25em;text-rendering:optimizeLegibility}
p aside{font-size:.875em;line-height:1.35;font-style:italic}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{font-family:"Open Sans","DejaVu Sans",sans-serif;font-weight:300;font-style:normal;color:#ba3925;text-rendering:optimizeLegibility;margin-top:1em;margin-bottom:.5em;line-height:1.0125em}
h1 small,h2 small,h3 small,#toctitle small,.sidebarblock>.content>.title small,h4 small,h5 small,h6 small{font-size:60%;color:#e99b8f;line-height:0}
h1{font-size:2.125em}
h2{font-size:1.6875em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.375em}
h4,h5{font-size:1.125em}
h6{font-size:1em}
hr{border:solid #dddddf;border-width:1px 0 0;clear:both;margin:1.25em 0 1.1875em}
em,i{font-style:italic;line-height:inherit}
strong,b{font-weight:bold;line-height:inherit}
small{font-size:60%;line-height:inherit}
code{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:400;color:rgba(0,0,0,.9)}
ul,ol,dl{line-height:1.6;margin-bottom:1.25em;list-style-position:outside;font-family:inherit}
ul,ol{margin-left:1.5em}
ul li ul,ul li ol{margin-left:1.25em;margin-bottom:0}
ul.circle{list-style-type:circle}
ul.disc{list-style-type:disc}
ul.square{list-style-type:square}
ul.circle ul:not([class]),ul.disc ul:not([class]),ul.square ul:not([class]){list-style:inherit}
ol li ul,ol li ol{margin-left:1.25em;margin-bottom:0}
dl dt{margin-bottom:.3125em;font-weight:bold}
dl dd{margin-bottom:1.25em}
blockquote{margin:0 0 1.25em;padding:.5625em 1.25em 0 1.1875em;border-left:1px solid #ddd}
blockquote,blockquote p{line-height:1.6;color:rgba(0,0,0,.85)}
@media screen and (min-width:768px){h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2}
h1{font-size:2.75em}
h2{font-size:2.3125em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.6875em}
h4{font-size:1.4375em}}
table{background:#fff;margin-bottom:1.25em;border:1px solid #dedede;word-wrap:normal}
table thead,table tfoot{background:#f7f8f7}
table thead tr th,table thead tr td,table tfoot tr th,table tfoot tr td{padding:.5em .625em .625em;font-size:inherit;color:rgba(0,0,0,.8);text-align:left}
table tr th,table tr td{padding:.5625em .625em;font-size:inherit;color:rgba(0,0,0,.8)}
table tr.even,table tr.alt{background:#f8f8f7}
table thead tr th,table tfoot tr th,table tbody tr td,table tr td,table tfoot tr td{line-height:1.6}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2;word-spacing:-.05em}
h1 strong,h2 strong,h3 strong,#toctitle strong,.sidebarblock>.content>.title strong,h4 strong,h5 strong,h6 strong{font-weight:400}
.center{margin-left:auto;margin-right:auto}
.stretch{width:100%}
.clearfix::before,.clearfix::after,.float-group::before,.float-group::after{content:" ";display:table}
.clearfix::after,.float-group::after{clear:both}
:not(pre).nobreak{word-wrap:normal}
:not(pre).nowrap{white-space:nowrap}
:not(pre).pre-wrap{white-space:pre-wrap}
:not(pre):not([class^=L])>code{font-size:.9375em;font-style:normal!important;letter-spacing:0;padding:.1em .5ex;word-spacing:-.15em;background:#f7f7f8;border-radius:4px;line-height:1.45;text-rendering:optimizeSpeed}
pre{color:rgba(0,0,0,.9);font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;line-height:1.45;text-rendering:optimizeSpeed}
pre code,pre pre{color:inherit;font-size:inherit;line-height:inherit}
pre>code{display:block}
pre.nowrap,pre.nowrap pre{white-space:pre;word-wrap:normal}
em em{font-style:normal}
strong strong{font-weight:400}
.keyseq{color:rgba(51,51,51,.8)}
kbd{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;display:inline-block;color:rgba(0,0,0,.8);font-size:.65em;line-height:1.45;background:#f7f7f7;border:1px solid #ccc;border-radius:3px;box-shadow:0 1px 0 rgba(0,0,0,.2),inset 0 0 0 .1em #fff;margin:0 .15em;padding:.2em .5em;vertical-align:middle;position:relative;top:-.1em;white-space:nowrap}
.keyseq kbd:first-child{margin-left:0}
.keyseq kbd:last-child{margin-right:0}
.menuseq,.menuref{color:#000}
.menuseq b:not(.caret),.menuref{font-weight:inherit}
.menuseq{word-spacing:-.02em}
.menuseq b.caret{font-size:1.25em;line-height:.8}
.menuseq i.caret{font-weight:bold;text-align:center;width:.45em}
b.button::before,b.button::after{position:relative;top:-1px;font-weight:400}
b.button::before{content:"[";padding:0 3px 0 2px}
b.button::after{content:"]";padding:0 2px 0 3px}
p a>code:hover{color:rgba(0,0,0,.9)}
#header,#content,#footnotes,#footer{width:100%;margin:0 auto;max-width:62.5em;*zoom:1;position:relative;padding-left:.9375em;padding-right:.9375em}
#header::before,#header::after,#content::before,#content::after,#footnotes::before,#footnotes::after,#footer::before,#footer::after{content:" ";display:table}
#header::after,#content::after,#footnotes::after,#footer::after{clear:both}
#content{margin-top:1.25em}
#content::before{content:none}
#header>h1:first-child{color:rgba(0,0,0,.85);margin-top:2.25rem;margin-bottom:0}
#header>h1:first-child+#toc{margin-top:8px;border-top:1px solid #dddddf}
#header>h1:only-child{border-bottom:1px solid #dddddf;padding-bottom:8px}
#header .details{border-bottom:1px solid #dddddf;line-height:1.45;padding-top:.25em;padding-bottom:.25em;padding-left:.25em;color:rgba(0,0,0,.6);display:flex;flex-flow:row wrap}
#header .details span:first-child{margin-left:-.125em}
#header .details span.email a{color:rgba(0,0,0,.85)}
#header .details br{display:none}
#header .details br+span::before{content:"\00a0\2013\00a0"}
#header .details br+span.author::before{content:"\00a0\22c5\00a0";color:rgba(0,0,0,.85)}
#header .details br+span#revremark::before{content:"\00a0|\00a0"}
#header #revnumber{text-transform:capitalize}
#header #revnumber::after{content:"\00a0"}
#content>h1:first-child:not([class]){color:rgba(0,0,0,.85);border-bottom:1px solid #dddddf;padding-bottom:8px;margin-top:0;padding-top:1rem;margin-bottom:1.25rem}
#toc{border-bottom:1px solid #e7e7e9;padding-bottom:.5em}
#toc>ul{margin-left:.125em}
#toc ul.sectlevel0>li>a{font-style:italic}
#toc ul.sectlevel0 ul.sectlevel1{margin:.5em 0}
#toc ul{font-family:"Open Sans","DejaVu Sans",sans-serif;list-style-type:none}
#toc li{line-height:1.3334;margin-top:.3334em}
#toc a{text-decoration:none}
#toc a:active{text-decoration:underline}
#toctitle{color:#7a2518;font-size:1.2em}
@media screen and (min-width:768px){#toctitle{font-size:1.375em}
body.toc2{padding-left:15em;padding-right:0}
body.toc2 #header>h1:nth-last-child(2){border-bottom:1px solid #dddddf;padding-bottom:8px}
#toc.toc2{margin-top:0!important;background:#f8f8f7;position:fixed;width:15em;left:0;top:0;border-right:1px solid #e7e7e9;border-top-width:0!important;border-bottom-width:0!important;z-index:1000;padding:1.25em 1em;height:100%;overflow:auto}
#toc.toc2 #toctitle{margin-top:0;margin-bottom:.8rem;font-size:1.2em}
#toc.toc2>ul{font-size:.9em;margin-bottom:0}
#toc.toc2 ul ul{margin-left:0;padding-left:1em}
#toc.toc2 ul.sectlevel0 ul.sectlevel1{padding-left:0;margin-top:.5em;margin-bottom:.5em}
body.toc2.toc-right{padding-left:0;padding-right:15em}
body.toc2.toc-right #toc.toc2{border-right-width:0;border-left:1px solid #e7e7e9;left:auto;right:0}}
@media screen and (min-width:1280px){body.toc2{padding-left:20em;padding-right:0}
#toc.toc2{width:20em}
#toc.toc2 #toctitle{font-size:1.375em}
#toc.toc2>ul{font-size:.95em}
#toc.toc2 ul ul{padding-left:1.25em}
body.toc2.toc-right{padding-left:0;padding-right:20em}}
#content #toc{border:1px solid #e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;border-radius:4px}
#content #toc>:first-child{margin-top:0}
#content #toc>:last-child{margin-bottom:0}
#footer{max-width:none;background:rgba(0,0,0,.8);padding:1.25em}
#footer-text{color:hsla(0,0%,100%,.8);line-height:1.44}
#content{margin-bottom:.625em}
.sect1{padding-bottom:.625em}
@media screen and (min-width:768px){#content{margin-bottom:1.25em}
.sect1{padding-bottom:1.25em}}
.sect1:last-child{padding-bottom:0}
.sect1+.sect1{border-top:1px solid #e7e7e9}
#content h1>a.anchor,h2>a.anchor,h3>a.anchor,#toctitle>a.anchor,.sidebarblock>.content>.title>a.anchor,h4>a.anchor,h5>a.anchor,h6>a.anchor{position:absolute;z-index:1001;width:1.5ex;margin-left:-1.5ex;display:block;text-decoration:none!important;visibility:hidden;text-align:center;font-weight:400}
#content h1>a.anchor::before,h2>a.anchor::before,h3>a.anchor::before,#toctitle>a.anchor::before,.sidebarblock>.content>.title>a.anchor::before,h4>a.anchor::before,h5>a.anchor::before,h6>a.anchor::before{content:"\00A7";font-size:.85em;display:block;padding-top:.1em}
#content h1:hover>a.anchor,#content h1>a.anchor:hover,h2:hover>a.anchor,h2>a.anchor:hover,h3:hover>a.anchor,#toctitle:hover>a.anchor,.sidebarblock>.content>.title:hover>a.anchor,h3>a.anchor:hover,#toctitle>a.anchor:hover,.sidebarblock>.content>.title>a.anchor:hover,h4:hover>a.anchor,h4>a.anchor:hover,h5:hover>a.anchor,h5>a.anchor:hover,h6:hover>a.anchor,h6>a.anchor:hover{visibility:visible}
#content h1>a.link,h2>a.link,h3>a.link,#toctitle>a.link,.sidebarblock>.content>.title>a.link,h4>a.link,h5>a.link,h6>a.link{color:#ba3925;text-decoration:none}
#content h1>a.link:hover,h2>a.link:hover,h3>a.link:hover,#toctitle>a.link:hover,.sidebarblock>.content>.title>a.link:hover,h4>a.link:hover,h5>a.link:hover,h6>a.link:hover{color:#a53221}
details,.audioblock,.imageblock,.literalblock,.listingblock,.stemblock,.videoblock{margin-bottom:1.25em}
details{margin-left:1.25rem}
details>summary{cursor:pointer;display:block;position:relative;line-height:1.6;margin-bottom:.625rem;outline:none;-webkit-tap-highlight-color:transparent}
details>summary::-webkit-details-marker{display:none}
details>summary::before{content:"";border:solid transparent;border-left:solid;border-width:.3em 0 .3em .5em;position:absolute;top:.5em;left:-1.25rem;transform:translateX(15%)}
details[open]>summary::before{border:solid transparent;border-top:solid;border-width:.5em .3em 0;transform:translateY(15%)}
details>summary::after{content:"";width:1.25rem;height:1em;position:absolute;top:.3em;left:-1.25rem}
.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{text-rendering:optimizeLegibility;text-align:left;font-family:"Noto Serif","DejaVu Serif",serif;font-size:1rem;font-style:italic}
table.tableblock.fit-content>caption.title{white-space:nowrap;width:0}
.paragraph.lead>p,#preamble>.sectionbody>[class=paragraph]:first-of-type p{font-size:1.21875em;line-height:1.6;color:rgba(0,0,0,.85)}
.admonitionblock>table{border-collapse:separate;border:0;background:none;width:100%}
.admonitionblock>table td.icon{text-align:center;width:80px}
.admonitionblock>table td.icon img{max-width:none}
.admonitionblock>table td.icon .title{font-weight:bold;font-family:"Open Sans","DejaVu Sans",sans-serif;text-transform:uppercase}
.admonitionblock>table td.content{padding-left:1.125em;padding-right:1.25em;border-left:1px solid #dddddf;color:rgba(0,0,0,.6);word-wrap:anywhere}
.admonitionblock>table td.content>:last-child>:last-child{margin-bottom:0}
.exampleblock>.content{border:1px solid #e6e6e6;margin-bottom:1.25em;padding:1.25em;background:#fff;border-radius:4px}
.sidebarblock{border:1px solid #dbdbd6;margin-bottom:1.25em;padding:1.25em;background:#f3f3f2;border-radius:4px}
.sidebarblock>.content>.title{color:#7a2518;margin-top:0;text-align:center}
.exampleblock>.content>:first-child,.sidebarblock>.content>:first-child{margin-top:0}
.exampleblock>.content>:last-child,.exampleblock>.content>:last-child>:last-child,.exampleblock>.content .olist>ol>li:last-child>:last-child,.exampleblock>.content .ulist>ul>li:last-child>:last-child,.exampleblock>.content .qlist>ol>li:last-child>:last-child,.sidebarblock>.content>:last-child,.sidebarblock>.content>:last-child>:last-child,.sidebarblock>.content .olist>ol>li:last-child>:last-child,.sidebarblock>.content .ulist>ul>li:last-child>:last-child,.sidebarblock>.content .qlist>ol>li:last-child>:last-child{margin-bottom:0}
.literalblock pre,.listingblock>.content>pre{border-radius:4px;overflow-x:auto;padding:1em;font-size:.8125em}
@media screen and (min-width:768px){.literalblock pre,.listingblock>.content>pre{font-size:.90625em}}
@media screen and (min-width:1280px){.literalblock pre,.listingblock>.content>pre{font-size:1em}}
.literalblock pre,.listingblock>.content>pre:not(.highlight),.listingblock>.content>pre[class=highlight],.listingblock>.content>pre[class^="highlight "]{background:#f7f7f8}
.literalblock.output pre{color:#f7f7f8;background:rgba(0,0,0,.9)}
.listingblock>.content{position:relative}
.listingblock code[data-lang]::before{display:none;content:attr(data-lang);position:absolute;font-size:.75em;top:.425rem;right:.5rem;line-height:1;text-transform:uppercase;color:inherit;opacity:.5}
.listingblock:hover code[data-lang]::before{display:block}
.listingblock.terminal pre .command::before{content:attr(data-prompt);padding-right:.5em;color:inherit;opacity:.5}
.listingblock.terminal pre .command:not([data-prompt])::before{content:"$"}
.listingblock pre.highlightjs{padding:0}
.listingblock pre.highlightjs>code{padding:1em;border-radius:4px}
.listingblock pre.prettyprint{border-width:0}
.prettyprint{background:#f7f7f8}
pre.prettyprint .linenums{line-height:1.45;margin-left:2em}
pre.prettyprint li{background:none;list-style-type:inherit;padding-left:0}
pre.prettyprint li code[data-lang]::before{opacity:1}
pre.prettyprint li:not(:first-child) code[data-lang]::before{display:none}
table.linenotable{border-collapse:separate;border:0;margin-bottom:0;background:none}
table.linenotable td[class]{color:inherit;vertical-align:top;padding:0;line-height:inherit;white-space:normal}
table.linenotable td.code{padding-left:.75em}
table.linenotable td.linenos,pre.pygments .linenos{border-right:1px solid;opacity:.35;padding-right:.5em;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}
pre.pygments span.linenos{display:inline-block;margin-right:.75em}
.quoteblock{margin:0 1em 1.25em 1.5em;display:table}
.quoteblock:not(.excerpt)>.title{margin-left:-1.5em;margin-bottom:.75em}
.quoteblock blockquote,.quoteblock p{color:rgba(0,0,0,.85);font-size:1.15rem;line-height:1.75;word-spacing:.1em;letter-spacing:0;font-style:italic;text-align:justify}
.quoteblock blockquote{margin:0;padding:0;border:0}
.quoteblock blockquote::before{content:"\201c";float:left;font-size:2.75em;font-weight:bold;line-height:.6em;margin-left:-.6em;color:#7a2518;text-shadow:0 1px 2px rgba(0,0,0,.1)}
.quoteblock blockquote>.paragraph:last-child p{margin-bottom:0}
.quoteblock .attribution{margin-top:.75em;margin-right:.5ex;text-align:right}
.verseblock{margin:0 1em 1.25em}
.verseblock pre{font-family:"Open Sans","DejaVu Sans",sans-serif;font-size:1.15rem;color:rgba(0,0,0,.85);font-weight:300;text-rendering:optimizeLegibility}
.verseblock pre strong{font-weight:400}
.verseblock .attribution{margin-top:1.25rem;margin-left:.5ex}
.quoteblock .attribution,.verseblock .attribution{font-size:.9375em;line-height:1.45;font-style:italic}
.quoteblock .attribution br,.verseblock .attribution br{display:none}
.quoteblock .attribution cite,.verseblock .attribution cite{display:block;letter-spacing:-.025em;color:rgba(0,0,0,.6)}
.quoteblock.abstract blockquote::before,.quoteblock.excerpt blockquote::before,.quoteblock .quoteblock blockquote::before{display:none}
.quoteblock.abstract blockquote,.quoteblock.abstract p,.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{line-height:1.6;word-spacing:0}
.quoteblock.abstract{margin:0 1em 1.25em;display:block}
.quoteblock.abstract>.title{margin:0 0 .375em;font-size:1.15em;text-align:center}
.quoteblock.excerpt>blockquote,.quoteblock .quoteblock{padding:0 0 .25em 1em;border-left:.25em solid #dddddf}
.quoteblock.excerpt,.quoteblock .quoteblock{margin-left:0}
.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{color:inherit;font-size:1.0625rem}
.quoteblock.excerpt .attribution,.quoteblock .quoteblock .attribution{color:inherit;font-size:.85rem;text-align:left;margin-right:0}
p.tableblock:last-child{margin-bottom:0}
td.tableblock>.content{margin-bottom:1.25em;word-wrap:anywhere}
td.tableblock>.content>:last-child{margin-bottom:-1.25em}
table.tableblock,th.tableblock,td.tableblock{border:0 solid #dedede}
table.grid-all>*>tr>*{border-width:1px}
table.grid-cols>*>tr>*{border-width:0 1px}
table.grid-rows>*>tr>*{border-width:1px 0}
table.frame-all{border-width:1px}
table.frame-ends{border-width:1px 0}
table.frame-sides{border-width:0 1px}
table.frame-none>colgroup+*>:first-child>*,table.frame-sides>colgroup+*>:first-child>*{border-top-width:0}
table.frame-none>:last-child>:last-child>*,table.frame-sides>:last-child>:last-child>*{border-bottom-width:0}
table.frame-none>*>tr>:first-child,table.frame-ends>*>tr>:first-child{border-left-width:0}
table.frame-none>*>tr>:last-child,table.frame-ends>*>tr>:last-child{border-right-width:0}
table.stripes-all>*>tr,table.stripes-odd>*>tr:nth-of-type(odd),table.stripes-even>*>tr:nth-of-type(even),table.stripes-hover>*>tr:hover{background:#f8f8f7}
th.halign-left,td.halign-left{text-align:left}
th.halign-right,td.halign-right{text-align:right}
th.halign-center,td.halign-center{text-align:center}
th.valign-top,td.valign-top{vertical-align:top}
th.valign-bottom,td.valign-bottom{vertical-align:bottom}
th.valign-middle,td.valign-middle{vertical-align:middle}
table thead th,table tfoot th{font-weight:bold}
tbody tr th{background:#f7f8f7}
tbody tr th,tbody tr th p,tfoot tr th,tfoot tr th p{color:rgba(0,0,0,.8);font-weight:bold}
p.tableblock>code:only-child{background:none;padding:0}
p.tableblock{font-size:1em}
ol{margin-left:1.75em}
ul li ol{margin-left:1.5em}
dl dd{margin-left:1.125em}
dl dd:last-child,dl dd:last-child>:last-child{margin-bottom:0}
li p,ul dd,ol dd,.olist .olist,.ulist .ulist,.ulist .olist,.olist .ulist{margin-bottom:.625em}
ul.checklist,ul.none,ol.none,ul.no-bullet,ol.no-bullet,ol.unnumbered,ul.unstyled,ol.unstyled{list-style-type:none}
ul.no-bullet,ol.no-bullet,ol.unnumbered{margin-left:.625em}
ul.unstyled,ol.unstyled{margin-left:0}
li>p:empty:only-child::before{content:"";display:inline-block}
ul.checklist>li>p:first-child{margin-left:-1em}
ul.checklist>li>p:first-child>.fa-square-o:first-child,ul.checklist>li>p:first-child>.fa-check-square-o:first-child{width:1.25em;font-size:.8em;position:relative;bottom:.125em}
ul.checklist>li>p:first-child>input[type=checkbox]:first-child{margin-right:.25em}
ul.inline{display:flex;flex-flow:row wrap;list-style:none;margin:0 0 .625em -1.25em}
ul.inline>li{margin-left:1.25em}
.unstyled dl dt{font-weight:400;font-style:normal}
ol.arabic{list-style-type:decimal}
ol.decimal{list-style-type:decimal-leading-zero}
ol.loweralpha{list-style-type:lower-alpha}
ol.upperalpha{list-style-type:upper-alpha}
ol.lowerroman{list-style-type:lower-roman}
ol.upperroman{list-style-type:upper-roman}
ol.lowergreek{list-style-type:lower-greek}
.hdlist>table,.colist>table{border:0;background:none}
.hdlist>table>tbody>tr,.colist>table>tbody>tr{background:none}
td.hdlist1,td.hdlist2{vertical-align:top;padding:0 .625em}
td.hdlist1{font-weight:bold;padding-bottom:1.25em}
td.hdlist2{word-wrap:anywhere}
.literalblock+.colist,.listingblock+.colist{margin-top:-.5em}
.colist td:not([class]):first-child{padding:.4em .75em 0;line-height:1;vertical-align:top}
.colist td:not([class]):first-child img{max-width:none}
.colist td:not([class]):last-child{padding:.25em 0}
.thumb,.th{line-height:0;display:inline-block;border:4px solid #fff;box-shadow:0 0 0 1px #ddd}
.imageblock.left{margin:.25em .625em 1.25em 0}
.imageblock.right{margin:.25em 0 1.25em .625em}
.imageblock>.title{margin-bottom:0}
.imageblock.thumb,.imageblock.th{border-width:6px}
.imageblock.thumb>.title,.imageblock.th>.title{padding:0 .125em}
.image.left,.image.right{margin-top:.25em;margin-bottom:.25em;display:inline-block;line-height:0}
.image.left{margin-right:.625em}
.image.right{margin-left:.625em}
a.image{text-decoration:none;display:inline-block}
a.image object{pointer-events:none}
sup.footnote,sup.footnoteref{font-size:.875em;position:static;vertical-align:super}
sup.footnote a,sup.footnoteref a{text-decoration:none}
sup.footnote a:active,sup.footnoteref a:active,#footnotes .footnote a:first-of-type:active{text-decoration:underline}
#footnotes{padding-top:.75em;padding-bottom:.75em;margin-bottom:.625em}
#footnotes hr{width:20%;min-width:6.25em;margin:-.25em 0 .75em;border-width:1px 0 0}
#footnotes .footnote{padding:0 .375em 0 .225em;line-height:1.3334;font-size:.875em;margin-left:1.2em;margin-bottom:.2em}
#footnotes .footnote a:first-of-type{font-weight:bold;text-decoration:none;margin-left:-1.05em}
#footnotes .footnote:last-of-type{margin-bottom:0}
#content #footnotes{margin-top:-.625em;margin-bottom:0;padding:.75em 0}
div.unbreakable{page-break-inside:avoid}
.big{font-size:larger}
.small{font-size:smaller}
.underline{text-decoration:underline}
.overline{text-decoration:overline}
.line-through{text-decoration:line-through}
.aqua{color:#00bfbf}
.aqua-background{background:#00fafa}
.black{color:#000}
.black-background{background:#000}
.blue{color:#0000bf}
.blue-background{background:#0000fa}
.fuchsia{color:#bf00bf}
.fuchsia-background{background:#fa00fa}
.gray{color:#606060}
.gray-background{background:#7d7d7d}
.green{color:#006000}
.green-background{background:#007d00}
.lime{color:#00bf00}
.lime-background{background:#00fa00}
.maroon{color:#600000}
.maroon-background{background:#7d0000}
.navy{color:#000060}
.navy-background{background:#00007d}
.olive{color:#606000}
.olive-background{background:#7d7d00}
.purple{color:#600060}
.purple-background{background:#7d007d}
.red{color:#bf0000}
.red-background{background:#fa0000}
.silver{color:#909090}
.silver-background{background:#bcbcbc}
.teal{color:#006060}
.teal-background{background:#007d7d}
.white{color:#bfbfbf}
.white-background{background:#fafafa}
.yellow{color:#bfbf00}
.yellow-background{background:#fafa00}
span.icon>.fa{cursor:default}
a span.icon>.fa{cursor:inherit}
.admonitionblock td.icon [class^="fa icon-"]{font-size:2.5em;text-shadow:1px 1px 2px rgba(0,0,0,.5);cursor:default}
.admonitionblock td.icon .icon-note::before{content:"\f05a";color:#19407c}
.admonitionblock td.icon .icon-tip::before{content:"\f0eb";text-shadow:1px 1px 2px rgba(155,155,0,.8);color:#111}
.admonitionblock td.icon .icon-warning::before{content:"\f071";color:#bf6900}
.admonitionblock td.icon .icon-caution::before{content:"\f06d";color:#bf3400}
.admonitionblock td.icon .icon-important::before{content:"\f06a";color:#bf0000}
.conum[data-value]{display:inline-block;color:#fff!important;background:rgba(0,0,0,.8);border-radius:50%;text-align:center;font-size:.75em;width:1.67em;height:1.67em;line-height:1.67em;font-family:"Open Sans","DejaVu Sans",sans-serif;font-style:normal;font-weight:bold}
.conum[data-value] *{color:#fff!important}
.conum[data-value]+b{display:none}
.conum[data-value]::after{content:attr(data-value)}
pre .conum[data-value]{position:relative;top:-.125em}
b.conum *{color:inherit!important}
.conum:not([data-value]):empty{display:none}
dt,th.tableblock,td.content,div.footnote{text-rendering:optimizeLegibility}
h1,h2,p,td.content,span.alt,summary{letter-spacing:-.01em}
p strong,td.content strong,div.footnote strong{letter-spacing:-.005em}
p,blockquote,dt,td.content,td.hdlist1,span.alt,summary{font-size:1.0625rem}
p{margin-bottom:1.25rem}
.sidebarblock p,.sidebarblock dt,.sidebarblock td.content,p.tableblock{font-size:1em}
.exampleblock>.content{background:#fffef7;border-color:#e0e0dc;box-shadow:0 1px 4px #e0e0dc}
.print-only{display:none!important}
@page{margin:1.25cm .75cm}
@media print{*{box-shadow:none!important;text-shadow:none!important}
html{font-size:80%}
a{color:inherit!important;text-decoration:underline!important}
a.bare,a[href^="#"],a[href^="mailto:"]{text-decoration:none!important}
a[href^="http:"]:not(.bare)::after,a[href^="https:"]:not(.bare)::after{content:"(" attr(href) ")";display:inline-block;font-size:.875em;padding-left:.25em}
abbr[title]{border-bottom:1px dotted}
abbr[title]::after{content:" (" attr(title) ")"}
pre,blockquote,tr,img,object,svg{page-break-inside:avoid}
thead{display:table-header-group}
svg{max-width:100%}
p,blockquote,dt,td.content{font-size:1em;orphans:3;widows:3}
h2,h3,#toctitle,.sidebarblock>.content>.title{page-break-after:avoid}
#header,#content,#footnotes,#footer{max-width:none}
#toc,.sidebarblock,.exampleblock>.content{background:none!important}
#toc{border-bottom:1px solid #dddddf!important;padding-bottom:0!important}
body.book #header{text-align:center}
body.book #header>h1:first-child{border:0!important;margin:2.5em 0 1em}
body.book #header .details{border:0!important;display:block;padding:0!important}
body.book #header .details span:first-child{margin-left:0!important}
body.book #header .details br{display:block}
body.book #header .details br+span::before{content:none!important}
body.book #toc{border:0!important;text-align:left!important;padding:0!important;margin:0!important}
body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-break-before:always}
.listingblock code[data-lang]::before{display:block}
#footer{padding:0 .9375em}
.hide-on-print{display:none!important}
.print-only{display:block!important}
.hide-for-print{display:none!important}
.show-for-print{display:inherit!important}}
@media amzn-kf8,print{#header>h1:first-child{margin-top:1.25rem}
.sect1{padding:0!important}
.sect1+.sect1{border:0}
#footer{background:none}
#footer-text{color:rgba(0,0,0,.6);font-size:.9em}}
@media amzn-kf8{#header,#content,#footnotes,#footer{padding:0}}
</style>
</head>
<body class="article">
<div id="header">
<h1>Use of index and Racy Git problem</h1>
</div>
<div id="content">
<div class="sect1">
<h2 id="_background">Background</h2>
<div class="sectionbody">
<div class="paragraph">
<p>The index is one of the most important data structures in Git.
It represents a virtual working tree state by recording list of
paths and their object names and serves as a staging area to
write out the next tree object to be committed. The state is
"virtual" in the sense that it does not necessarily have to, and
often does not, match the files in the working tree.</p>
</div>
<div class="paragraph">
<p>There are cases where Git needs to examine the differences between the
virtual working tree state in the index and the files in the
working tree. The most obvious case is when the user asks <code>git</code>
<code>diff</code> (or its low level implementation, <code>git</code> <code>diff-files</code>) or
<code>git-ls-files</code> <code>--modified</code>. In addition, Git internally checks
if the files in the working tree are different from what are
recorded in the index to avoid stomping on local changes in them
during patch application, switching branches, and merging.</p>
</div>
<div class="paragraph">
<p>In order to speed up this comparison between the files in the
working tree and the index entries, the index entries record the
information obtained from the filesystem via <code>lstat</code>(<code>2</code>) system
call when they were last updated. When checking if they differ,
Git first runs <code>lstat</code>(<code>2</code>) on the files and compares the result
with this information (this is what was originally done by the
<code>ce_match_stat</code>() function, but the current code does it in
<code>ce_match_stat_basic</code>() function). If some of these "cached
stat information" fields do not match, Git can tell that the
files are modified without even looking at their contents.</p>
</div>
<div class="paragraph">
<p>Note: not all members in <code>struct</code> <code>stat</code> obtained via <code>lstat</code>(<code>2</code>)
are used for this comparison. For example, <code>st_atime</code> obviously
is not useful. Currently, Git compares the file type (regular
files vs symbolic links) and executable bits (only for regular
files) from <code>st_mode</code> member, <code>st_mtime</code> and <code>st_ctime</code>
timestamps, <code>st_uid</code>, <code>st_gid</code>, <code>st_ino</code>, and <code>st_size</code> members.
With a <code>USE_STDEV</code> compile-time option, <code>st_dev</code> is also
compared, but this is not enabled by default because this member
is not stable on network filesystems. With <code>USE_NSEC</code>
compile-time option, <code>st_mtim.tv_nsec</code> and <code>st_ctim.tv_nsec</code>
members are also compared. On Linux, this is not enabled by default
because in-core timestamps can have finer granularity than
on-disk timestamps, resulting in meaningless changes when an
inode is evicted from the inode cache. See commit 8ce13b0
of git://git.kernel.org/pub/scm/linux/kernel/git/tglx/history.git
([PATCH] Sync in core time granularity with filesystems,
2005-01-04). This patch is included in kernel 2.6.11 and newer, but
only fixes the issue for file systems with exactly 1 ns or 1 s
resolution. Other file systems are still broken in current Linux
kernels (e.g. CEPH, CIFS, NTFS, UDF), see
<a href="https://lore.kernel.org/lkml/5577240D.7020309@gmail.com/" class="bare">https://lore.kernel.org/lkml/5577240D.7020309@gmail.com/</a></p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_racy_git">Racy Git</h2>
<div class="sectionbody">
<div class="paragraph">
<p>There is one slight problem with the optimization based on the
cached stat information. Consider this sequence:</p>
</div>
<div class="literalblock">
<div class="content">
<pre>: modify 'foo'
$ git update-index 'foo'
: modify 'foo' again, in-place, without changing its size</pre>
</div>
</div>
<div class="paragraph">
<p>The first <code>update-index</code> computes the object name of the
contents of file <code>foo</code> and updates the index entry for <code>foo</code>
along with the <code>struct</code> <code>stat</code> information. If the modification
that follows it happens very fast so that the file&#8217;s <code>st_mtime</code>
timestamp does not change, after this sequence, the cached stat
information the index entry records still exactly match what you
would see in the filesystem, even though the file <code>foo</code> is now
different.
This way, Git can incorrectly think files in the working tree
are unmodified even though they actually are. This is called
the "racy Git" problem (discovered by Pasky), and the entries
that appear clean when they may not be because of this problem
are called "racily clean".</p>
</div>
<div class="paragraph">
<p>To avoid this problem, Git does two things:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>When the cached stat information says the file has not been
modified, and the <code>st_mtime</code> is the same as (or newer than)
the timestamp of the index file itself (which is the time <code>git</code>
<code>update-index</code> <code>foo</code> finished running in the above example), it
also compares the contents with the object registered in the
index entry to make sure they match.</p>
</li>
<li>
<p>When the index file is updated that contains racily clean
entries, cached <code>st_size</code> information is truncated to zero
before writing a new version of the index file.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>Because the index file itself is written after collecting all
the stat information from updated paths, <code>st_mtime</code> timestamp of
it is usually the same as or newer than any of the paths the
index contains. And no matter how quick the modification that
follows <code>git</code> <code>update-index</code> <code>foo</code> finishes, the resulting
<code>st_mtime</code> timestamp on <code>foo</code> cannot get a value earlier
than the index file. Therefore, index entries that can be
racily clean are limited to the ones that have the same
timestamp as the index file itself.</p>
</div>
<div class="paragraph">
<p>The callers that want to check if an index entry matches the
corresponding file in the working tree continue to call
<code>ce_match_stat</code>(), but with this change, <code>ce_match_stat</code>() uses
<code>ce_modified_check_fs</code>() to see if racily clean ones are
actually clean after comparing the cached stat information using
<code>ce_match_stat_basic</code>().</p>
</div>
<div class="paragraph">
<p>The problem the latter solves is this sequence:</p>
</div>
<div class="literalblock">
<div class="content">
<pre>$ git update-index 'foo'
: modify 'foo' in-place without changing its size
: wait for enough time
$ git update-index 'bar'</pre>
</div>
</div>
<div class="paragraph">
<p>Without the latter, the timestamp of the index file gets a newer
value, and falsely clean entry <code>foo</code> would not be caught by the
timestamp comparison check done with the former logic anymore.
The latter makes sure that the cached stat information for <code>foo</code>
would never match with the file in the working tree, so later
checks by <code>ce_match_stat_basic</code>() would report that the index entry
does not match the file and Git does not have to fall back on more
expensive <code>ce_modified_check_fs</code>().</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_runtime_penalty">Runtime penalty</h2>
<div class="sectionbody">
<div class="paragraph">
<p>The runtime penalty of falling back to <code>ce_modified_check_fs</code>()
from <code>ce_match_stat</code>() can be very expensive when there are many
racily clean entries. An obvious way to artificially create
this situation is to give the same timestamp to all the files in
the working tree in a large project, run <code>git</code> <code>update-index</code> on
them, and give the same timestamp to the index file:</p>
</div>
<div class="literalblock">
<div class="content">
<pre>$ date &gt;.datestamp
$ git ls-files | xargs touch -r .datestamp
$ git ls-files | git update-index --stdin
$ touch -r .datestamp .git/index</pre>
</div>
</div>
<div class="paragraph">
<p>This will make all index entries racily clean. The linux project, for
example, there are over 20,000 files in the working tree. On my
Athlon 64 X2 3800+, after the above:</p>
</div>
<div class="literalblock">
<div class="content">
<pre>$ /usr/bin/time git diff-files
1.68user 0.54system 0:02.22elapsed 100%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+67111minor)pagefaults 0swaps
$ git update-index MAINTAINERS
$ /usr/bin/time git diff-files
0.02user 0.12system 0:00.14elapsed 100%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+935minor)pagefaults 0swaps</pre>
</div>
</div>
<div class="paragraph">
<p>Running <code>git</code> <code>update-index</code> in the middle checked the racily
clean entries, and left the cached <code>st_mtime</code> for all the paths
intact because they were actually clean (so this step took about
the same amount of time as the first <code>git</code> <code>diff-files</code>). After
that, they are not racily clean anymore but are truly clean, so
the second invocation of <code>git</code> <code>diff-files</code> fully took advantage
of the cached stat information.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_avoiding_runtime_penalty">Avoiding runtime penalty</h2>
<div class="sectionbody">
<div class="paragraph">
<p>In order to avoid the above runtime penalty, post 1.4.2 Git used
to have a code that made sure the index file
got a timestamp newer than the youngest files in the index when
there were many young files with the same timestamp as the
resulting index file otherwise would have by waiting
before finishing writing the index file out.</p>
</div>
<div class="paragraph">
<p>I suspected that in practice the situation where many paths in the
index are all racily clean was quite rare. The only code paths
that can record recent timestamp for large number of paths are:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Initial <code>git</code> <code>add</code> . of a large project.</p>
</li>
<li>
<p><code>git</code> <code>checkout</code> of a large project from an empty index into an
unpopulated working tree.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>Note: switching branches with <code>git</code> <code>checkout</code> keeps the cached
stat information of existing working tree files that are the
same between the current branch and the new branch, which are
all older than the resulting index file, and they will not
become racily clean. Only the files that are actually checked
out can become racily clean.</p>
</div>
<div class="paragraph">
<p>In a large project where raciness avoidance cost really matters,
however, the initial computation of all object names in the
index takes more than one second, and the index file is written
out after all that happens. Therefore the timestamp of the
index file will be more than one second later than the
youngest file in the working tree. This means that in these
cases there actually will not be any racily clean entry in
the resulting index.</p>
</div>
<div class="paragraph">
<p>Based on this discussion, the current code does not use the
"workaround" to avoid the runtime penalty that does not exist in
practice anymore. This was done with commit 0fc82cff on Aug 15,
2006.</p>
</div>
</div>
</div>
</div>
<div id="footer">
<div id="footer-text">
Last updated 2025-03-17 07:09:38 UTC
</div>
</div>
</body>
</html>

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,671 @@
Rebases and cherry-picks involve a sequence of merges whose results are
recorded as new single-parent commits. The first parent side of those
merges represent the "upstream" side, and often include a far larger set of
changes than the second parent side. Traditionally, the renames on the
first-parent side of that sequence of merges were repeatedly re-detected
for every merge. This file explains why it is safe and effective during
rebases and cherry-picks to remember renames on the upstream side of
history as an optimization, assuming all merges are automatic and clean
(i.e. no conflicts and not interrupted for user input or editing).
Outline:
0. Assumptions
1. How rebasing and cherry-picking work
2. Why the renames on MERGE_SIDE1 in any given pick are *always* a
superset of the renames on MERGE_SIDE1 for the next pick.
3. Why any rename on MERGE_SIDE1 in any given pick is _almost_ always also
a rename on MERGE_SIDE1 for the next pick
4. A detailed description of the counter-examples to #3.
5. Why the special cases in #4 are still fully reasonable to use to pair
up files for three-way content merging in the merge machinery, and why
they do not affect the correctness of the merge.
6. Interaction with skipping of "irrelevant" renames
7. Additional items that need to be cached
8. How directory rename detection interacts with the above and why this
optimization is still safe even if merge.directoryRenames is set to
"true".
=== 0. Assumptions ===
There are two assumptions that will hold throughout this document:
* The upstream side where commits are transplanted to is treated as the
first parent side when rebase/cherry-pick call the merge machinery
* All merges are fully automatic
and a third that will hold in sections 2-5 for simplicity, that I'll later
address in section 8:
* No directory renames occur
Let me explain more about each assumption and why I include it:
The first assumption is merely for the purposes of making this document
clearer; the optimization implementation does not actually depend upon it.
However, the assumption does hold in all cases because it reflects the way
that both rebase and cherry-pick were implemented; and the implementation
of cherry-pick and rebase are not readily changeable for backwards
compatibility reasons (see for example the discussion of the --ours and
--theirs flag in the documentation of `git checkout`, particularly the
comments about how they behave with rebase). The optimization avoids
checking first-parent-ness, though. It checks the conditions that make the
optimization valid instead, so it would still continue working if someone
changed the parent ordering that cherry-pick and rebase use. But making
this assumption does make this document much clearer and prevents me from
having to repeat every example twice.
If the second assumption is violated, then the optimization simply is
turned off and thus isn't relevant to consider. The second assumption can
also be stated as "there is no interruption for a user to resolve conflicts
or to just further edit or tweak files". While real rebases and
cherry-picks are often interrupted (either because it's an interactive
rebase where the user requested to stop and edit, or because there were
conflicts that the user needs to resolve), the cache of renames is not
stored on disk, and thus is thrown away as soon as the rebase or cherry
pick stops for the user to resolve the operation.
The third assumption makes sections 2-5 simpler, and allows people to
understand the basics of why this optimization is safe and effective, and
then I can go back and address the specifics in section 8. It is probably
also worth noting that if directory renames do occur, then the default of
merge.directoryRenames being set to "conflict" means that the operation
will stop for users to resolve the conflicts and the cache will be thrown
away, and thus that there won't be an optimization to apply. So, the only
reason we need to address directory renames specifically, is that some
users will have set merge.directoryRenames to "true" to allow the merges to
continue to proceed automatically. The optimization is still safe with
this config setting, but we have to discuss a few more cases to show why;
this discussion is deferred until section 8.
=== 1. How rebasing and cherry-picking work ===
Consider the following setup (from the git-rebase manpage):
A---B---C topic
/
D---E---F---G main
After rebasing or cherry-picking topic onto main, this will appear as:
A'--B'--C' topic
/
D---E---F---G main
The way the commits A', B', and C' are created is through a series of
merges, where rebase or cherry-pick sequentially uses each of the three
A-B-C commits in a special merge operation. Let's label the three commits
in the merge operation as MERGE_BASE, MERGE_SIDE1, and MERGE_SIDE2. For
this picture, the three commits for each of the three merges would be:
To create A':
MERGE_BASE: E
MERGE_SIDE1: G
MERGE_SIDE2: A
To create B':
MERGE_BASE: A
MERGE_SIDE1: A'
MERGE_SIDE2: B
To create C':
MERGE_BASE: B
MERGE_SIDE1: B'
MERGE_SIDE2: C
Sometimes, folks are surprised that these three-way merges are done. It
can be useful in understanding these three-way merges to view them in a
slightly different light. For example, in creating C', you can view it as
either:
* Apply the changes between B & C to B'
* Apply the changes between B & B' to C
Conceptually the two statements above are the same as a three-way merge of
B, B', and C, at least the parts before you decide to record a commit.
=== 2. Why the renames on MERGE_SIDE1 in any given pick are always a ===
=== superset of the renames on MERGE_SIDE1 for the next pick. ===
The merge machinery uses the filenames it is fed from MERGE_BASE,
MERGE_SIDE1, and MERGE_SIDE2. It will only move content to a different
filename under one of three conditions:
* To make both pieces of a conflict available to a user during conflict
resolution (examples: directory/file conflict, add/add type conflict
such as symlink vs. regular file)
* When MERGE_SIDE1 renames the file.
* When MERGE_SIDE2 renames the file.
First, let's remember what commits are involved in the first and second
picks of the cherry-pick or rebase sequence:
To create A':
MERGE_BASE: E
MERGE_SIDE1: G
MERGE_SIDE2: A
To create B':
MERGE_BASE: A
MERGE_SIDE1: A'
MERGE_SIDE2: B
So, in particular, we need to show that the renames between E and G are a
superset of those between A and A'.
A' is created by the first merge. A' will only have renames for one of the
three reasons listed above. The first case, a conflict, results in a
situation where the cache is dropped and thus this optimization doesn't
take effect, so we need not consider that case. The third case, a rename
on MERGE_SIDE2 (i.e. from G to A), will show up in A' but it also shows up
in A -- therefore when diffing A and A' that path does not show up as a
rename. The only remaining way for renames to show up in A' is for the
rename to come from MERGE_SIDE1. Therefore, all renames between A and A'
are a subset of those between E and G. Equivalently, all renames between E
and G are a superset of those between A and A'.
=== 3. Why any rename on MERGE_SIDE1 in any given pick is _almost_ ===
=== always also a rename on MERGE_SIDE1 for the next pick. ===
Let's again look at the first two picks:
To create A':
MERGE_BASE: E
MERGE_SIDE1: G
MERGE_SIDE2: A
To create B':
MERGE_BASE: A
MERGE_SIDE1: A'
MERGE_SIDE2: B
Now let's look at any given rename from MERGE_SIDE1 of the first pick, i.e.
any given rename from E to G. Let's use the filenames 'oldfile' and
'newfile' for demonstration purposes. That first pick will function as
follows; when the rename is detected, the merge machinery will do a
three-way content merge of the following:
E:oldfile
G:newfile
A:oldfile
and produce a new result:
A':newfile
Note above that I've assumed that E->A did not rename oldfile. If that
side did rename, then we most likely have a rename/rename(1to2) conflict
that will cause the rebase or cherry-pick operation to halt and drop the
in-memory cache of renames and thus doesn't need to be considered further.
In the special case that E->A does rename the file but also renames it to
newfile, then there is no conflict from the renaming and the merge can
succeed. In this special case, the rename is not valid to cache because
the second merge will find A:newfile in the MERGE_BASE (see also the new
testcases in t6429 with "rename same file identically" in their
description). So a rename/rename(1to1) needs to be specially handled by
pruning renames from the cache and decrementing the dir_rename_counts in
the current and leading directories associated with those renames. Or,
since these are really rare, one could just take the easy way out and
disable the remembering renames optimization when a rename/rename(1to1)
happens.
The previous paragraph handled the cases for E->A renaming oldfile, let's
continue assuming that oldfile is not renamed in A.
As per the diagram for creating B', MERGE_SIDE1 involves the changes from A
to A'. So, we are curious whether A:oldfile and A':newfile will be viewed
as renames. Note that:
* There will be no A':oldfile (because there could not have been a
G:oldfile as we do not do break detection in the merge machinery and
G:newfile was detected as a rename, and by the construction of the
rename above that merged cleanly, the merge machinery will ensure there
is no 'oldfile' in the result).
* There will be no A:newfile (if there had been, we would have had a
rename/add conflict).
* Clearly A:oldfile and A':newfile are "related" (A':newfile came from a
clean three-way content merge involving A:oldfile).
We can also expound on the third point above, by noting that three-way
content merges can also be viewed as applying the differences between the
base and one side to the other side. Thus we can view A':newfile as
having been created by taking the changes between E:oldfile and G:newfile
(which were detected as being related, i.e. <50% changed) to A:oldfile.
Thus A:oldfile and A':newfile are just as related as E:oldfile and
G:newfile are -- they have exactly identical differences. Since the latter
were detected as renames, A:oldfile and A':newfile should also be
detectable as renames almost always.
=== 4. A detailed description of the counter-examples to #3. ===
We already noted in section 3 that rename/rename(1to1) (i.e. both sides
renaming a file the same way) was one counter-example. The more
interesting bit, though, is why did we need to use the "almost" qualifier
when stating that A:oldfile and A':newfile are "almost" always detectable
as renames?
Let's repeat an earlier point that section 3 made:
A':newfile was created by applying the changes between E:oldfile and
G:newfile to A:oldfile. The changes between E:oldfile and G:newfile were
<50% of the size of E:oldfile.
If those changes that were <50% of the size of E:oldfile are also <50% of
the size of A:oldfile, then A:oldfile and A':newfile will be detectable as
renames. However, if there is a dramatic size reduction between E:oldfile
and A:oldfile (but the changes between E:oldfile, G:newfile, and A:oldfile
still somehow merge cleanly), then traditional rename detection would not
detect A:oldfile and A':newfile as renames.
Here's an example where that can happen:
* E:oldfile had 20 lines
* G:newfile added 10 new lines at the beginning of the file
* A:oldfile kept the first 3 lines of the file, and deleted all the rest
then
=> A':newfile would have 13 lines, 3 of which matches those in A:oldfile.
E:oldfile -> G:newfile would be detected as a rename, but A:oldfile and
A':newfile would not be.
=== 5. Why the special cases in #4 are still fully reasonable to use to ===
=== pair up files for three-way content merging in the merge machinery, ===
=== and why they do not affect the correctness of the merge. ===
In the rename/rename(1to1) case, A:newfile and A':newfile are not renames
since they use the *same* filename. However, files with the same filename
are obviously fine to pair up for three-way content merging (the merge
machinery has never employed break detection). The interesting
counter-example case is thus not the rename/rename(1to1) case, but the case
where A did not rename oldfile. That was the case that we spent most of
the time discussing in sections 3 and 4. The remainder of this section
will be devoted to that case as well.
So, even if A:oldfile and A':newfile aren't detectable as renames, why is
it still reasonable to pair them up for three-way content merging in the
merge machinery? There are multiple reasons:
* As noted in sections 3 and 4, the diff between A:oldfile and A':newfile
is *exactly* the same as the diff between E:oldfile and G:newfile. The
latter pair were detected as renames, so it seems unlikely to surprise
users for us to treat A:oldfile and A':newfile as renames.
* In fact, "oldfile" and "newfile" were at one point detected as renames
due to how they were constructed in the E..G chain. And we used that
information once already in this rebase/cherry-pick. I think users
would be unlikely to be surprised at us continuing to treat the files
as renames and would quickly understand why we had done so.
* Marking or declaring files as renames is *not* the end goal for merges.
Merges use renames to determine which files make sense to be paired up
for three-way content merges.
* A:oldfile and A':newfile were _already_ paired up in a three-way
content merge; that is how A':newfile was created. In fact, that
three-way content merge was clean. So using them again in a later
three-way content merge seems very reasonable.
However, the above is focusing on the common scenarios. Let's try to look
at all possible unusual scenarios and compare without the optimization to
with the optimization. Consider the following theoretical cases; we will
then dive into each to determine which of them are possible,
and if so, what they mean:
1. Without the optimization, the second merge results in a conflict.
With the optimization, the second merge also results in a conflict.
Questions: Are the conflicts confusingly different? Better in one case?
2. Without the optimization, the second merge results in NO conflict.
With the optimization, the second merge also results in NO conflict.
Questions: Are the merges the same?
3. Without the optimization, the second merge results in a conflict.
With the optimization, the second merge results in NO conflict.
Questions: Possible? Bug, bugfix, or something else?
4. Without the optimization, the second merge results in NO conflict.
With the optimization, the second merge results in a conflict.
Questions: Possible? Bug, bugfix, or something else?
I'll consider all four cases, but out of order.
The fourth case is impossible. For the code without the remembering
renames optimization to not get a conflict, B:oldfile would need to exactly
match A:oldfile -- if it doesn't, there would be a modify/delete conflict.
If A:oldfile matches B:oldfile exactly, then a three-way content merge
between A:oldfile, A':newfile, and B:oldfile would have no conflict and
just give us the version of newfile from A' as the result.
From the same logic as the above paragraph, the second case would indeed
result in identical merges. When A:oldfile exactly matches B:oldfile, an
undetected rename would say, "Oh, I see one side didn't modify 'oldfile'
and the other side deleted it. I'll delete it. And I see you have this
brand new file named 'newfile' in A', so I'll keep it." That gives the
same results as three-way content merging A:oldfile, A':newfile, and
B:oldfile -- a removal of oldfile with the version of newfile from A'
showing up in the result.
The third case is interesting. It means that A:oldfile and A':newfile were
not just similar enough, but that the changes between them did not conflict
with the changes between A:oldfile and B:oldfile. This would validate our
hunch that the files were similar enough to be used in a three-way content
merge, and thus seems entirely correct for us to have used them that way.
(Sidenote: One particular example here may be enlightening. Let's say that
B was an immediate revert of A. B clearly would have been a clean revert
of A, since A was B's immediate parent. One would assume that if you can
pick a commit, you should also be able to cherry-pick its immediate revert.
However, this is one of those funny corner cases; without this
optimization, we just successfully picked a commit cleanly, but we are
unable to cherry-pick its immediate revert due to the size differences
between E:oldfile and A:oldfile.)
That leaves only the first case to consider -- when we get conflicts both
with or without the optimization. Without the optimization, we'll have a
modify/delete conflict, where both A':newfile and B:oldfile are left in the
tree for the user to deal with and no hints about the potential similarity
between the two. With the optimization, we'll have a three-way content
merged A:oldfile, A':newfile, and B:oldfile with conflict markers
suggesting we thought the files were related but giving the user the chance
to resolve. As noted above, I don't think users will find us treating
'oldfile' and 'newfile' as related as a surprise since they were between E
and G. In any event, though, this case shouldn't be concerning since we
hit a conflict in both cases, told the user what we know, and asked them to
resolve it.
So, in summary, case 4 is impossible, case 2 yields the same behavior, and
cases 1 and 3 seem to provide as good or better behavior with the
optimization than without.
=== 6. Interaction with skipping of "irrelevant" renames ===
Previous optimizations involved skipping rename detection for paths
considered to be "irrelevant". See for example the following commits:
* 32a56dfb99 ("merge-ort: precompute subset of sources for which we
need rename detection", 2021-03-11)
* 2fd9eda462 ("merge-ort: precompute whether directory rename
detection is needed", 2021-03-11)
* 9bd342137e ("diffcore-rename: determine which relevant_sources are
no longer relevant", 2021-03-13)
Relevance is always determined by what the _other_ side of history has
done, in terms of modifying a file that our side renamed, or adding a
file to a directory which our side renamed. This means that a path
that is "irrelevant" when picking the first commit of a series in a
rebase or cherry-pick, may suddenly become "relevant" when picking the
next commit.
The upshot of this is that we can only cache rename detection results
for relevant paths, and need to re-check relevance in subsequent
commits. If those subsequent commits have additional paths that are
relevant for rename detection, then we will need to redo rename
detection -- though we can limit it to the paths for which we have not
already detected renames.
=== 7. Additional items that need to be cached ===
It turns out we have to cache more than just renames; we also cache:
A) non-renames (i.e. unpaired deletes)
B) counts of renames within directories
C) sources that were marked as RELEVANT_LOCATION, but which were
downgraded to RELEVANT_NO_MORE
D) the toplevel trees involved in the merge
These are all stored in struct rename_info, and respectively appear in
* cached_pairs (along side actual renames, just with a value of NULL)
* dir_rename_counts
* cached_irrelevant
* merge_trees
The reason for (A) comes from the irrelevant renames skipping
optimization discussed in section 6. The fact that irrelevant renames
are skipped means we only get a subset of the potential renames
detected and subsequent commits may need to run rename detection on
the upstream side on a subset of the remaining renames (to get the
renames that are relevant for that later commit). Since unpaired
deletes are involved in rename detection too, we don't want to
repeatedly check that those paths remain unpaired on the upstream side
with every commit we are transplanting.
The reason for (B) is that diffcore_rename_extended() is what
generates the counts of renames by directory which is needed in
directory rename detection, and if we don't run
diffcore_rename_extended() again then we need to have the output from
it, including dir_rename_counts, from the previous run.
The reason for (C) is that merge-ort's tree traversal will again think
those paths are relevant (marking them as RELEVANT_LOCATION), but the
fact that they were downgraded to RELEVANT_NO_MORE means that
dir_rename_counts already has the information we need for directory
rename detection. (A path which becomes RELEVANT_CONTENT in a
subsequent commit will be removed from cached_irrelevant.)
The reason for (D) is that is how we determine whether the remember
renames optimization can be used. In particular, remembering that our
sequence of merges looks like:
Merge 1:
MERGE_BASE: E
MERGE_SIDE1: G
MERGE_SIDE2: A
=> Creates A'
Merge 2:
MERGE_BASE: A
MERGE_SIDE1: A'
MERGE_SIDE2: B
=> Creates B'
It is the fact that the trees A and A' appear both in Merge 1 and in
Merge 2, with A as a parent of A' that allows this optimization. So
we store the trees to compare with what we are asked to merge next
time.
=== 8. How directory rename detection interacts with the above and ===
=== why this optimization is still safe even if ===
=== merge.directoryRenames is set to "true". ===
As noted in the assumptions section:
"""
...if directory renames do occur, then the default of
merge.directoryRenames being set to "conflict" means that the operation
will stop for users to resolve the conflicts and the cache will be
thrown away, and thus that there won't be an optimization to apply.
So, the only reason we need to address directory renames specifically,
is that some users will have set merge.directoryRenames to "true" to
allow the merges to continue to proceed automatically.
"""
Let's remember that we need to look at how any given pick affects the next
one. So let's again use the first two picks from the diagram in section
one:
First pick does this three-way merge:
MERGE_BASE: E
MERGE_SIDE1: G
MERGE_SIDE2: A
=> creates A'
Second pick does this three-way merge:
MERGE_BASE: A
MERGE_SIDE1: A'
MERGE_SIDE2: B
=> creates B'
Now, directory rename detection exists so that if one side of history
renames a directory, and the other side adds a new file to the old
directory, then the merge (with merge.directoryRenames=true) can move the
file into the new directory. There are two qualitatively different ways to
add a new file to an old directory: create a new file, or rename a file
into that directory. Also, directory renames can be done on either side of
history, so there are four cases to consider:
* MERGE_SIDE1 renames old dir, MERGE_SIDE2 adds new file to old dir
* MERGE_SIDE1 renames old dir, MERGE_SIDE2 renames file into old dir
* MERGE_SIDE1 adds new file to old dir, MERGE_SIDE2 renames old dir
* MERGE_SIDE1 renames file into old dir, MERGE_SIDE2 renames old dir
One last note before we consider these four cases: There are some
important properties about how we implement this optimization with
respect to directory rename detection that we need to bear in mind
while considering all of these cases:
* rename caching occurs *after* applying directory renames
* a rename created by directory rename detection is recorded for the side
of history that did the directory rename.
* dir_rename_counts, the nested map of
{oldname => {newname => count}},
is cached between runs as well. This basically means that directory
rename detection is also cached, though only on the side of history
that we cache renames for (MERGE_SIDE1 as far as this document is
concerned; see the assumptions section). Two interesting sub-notes
about these counts:
* If we need to perform rename-detection again on the given side (e.g.
some paths are relevant for rename detection that weren't before),
then we clear dir_rename_counts and recompute it, making use of
cached_pairs. The reason it is important to do this is optimizations
around RELEVANT_LOCATION exist to prevent us from computing
unnecessary renames for directory rename detection and from computing
dir_rename_counts for irrelevant directories; but those same renames
or directories may become necessary for subsequent merges. The
easiest way to "fix up" dir_rename_counts in such cases is to just
recompute it.
* If we prune rename/rename(1to1) entries from the cache, then we also
need to update dir_rename_counts to decrement the counts for the
involved directory and any relevant parent directories (to undo what
update_dir_rename_counts() in diffcore-rename.c incremented when the
rename was initially found). If we instead just disable the
remembering renames optimization when the exceedingly rare
rename/rename(1to1) cases occur, then dir_rename_counts will get
re-computed the next time rename detection occurs, as noted above.
* the side with multiple commits to pick, is the side of history that we
do NOT cache renames for. Thus, there are no additional commits to
change the number of renames in a directory, except for those done by
directory rename detection (which always pad the majority).
* the "renames" we cache are modified slightly by any directory rename,
as noted below.
Now, with those notes out of the way, let's go through the four cases
in order:
Case 1: MERGE_SIDE1 renames old dir, MERGE_SIDE2 adds new file to old dir
This case looks like this:
MERGE_BASE: E, Has olddir/
MERGE_SIDE1: G, Renames olddir/ -> newdir/
MERGE_SIDE2: A, Adds olddir/newfile
=> creates A', With newdir/newfile
MERGE_BASE: A, Has olddir/newfile
MERGE_SIDE1: A', Has newdir/newfile
MERGE_SIDE2: B, Modifies olddir/newfile
=> expected B', with threeway-merged newdir/newfile from above
In this case, with the optimization, note that after the first commit:
* MERGE_SIDE1 remembers olddir/ -> newdir/
* MERGE_SIDE1 has cached olddir/newfile -> newdir/newfile
Given the cached rename noted above, the second merge can proceed as
expected without needing to perform rename detection from A -> A'.
Case 2: MERGE_SIDE1 renames old dir, MERGE_SIDE2 renames file into old dir
This case looks like this:
MERGE_BASE: E oldfile, olddir/
MERGE_SIDE1: G oldfile, olddir/ -> newdir/
MERGE_SIDE2: A oldfile -> olddir/newfile
=> creates A', With newdir/newfile representing original oldfile
MERGE_BASE: A olddir/newfile
MERGE_SIDE1: A' newdir/newfile
MERGE_SIDE2: B modify olddir/newfile
=> expected B', with threeway-merged newdir/newfile from above
In this case, with the optimization, note that after the first commit:
* MERGE_SIDE1 remembers olddir/ -> newdir/
* MERGE_SIDE1 has cached olddir/newfile -> newdir/newfile
(NOT oldfile -> newdir/newfile; compare to case with
(p->status == 'R' && new_path) in possibly_cache_new_pair())
Given the cached rename noted above, the second merge can proceed as
expected without needing to perform rename detection from A -> A'.
Case 3: MERGE_SIDE1 adds new file to old dir, MERGE_SIDE2 renames old dir
This case looks like this:
MERGE_BASE: E, Has olddir/
MERGE_SIDE1: G, Adds olddir/newfile
MERGE_SIDE2: A, Renames olddir/ -> newdir/
=> creates A', With newdir/newfile
MERGE_BASE: A, Has newdir/, but no notion of newdir/newfile
MERGE_SIDE1: A', Has newdir/newfile
MERGE_SIDE2: B, Has newdir/, but no notion of newdir/newfile
=> expected B', with newdir/newfile from A'
In this case, with the optimization, note that after the first commit there
were no renames on MERGE_SIDE1, and any renames on MERGE_SIDE2 are tossed.
But the second merge didn't need any renames so this is fine.
Case 4: MERGE_SIDE1 renames file into old dir, MERGE_SIDE2 renames old dir
This case looks like this:
MERGE_BASE: E, Has olddir/
MERGE_SIDE1: G, Renames oldfile -> olddir/newfile
MERGE_SIDE2: A, Renames olddir/ -> newdir/
=> creates A', With newdir/newfile representing original oldfile
MERGE_BASE: A, Has oldfile
MERGE_SIDE1: A', Has newdir/newfile
MERGE_SIDE2: B, Modifies oldfile
=> expected B', with threeway-merged newdir/newfile from above
In this case, with the optimization, note that after the first commit:
* MERGE_SIDE1 remembers oldfile -> newdir/newfile
(NOT oldfile -> olddir/newfile; compare to case of second
block under p->status == 'R' in possibly_cache_new_pair())
* MERGE_SIDE2 renames are tossed because only MERGE_SIDE1 is remembered
Given the cached rename noted above, the second merge can proceed as
expected without needing to perform rename detection from A -> A'.
Finally, I'll just note here that interactions with the
skip-irrelevant-renames optimization means we sometimes don't detect
renames for any files within a directory that was renamed, in which
case we will not have been able to detect any rename for the directory
itself. In such a case, we do not know whether the directory was
renamed; we want to be careful to avoid caching some kind of "this
directory was not renamed" statement. If we did, then a subsequent
commit being rebased could add a file to the old directory, and the
user would expect it to end up in the correct directory -- something
our erroneous "this directory was not renamed" cache would preclude.

View File

@@ -0,0 +1,70 @@
== Git Repository Format Versions
Every git repository is marked with a numeric version in the
`core.repositoryformatversion` key of its `config` file. This version
specifies the rules for operating on the on-disk repository data. An
implementation of git which does not understand a particular version
advertised by an on-disk repository MUST NOT operate on that repository;
doing so risks not only producing wrong results, but actually losing
data.
Because of this rule, version bumps should be kept to an absolute
minimum. Instead, we generally prefer these strategies:
- bumping format version numbers of individual data files (e.g.,
index, packfiles, etc). This restricts the incompatibilities only to
those files.
- introducing new data that gracefully degrades when used by older
clients (e.g., pack bitmap files are ignored by older clients, which
simply do not take advantage of the optimization they provide).
A whole-repository format version bump should only be part of a change
that cannot be independently versioned. For instance, if one were to
change the reachability rules for objects, or the rules for locking
refs, that would require a bump of the repository format version.
Note that this applies only to accessing the repository's disk contents
directly. An older client which understands only format `0` may still
connect via `git://` to a repository using format `1`, as long as the
server process understands format `1`.
The preferred strategy for rolling out a version bump (whether whole
repository or for a single file) is to teach git to read the new format,
and allow writing the new format with a config switch or command line
option (for experimentation or for those who do not care about backwards
compatibility with older gits). Then after a long period to allow the
reading capability to become common, we may switch to writing the new
format by default.
The currently defined format versions are:
=== Version `0`
This is the format defined by the initial version of git, including but
not limited to the format of the repository directory, the repository
configuration file, and the object and ref storage. Specifying the
complete behavior of git is beyond the scope of this document.
=== Version `1`
This format is identical to version `0`, with the following exceptions:
1. When reading the `core.repositoryformatversion` variable, a git
implementation which supports version 1 MUST also read any
configuration keys found in the `extensions` section of the
configuration file.
2. If a version-1 repository specifies any `extensions.*` keys that
the running git has not implemented, the operation MUST NOT
proceed. Similarly, if the value of any known key is not understood
by the implementation, the operation MUST NOT proceed.
Note that if no extensions are specified in the config file, then
`core.repositoryformatversion` SHOULD be set to `0` (setting it to `1`
provides no benefit, and makes the repository incompatible with older
implementations of git).
The defined extensions are given in the `extensions.*` section of
linkgit:git-config[1]. Any implementation wishing to define a new
extension should make a note of it there, in order to claim the name.

View File

@@ -0,0 +1,186 @@
Rerere
======
This document describes the rerere logic.
Conflict normalization
----------------------
To ensure recorded conflict resolutions can be looked up in the rerere
database, even when branches are merged in a different order,
different branches are merged that result in the same conflict, or
when different conflict style settings are used, rerere normalizes the
conflicts before writing them to the rerere database.
Different conflict styles and branch names are normalized by stripping
the labels from the conflict markers, and removing the common ancestor
version from the `diff3` or `zdiff3` conflict styles. Branches that
are merged in different order are normalized by sorting the conflict
hunks. More on each of those steps in the following sections.
Once these two normalization operations are applied, a conflict ID is
calculated based on the normalized conflict, which is later used by
rerere to look up the conflict in the rerere database.
Removing the common ancestor version
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Say we have three branches AB, AC and AC2. The common ancestor of
these branches has a file with a line containing the string "A" (for
brevity this is called "line A" in the rest of the document). In
branch AB this line is changed to "B", in AC, this line is changed to
"C", and branch AC2 is forked off of AC, after the line was changed to
"C".
Forking a branch ABAC off of branch AB and then merging AC into it, we
get a conflict like the following:
<<<<<<< HEAD
B
=======
C
>>>>>>> AC
Doing the analogous with AC2 (forking a branch ABAC2 off of branch AB
and then merging branch AC2 into it), using the diff3 or zdiff3
conflict style, we get a conflict like the following:
<<<<<<< HEAD
B
||||||| merged common ancestors
A
=======
C
>>>>>>> AC2
By resolving this conflict, to leave line D, the user declares:
After examining what branches AB and AC did, I believe that making
line A into line D is the best thing to do that is compatible with
what AB and AC wanted to do.
As branch AC2 refers to the same commit as AC, the above implies that
this is also compatible with what AB and AC2 wanted to do.
By extension, this means that rerere should recognize that the above
conflicts are the same. To do this, the labels on the conflict
markers are stripped, and the common ancestor version is removed. The above
examples would both result in the following normalized conflict:
<<<<<<<
B
=======
C
>>>>>>>
Sorting hunks
~~~~~~~~~~~~~
As before, let's imagine that a common ancestor had a file with line A
its early part, and line X in its late part. And then four branches
are forked that do these things:
- AB: changes A to B
- AC: changes A to C
- XY: changes X to Y
- XZ: changes X to Z
Now, forking a branch ABAC off of branch AB and then merging AC into
it, and forking a branch ACAB off of branch AC and then merging AB
into it, would yield the conflict in a different order. The former
would say "A became B or C, what now?" while the latter would say "A
became C or B, what now?"
As a reminder, the act of merging AC into ABAC and resolving the
conflict to leave line D means that the user declares:
After examining what branches AB and AC did, I believe that
making line A into line D is the best thing to do that is
compatible with what AB and AC wanted to do.
So the conflict we would see when merging AB into ACAB should be
resolved the same way--it is the resolution that is in line with that
declaration.
Imagine that similarly previously a branch XYXZ was forked from XY,
and XZ was merged into it, and resolved "X became Y or Z" into "X
became W".
Now, if a branch ABXY was forked from AB and then merged XY, then ABXY
would have line B in its early part and line Y in its later part.
Such a merge would be quite clean. We can construct 4 combinations
using these four branches ((AB, AC) x (XY, XZ)).
Merging ABXY and ACXZ would make "an early A became B or C, a late X
became Y or Z" conflict, while merging ACXY and ABXZ would make "an
early A became C or B, a late X became Y or Z". We can see there are
4 combinations of ("B or C", "C or B") x ("X or Y", "Y or X").
By sorting, the conflict is given its canonical name, namely, "an
early part became B or C, a late part became X or Y", and whenever
any of these four patterns appear, and we can get to the same conflict
and resolution that we saw earlier.
Without the sorting, we'd have to somehow find a previous resolution
from combinatorial explosion.
Conflict ID calculation
~~~~~~~~~~~~~~~~~~~~~~~
Once the conflict normalization is done, the conflict ID is calculated
as the sha1 hash of the conflict hunks appended to each other,
separated by <NUL> characters. The conflict markers are stripped out
before the sha1 is calculated. So in the example above, where we
merge branch AC which changes line A to line C, into branch AB, which
changes line A to line C, the conflict ID would be
SHA1('B<NUL>C<NUL>').
If there are multiple conflicts in one file, the sha1 is calculated
the same way with all hunks appended to each other, in the order in
which they appear in the file, separated by a <NUL> character.
Nested conflicts
~~~~~~~~~~~~~~~~
Nested conflicts are handled very similarly to "simple" conflicts.
Similar to simple conflicts, the conflict is first normalized by
stripping the labels from conflict markers, stripping the common ancestor
version, and sorting the conflict hunks, both for the outer and the
inner conflict. This is done recursively, so any number of nested
conflicts can be handled.
Note that this only works for conflict markers that "cleanly nest". If
there are any unmatched conflict markers, rerere will fail to handle
the conflict and record a conflict resolution.
The only difference is in how the conflict ID is calculated. For the
inner conflict, the conflict markers themselves are not stripped out
before calculating the sha1.
Say we have the following conflict for example:
<<<<<<< HEAD
1
=======
<<<<<<< HEAD
3
=======
2
>>>>>>> branch-2
>>>>>>> branch-3~
After stripping out the labels of the conflict markers, and sorting
the hunks, the conflict would look as follows:
<<<<<<<
1
=======
<<<<<<<
2
=======
3
>>>>>>>
>>>>>>>
and finally the conflict ID would be calculated as:
`sha1('1<NUL><<<<<<<\n3\n=======\n2\n>>>>>>><NUL>')`

View File

@@ -0,0 +1,66 @@
Scalar
======
Scalar is a repository management tool that optimizes Git for use in large
repositories. It accomplishes this by helping users to take advantage of
advanced performance features in Git. Unlike most other Git built-in commands,
Scalar is not executed as a subcommand of 'git'; rather, it is built as a
separate executable containing its own series of subcommands.
Background
----------
Scalar was originally designed as an add-on to Git and implemented as a .NET
Core application. It was created based on the learnings from the VFS for Git
project (another application aimed at improving the experience of working with
large repositories). As part of its initial implementation, Scalar relied on
custom features in the Microsoft fork of Git that have since been integrated
into core Git:
* partial clone,
* commit graphs,
* multi-pack index,
* sparse checkout (cone mode),
* scheduled background maintenance,
* etc
With the requisite Git functionality in place and a desire to bring the benefits
of Scalar to the larger Git community, the Scalar application itself was ported
from C# to C and integrated upstream.
Features
--------
Scalar is comprised of two major pieces of functionality: automatically
configuring built-in Git performance features and managing repository
enlistments.
The Git performance features configured by Scalar (see "Background" for
examples) confer substantial performance benefits to large repositories, but are
either too experimental to enable for all of Git yet, or only benefit large
repositories. As new features are introduced, Scalar should be updated
accordingly to incorporate them. This will prevent the tool from becoming stale
while also providing a path for more easily bringing features to the appropriate
users.
Enlistments are how Scalar knows which repositories on a user's system should
utilize Scalar-configured features. This allows it to update performance
settings when new ones are added to the tool, as well as centrally manage
repository maintenance. The enlistment structure - a root directory with a
`src/` subdirectory containing the cloned repository itself - is designed to
encourage users to route build outputs outside of the repository to avoid the
performance-limiting overhead of ignoring those files in Git.
Design
------
Scalar is implemented in C and interacts with Git via a mix of child process
invocations of Git and direct usage of `libgit.a`. Internally, it is structured
much like other built-ins with subcommands (e.g., `git stash`), containing a
`cmd_<subcommand>()` function for each subcommand, routed through a `cmd_main()`
function. Most options are unique to each subcommand, with `scalar` respecting
some "global" `git` options (e.g., `-c` and `-C`).
Because `scalar` is not invoked as a Git subcommand (like `git scalar`), it is
built and installed as its own executable in the `bin/` directory, alongside
`git`, `git-gui`, etc.

View File

@@ -0,0 +1,546 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="en">
<head>
<meta charset="UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=edge"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<meta name="generator" content="Asciidoctor 2.0.23"/>
<title>Scalar</title>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"/>
<style>
/*! Asciidoctor default stylesheet | MIT License | https://asciidoctor.org */
/* Uncomment the following line when using as a custom stylesheet */
/* @import "https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"; */
html{font-family:sans-serif;-webkit-text-size-adjust:100%}
a{background:none}
a:focus{outline:thin dotted}
a:active,a:hover{outline:0}
h1{font-size:2em;margin:.67em 0}
b,strong{font-weight:bold}
abbr{font-size:.9em}
abbr[title]{cursor:help;border-bottom:1px dotted #dddddf;text-decoration:none}
dfn{font-style:italic}
hr{height:0}
mark{background:#ff0;color:#000}
code,kbd,pre,samp{font-family:monospace;font-size:1em}
pre{white-space:pre-wrap}
q{quotes:"\201C" "\201D" "\2018" "\2019"}
small{font-size:80%}
sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline}
sup{top:-.5em}
sub{bottom:-.25em}
img{border:0}
svg:not(:root){overflow:hidden}
figure{margin:0}
audio,video{display:inline-block}
audio:not([controls]){display:none;height:0}
fieldset{border:1px solid silver;margin:0 2px;padding:.35em .625em .75em}
legend{border:0;padding:0}
button,input,select,textarea{font-family:inherit;font-size:100%;margin:0}
button,input{line-height:normal}
button,select{text-transform:none}
button,html input[type=button],input[type=reset],input[type=submit]{-webkit-appearance:button;cursor:pointer}
button[disabled],html input[disabled]{cursor:default}
input[type=checkbox],input[type=radio]{padding:0}
button::-moz-focus-inner,input::-moz-focus-inner{border:0;padding:0}
textarea{overflow:auto;vertical-align:top}
table{border-collapse:collapse;border-spacing:0}
*,::before,::after{box-sizing:border-box}
html,body{font-size:100%}
body{background:#fff;color:rgba(0,0,0,.8);padding:0;margin:0;font-family:"Noto Serif","DejaVu Serif",serif;line-height:1;position:relative;cursor:auto;-moz-tab-size:4;-o-tab-size:4;tab-size:4;word-wrap:anywhere;-moz-osx-font-smoothing:grayscale;-webkit-font-smoothing:antialiased}
a:hover{cursor:pointer}
img,object,embed{max-width:100%;height:auto}
object,embed{height:100%}
img{-ms-interpolation-mode:bicubic}
.left{float:left!important}
.right{float:right!important}
.text-left{text-align:left!important}
.text-right{text-align:right!important}
.text-center{text-align:center!important}
.text-justify{text-align:justify!important}
.hide{display:none}
img,object,svg{display:inline-block;vertical-align:middle}
textarea{height:auto;min-height:50px}
select{width:100%}
.subheader,.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{line-height:1.45;color:#7a2518;font-weight:400;margin-top:0;margin-bottom:.25em}
div,dl,dt,dd,ul,ol,li,h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6,pre,form,p,blockquote,th,td{margin:0;padding:0}
a{color:#2156a5;text-decoration:underline;line-height:inherit}
a:hover,a:focus{color:#1d4b8f}
a img{border:0}
p{line-height:1.6;margin-bottom:1.25em;text-rendering:optimizeLegibility}
p aside{font-size:.875em;line-height:1.35;font-style:italic}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{font-family:"Open Sans","DejaVu Sans",sans-serif;font-weight:300;font-style:normal;color:#ba3925;text-rendering:optimizeLegibility;margin-top:1em;margin-bottom:.5em;line-height:1.0125em}
h1 small,h2 small,h3 small,#toctitle small,.sidebarblock>.content>.title small,h4 small,h5 small,h6 small{font-size:60%;color:#e99b8f;line-height:0}
h1{font-size:2.125em}
h2{font-size:1.6875em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.375em}
h4,h5{font-size:1.125em}
h6{font-size:1em}
hr{border:solid #dddddf;border-width:1px 0 0;clear:both;margin:1.25em 0 1.1875em}
em,i{font-style:italic;line-height:inherit}
strong,b{font-weight:bold;line-height:inherit}
small{font-size:60%;line-height:inherit}
code{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:400;color:rgba(0,0,0,.9)}
ul,ol,dl{line-height:1.6;margin-bottom:1.25em;list-style-position:outside;font-family:inherit}
ul,ol{margin-left:1.5em}
ul li ul,ul li ol{margin-left:1.25em;margin-bottom:0}
ul.circle{list-style-type:circle}
ul.disc{list-style-type:disc}
ul.square{list-style-type:square}
ul.circle ul:not([class]),ul.disc ul:not([class]),ul.square ul:not([class]){list-style:inherit}
ol li ul,ol li ol{margin-left:1.25em;margin-bottom:0}
dl dt{margin-bottom:.3125em;font-weight:bold}
dl dd{margin-bottom:1.25em}
blockquote{margin:0 0 1.25em;padding:.5625em 1.25em 0 1.1875em;border-left:1px solid #ddd}
blockquote,blockquote p{line-height:1.6;color:rgba(0,0,0,.85)}
@media screen and (min-width:768px){h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2}
h1{font-size:2.75em}
h2{font-size:2.3125em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.6875em}
h4{font-size:1.4375em}}
table{background:#fff;margin-bottom:1.25em;border:1px solid #dedede;word-wrap:normal}
table thead,table tfoot{background:#f7f8f7}
table thead tr th,table thead tr td,table tfoot tr th,table tfoot tr td{padding:.5em .625em .625em;font-size:inherit;color:rgba(0,0,0,.8);text-align:left}
table tr th,table tr td{padding:.5625em .625em;font-size:inherit;color:rgba(0,0,0,.8)}
table tr.even,table tr.alt{background:#f8f8f7}
table thead tr th,table tfoot tr th,table tbody tr td,table tr td,table tfoot tr td{line-height:1.6}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2;word-spacing:-.05em}
h1 strong,h2 strong,h3 strong,#toctitle strong,.sidebarblock>.content>.title strong,h4 strong,h5 strong,h6 strong{font-weight:400}
.center{margin-left:auto;margin-right:auto}
.stretch{width:100%}
.clearfix::before,.clearfix::after,.float-group::before,.float-group::after{content:" ";display:table}
.clearfix::after,.float-group::after{clear:both}
:not(pre).nobreak{word-wrap:normal}
:not(pre).nowrap{white-space:nowrap}
:not(pre).pre-wrap{white-space:pre-wrap}
:not(pre):not([class^=L])>code{font-size:.9375em;font-style:normal!important;letter-spacing:0;padding:.1em .5ex;word-spacing:-.15em;background:#f7f7f8;border-radius:4px;line-height:1.45;text-rendering:optimizeSpeed}
pre{color:rgba(0,0,0,.9);font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;line-height:1.45;text-rendering:optimizeSpeed}
pre code,pre pre{color:inherit;font-size:inherit;line-height:inherit}
pre>code{display:block}
pre.nowrap,pre.nowrap pre{white-space:pre;word-wrap:normal}
em em{font-style:normal}
strong strong{font-weight:400}
.keyseq{color:rgba(51,51,51,.8)}
kbd{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;display:inline-block;color:rgba(0,0,0,.8);font-size:.65em;line-height:1.45;background:#f7f7f7;border:1px solid #ccc;border-radius:3px;box-shadow:0 1px 0 rgba(0,0,0,.2),inset 0 0 0 .1em #fff;margin:0 .15em;padding:.2em .5em;vertical-align:middle;position:relative;top:-.1em;white-space:nowrap}
.keyseq kbd:first-child{margin-left:0}
.keyseq kbd:last-child{margin-right:0}
.menuseq,.menuref{color:#000}
.menuseq b:not(.caret),.menuref{font-weight:inherit}
.menuseq{word-spacing:-.02em}
.menuseq b.caret{font-size:1.25em;line-height:.8}
.menuseq i.caret{font-weight:bold;text-align:center;width:.45em}
b.button::before,b.button::after{position:relative;top:-1px;font-weight:400}
b.button::before{content:"[";padding:0 3px 0 2px}
b.button::after{content:"]";padding:0 2px 0 3px}
p a>code:hover{color:rgba(0,0,0,.9)}
#header,#content,#footnotes,#footer{width:100%;margin:0 auto;max-width:62.5em;*zoom:1;position:relative;padding-left:.9375em;padding-right:.9375em}
#header::before,#header::after,#content::before,#content::after,#footnotes::before,#footnotes::after,#footer::before,#footer::after{content:" ";display:table}
#header::after,#content::after,#footnotes::after,#footer::after{clear:both}
#content{margin-top:1.25em}
#content::before{content:none}
#header>h1:first-child{color:rgba(0,0,0,.85);margin-top:2.25rem;margin-bottom:0}
#header>h1:first-child+#toc{margin-top:8px;border-top:1px solid #dddddf}
#header>h1:only-child{border-bottom:1px solid #dddddf;padding-bottom:8px}
#header .details{border-bottom:1px solid #dddddf;line-height:1.45;padding-top:.25em;padding-bottom:.25em;padding-left:.25em;color:rgba(0,0,0,.6);display:flex;flex-flow:row wrap}
#header .details span:first-child{margin-left:-.125em}
#header .details span.email a{color:rgba(0,0,0,.85)}
#header .details br{display:none}
#header .details br+span::before{content:"\00a0\2013\00a0"}
#header .details br+span.author::before{content:"\00a0\22c5\00a0";color:rgba(0,0,0,.85)}
#header .details br+span#revremark::before{content:"\00a0|\00a0"}
#header #revnumber{text-transform:capitalize}
#header #revnumber::after{content:"\00a0"}
#content>h1:first-child:not([class]){color:rgba(0,0,0,.85);border-bottom:1px solid #dddddf;padding-bottom:8px;margin-top:0;padding-top:1rem;margin-bottom:1.25rem}
#toc{border-bottom:1px solid #e7e7e9;padding-bottom:.5em}
#toc>ul{margin-left:.125em}
#toc ul.sectlevel0>li>a{font-style:italic}
#toc ul.sectlevel0 ul.sectlevel1{margin:.5em 0}
#toc ul{font-family:"Open Sans","DejaVu Sans",sans-serif;list-style-type:none}
#toc li{line-height:1.3334;margin-top:.3334em}
#toc a{text-decoration:none}
#toc a:active{text-decoration:underline}
#toctitle{color:#7a2518;font-size:1.2em}
@media screen and (min-width:768px){#toctitle{font-size:1.375em}
body.toc2{padding-left:15em;padding-right:0}
body.toc2 #header>h1:nth-last-child(2){border-bottom:1px solid #dddddf;padding-bottom:8px}
#toc.toc2{margin-top:0!important;background:#f8f8f7;position:fixed;width:15em;left:0;top:0;border-right:1px solid #e7e7e9;border-top-width:0!important;border-bottom-width:0!important;z-index:1000;padding:1.25em 1em;height:100%;overflow:auto}
#toc.toc2 #toctitle{margin-top:0;margin-bottom:.8rem;font-size:1.2em}
#toc.toc2>ul{font-size:.9em;margin-bottom:0}
#toc.toc2 ul ul{margin-left:0;padding-left:1em}
#toc.toc2 ul.sectlevel0 ul.sectlevel1{padding-left:0;margin-top:.5em;margin-bottom:.5em}
body.toc2.toc-right{padding-left:0;padding-right:15em}
body.toc2.toc-right #toc.toc2{border-right-width:0;border-left:1px solid #e7e7e9;left:auto;right:0}}
@media screen and (min-width:1280px){body.toc2{padding-left:20em;padding-right:0}
#toc.toc2{width:20em}
#toc.toc2 #toctitle{font-size:1.375em}
#toc.toc2>ul{font-size:.95em}
#toc.toc2 ul ul{padding-left:1.25em}
body.toc2.toc-right{padding-left:0;padding-right:20em}}
#content #toc{border:1px solid #e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;border-radius:4px}
#content #toc>:first-child{margin-top:0}
#content #toc>:last-child{margin-bottom:0}
#footer{max-width:none;background:rgba(0,0,0,.8);padding:1.25em}
#footer-text{color:hsla(0,0%,100%,.8);line-height:1.44}
#content{margin-bottom:.625em}
.sect1{padding-bottom:.625em}
@media screen and (min-width:768px){#content{margin-bottom:1.25em}
.sect1{padding-bottom:1.25em}}
.sect1:last-child{padding-bottom:0}
.sect1+.sect1{border-top:1px solid #e7e7e9}
#content h1>a.anchor,h2>a.anchor,h3>a.anchor,#toctitle>a.anchor,.sidebarblock>.content>.title>a.anchor,h4>a.anchor,h5>a.anchor,h6>a.anchor{position:absolute;z-index:1001;width:1.5ex;margin-left:-1.5ex;display:block;text-decoration:none!important;visibility:hidden;text-align:center;font-weight:400}
#content h1>a.anchor::before,h2>a.anchor::before,h3>a.anchor::before,#toctitle>a.anchor::before,.sidebarblock>.content>.title>a.anchor::before,h4>a.anchor::before,h5>a.anchor::before,h6>a.anchor::before{content:"\00A7";font-size:.85em;display:block;padding-top:.1em}
#content h1:hover>a.anchor,#content h1>a.anchor:hover,h2:hover>a.anchor,h2>a.anchor:hover,h3:hover>a.anchor,#toctitle:hover>a.anchor,.sidebarblock>.content>.title:hover>a.anchor,h3>a.anchor:hover,#toctitle>a.anchor:hover,.sidebarblock>.content>.title>a.anchor:hover,h4:hover>a.anchor,h4>a.anchor:hover,h5:hover>a.anchor,h5>a.anchor:hover,h6:hover>a.anchor,h6>a.anchor:hover{visibility:visible}
#content h1>a.link,h2>a.link,h3>a.link,#toctitle>a.link,.sidebarblock>.content>.title>a.link,h4>a.link,h5>a.link,h6>a.link{color:#ba3925;text-decoration:none}
#content h1>a.link:hover,h2>a.link:hover,h3>a.link:hover,#toctitle>a.link:hover,.sidebarblock>.content>.title>a.link:hover,h4>a.link:hover,h5>a.link:hover,h6>a.link:hover{color:#a53221}
details,.audioblock,.imageblock,.literalblock,.listingblock,.stemblock,.videoblock{margin-bottom:1.25em}
details{margin-left:1.25rem}
details>summary{cursor:pointer;display:block;position:relative;line-height:1.6;margin-bottom:.625rem;outline:none;-webkit-tap-highlight-color:transparent}
details>summary::-webkit-details-marker{display:none}
details>summary::before{content:"";border:solid transparent;border-left:solid;border-width:.3em 0 .3em .5em;position:absolute;top:.5em;left:-1.25rem;transform:translateX(15%)}
details[open]>summary::before{border:solid transparent;border-top:solid;border-width:.5em .3em 0;transform:translateY(15%)}
details>summary::after{content:"";width:1.25rem;height:1em;position:absolute;top:.3em;left:-1.25rem}
.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{text-rendering:optimizeLegibility;text-align:left;font-family:"Noto Serif","DejaVu Serif",serif;font-size:1rem;font-style:italic}
table.tableblock.fit-content>caption.title{white-space:nowrap;width:0}
.paragraph.lead>p,#preamble>.sectionbody>[class=paragraph]:first-of-type p{font-size:1.21875em;line-height:1.6;color:rgba(0,0,0,.85)}
.admonitionblock>table{border-collapse:separate;border:0;background:none;width:100%}
.admonitionblock>table td.icon{text-align:center;width:80px}
.admonitionblock>table td.icon img{max-width:none}
.admonitionblock>table td.icon .title{font-weight:bold;font-family:"Open Sans","DejaVu Sans",sans-serif;text-transform:uppercase}
.admonitionblock>table td.content{padding-left:1.125em;padding-right:1.25em;border-left:1px solid #dddddf;color:rgba(0,0,0,.6);word-wrap:anywhere}
.admonitionblock>table td.content>:last-child>:last-child{margin-bottom:0}
.exampleblock>.content{border:1px solid #e6e6e6;margin-bottom:1.25em;padding:1.25em;background:#fff;border-radius:4px}
.sidebarblock{border:1px solid #dbdbd6;margin-bottom:1.25em;padding:1.25em;background:#f3f3f2;border-radius:4px}
.sidebarblock>.content>.title{color:#7a2518;margin-top:0;text-align:center}
.exampleblock>.content>:first-child,.sidebarblock>.content>:first-child{margin-top:0}
.exampleblock>.content>:last-child,.exampleblock>.content>:last-child>:last-child,.exampleblock>.content .olist>ol>li:last-child>:last-child,.exampleblock>.content .ulist>ul>li:last-child>:last-child,.exampleblock>.content .qlist>ol>li:last-child>:last-child,.sidebarblock>.content>:last-child,.sidebarblock>.content>:last-child>:last-child,.sidebarblock>.content .olist>ol>li:last-child>:last-child,.sidebarblock>.content .ulist>ul>li:last-child>:last-child,.sidebarblock>.content .qlist>ol>li:last-child>:last-child{margin-bottom:0}
.literalblock pre,.listingblock>.content>pre{border-radius:4px;overflow-x:auto;padding:1em;font-size:.8125em}
@media screen and (min-width:768px){.literalblock pre,.listingblock>.content>pre{font-size:.90625em}}
@media screen and (min-width:1280px){.literalblock pre,.listingblock>.content>pre{font-size:1em}}
.literalblock pre,.listingblock>.content>pre:not(.highlight),.listingblock>.content>pre[class=highlight],.listingblock>.content>pre[class^="highlight "]{background:#f7f7f8}
.literalblock.output pre{color:#f7f7f8;background:rgba(0,0,0,.9)}
.listingblock>.content{position:relative}
.listingblock code[data-lang]::before{display:none;content:attr(data-lang);position:absolute;font-size:.75em;top:.425rem;right:.5rem;line-height:1;text-transform:uppercase;color:inherit;opacity:.5}
.listingblock:hover code[data-lang]::before{display:block}
.listingblock.terminal pre .command::before{content:attr(data-prompt);padding-right:.5em;color:inherit;opacity:.5}
.listingblock.terminal pre .command:not([data-prompt])::before{content:"$"}
.listingblock pre.highlightjs{padding:0}
.listingblock pre.highlightjs>code{padding:1em;border-radius:4px}
.listingblock pre.prettyprint{border-width:0}
.prettyprint{background:#f7f7f8}
pre.prettyprint .linenums{line-height:1.45;margin-left:2em}
pre.prettyprint li{background:none;list-style-type:inherit;padding-left:0}
pre.prettyprint li code[data-lang]::before{opacity:1}
pre.prettyprint li:not(:first-child) code[data-lang]::before{display:none}
table.linenotable{border-collapse:separate;border:0;margin-bottom:0;background:none}
table.linenotable td[class]{color:inherit;vertical-align:top;padding:0;line-height:inherit;white-space:normal}
table.linenotable td.code{padding-left:.75em}
table.linenotable td.linenos,pre.pygments .linenos{border-right:1px solid;opacity:.35;padding-right:.5em;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}
pre.pygments span.linenos{display:inline-block;margin-right:.75em}
.quoteblock{margin:0 1em 1.25em 1.5em;display:table}
.quoteblock:not(.excerpt)>.title{margin-left:-1.5em;margin-bottom:.75em}
.quoteblock blockquote,.quoteblock p{color:rgba(0,0,0,.85);font-size:1.15rem;line-height:1.75;word-spacing:.1em;letter-spacing:0;font-style:italic;text-align:justify}
.quoteblock blockquote{margin:0;padding:0;border:0}
.quoteblock blockquote::before{content:"\201c";float:left;font-size:2.75em;font-weight:bold;line-height:.6em;margin-left:-.6em;color:#7a2518;text-shadow:0 1px 2px rgba(0,0,0,.1)}
.quoteblock blockquote>.paragraph:last-child p{margin-bottom:0}
.quoteblock .attribution{margin-top:.75em;margin-right:.5ex;text-align:right}
.verseblock{margin:0 1em 1.25em}
.verseblock pre{font-family:"Open Sans","DejaVu Sans",sans-serif;font-size:1.15rem;color:rgba(0,0,0,.85);font-weight:300;text-rendering:optimizeLegibility}
.verseblock pre strong{font-weight:400}
.verseblock .attribution{margin-top:1.25rem;margin-left:.5ex}
.quoteblock .attribution,.verseblock .attribution{font-size:.9375em;line-height:1.45;font-style:italic}
.quoteblock .attribution br,.verseblock .attribution br{display:none}
.quoteblock .attribution cite,.verseblock .attribution cite{display:block;letter-spacing:-.025em;color:rgba(0,0,0,.6)}
.quoteblock.abstract blockquote::before,.quoteblock.excerpt blockquote::before,.quoteblock .quoteblock blockquote::before{display:none}
.quoteblock.abstract blockquote,.quoteblock.abstract p,.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{line-height:1.6;word-spacing:0}
.quoteblock.abstract{margin:0 1em 1.25em;display:block}
.quoteblock.abstract>.title{margin:0 0 .375em;font-size:1.15em;text-align:center}
.quoteblock.excerpt>blockquote,.quoteblock .quoteblock{padding:0 0 .25em 1em;border-left:.25em solid #dddddf}
.quoteblock.excerpt,.quoteblock .quoteblock{margin-left:0}
.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{color:inherit;font-size:1.0625rem}
.quoteblock.excerpt .attribution,.quoteblock .quoteblock .attribution{color:inherit;font-size:.85rem;text-align:left;margin-right:0}
p.tableblock:last-child{margin-bottom:0}
td.tableblock>.content{margin-bottom:1.25em;word-wrap:anywhere}
td.tableblock>.content>:last-child{margin-bottom:-1.25em}
table.tableblock,th.tableblock,td.tableblock{border:0 solid #dedede}
table.grid-all>*>tr>*{border-width:1px}
table.grid-cols>*>tr>*{border-width:0 1px}
table.grid-rows>*>tr>*{border-width:1px 0}
table.frame-all{border-width:1px}
table.frame-ends{border-width:1px 0}
table.frame-sides{border-width:0 1px}
table.frame-none>colgroup+*>:first-child>*,table.frame-sides>colgroup+*>:first-child>*{border-top-width:0}
table.frame-none>:last-child>:last-child>*,table.frame-sides>:last-child>:last-child>*{border-bottom-width:0}
table.frame-none>*>tr>:first-child,table.frame-ends>*>tr>:first-child{border-left-width:0}
table.frame-none>*>tr>:last-child,table.frame-ends>*>tr>:last-child{border-right-width:0}
table.stripes-all>*>tr,table.stripes-odd>*>tr:nth-of-type(odd),table.stripes-even>*>tr:nth-of-type(even),table.stripes-hover>*>tr:hover{background:#f8f8f7}
th.halign-left,td.halign-left{text-align:left}
th.halign-right,td.halign-right{text-align:right}
th.halign-center,td.halign-center{text-align:center}
th.valign-top,td.valign-top{vertical-align:top}
th.valign-bottom,td.valign-bottom{vertical-align:bottom}
th.valign-middle,td.valign-middle{vertical-align:middle}
table thead th,table tfoot th{font-weight:bold}
tbody tr th{background:#f7f8f7}
tbody tr th,tbody tr th p,tfoot tr th,tfoot tr th p{color:rgba(0,0,0,.8);font-weight:bold}
p.tableblock>code:only-child{background:none;padding:0}
p.tableblock{font-size:1em}
ol{margin-left:1.75em}
ul li ol{margin-left:1.5em}
dl dd{margin-left:1.125em}
dl dd:last-child,dl dd:last-child>:last-child{margin-bottom:0}
li p,ul dd,ol dd,.olist .olist,.ulist .ulist,.ulist .olist,.olist .ulist{margin-bottom:.625em}
ul.checklist,ul.none,ol.none,ul.no-bullet,ol.no-bullet,ol.unnumbered,ul.unstyled,ol.unstyled{list-style-type:none}
ul.no-bullet,ol.no-bullet,ol.unnumbered{margin-left:.625em}
ul.unstyled,ol.unstyled{margin-left:0}
li>p:empty:only-child::before{content:"";display:inline-block}
ul.checklist>li>p:first-child{margin-left:-1em}
ul.checklist>li>p:first-child>.fa-square-o:first-child,ul.checklist>li>p:first-child>.fa-check-square-o:first-child{width:1.25em;font-size:.8em;position:relative;bottom:.125em}
ul.checklist>li>p:first-child>input[type=checkbox]:first-child{margin-right:.25em}
ul.inline{display:flex;flex-flow:row wrap;list-style:none;margin:0 0 .625em -1.25em}
ul.inline>li{margin-left:1.25em}
.unstyled dl dt{font-weight:400;font-style:normal}
ol.arabic{list-style-type:decimal}
ol.decimal{list-style-type:decimal-leading-zero}
ol.loweralpha{list-style-type:lower-alpha}
ol.upperalpha{list-style-type:upper-alpha}
ol.lowerroman{list-style-type:lower-roman}
ol.upperroman{list-style-type:upper-roman}
ol.lowergreek{list-style-type:lower-greek}
.hdlist>table,.colist>table{border:0;background:none}
.hdlist>table>tbody>tr,.colist>table>tbody>tr{background:none}
td.hdlist1,td.hdlist2{vertical-align:top;padding:0 .625em}
td.hdlist1{font-weight:bold;padding-bottom:1.25em}
td.hdlist2{word-wrap:anywhere}
.literalblock+.colist,.listingblock+.colist{margin-top:-.5em}
.colist td:not([class]):first-child{padding:.4em .75em 0;line-height:1;vertical-align:top}
.colist td:not([class]):first-child img{max-width:none}
.colist td:not([class]):last-child{padding:.25em 0}
.thumb,.th{line-height:0;display:inline-block;border:4px solid #fff;box-shadow:0 0 0 1px #ddd}
.imageblock.left{margin:.25em .625em 1.25em 0}
.imageblock.right{margin:.25em 0 1.25em .625em}
.imageblock>.title{margin-bottom:0}
.imageblock.thumb,.imageblock.th{border-width:6px}
.imageblock.thumb>.title,.imageblock.th>.title{padding:0 .125em}
.image.left,.image.right{margin-top:.25em;margin-bottom:.25em;display:inline-block;line-height:0}
.image.left{margin-right:.625em}
.image.right{margin-left:.625em}
a.image{text-decoration:none;display:inline-block}
a.image object{pointer-events:none}
sup.footnote,sup.footnoteref{font-size:.875em;position:static;vertical-align:super}
sup.footnote a,sup.footnoteref a{text-decoration:none}
sup.footnote a:active,sup.footnoteref a:active,#footnotes .footnote a:first-of-type:active{text-decoration:underline}
#footnotes{padding-top:.75em;padding-bottom:.75em;margin-bottom:.625em}
#footnotes hr{width:20%;min-width:6.25em;margin:-.25em 0 .75em;border-width:1px 0 0}
#footnotes .footnote{padding:0 .375em 0 .225em;line-height:1.3334;font-size:.875em;margin-left:1.2em;margin-bottom:.2em}
#footnotes .footnote a:first-of-type{font-weight:bold;text-decoration:none;margin-left:-1.05em}
#footnotes .footnote:last-of-type{margin-bottom:0}
#content #footnotes{margin-top:-.625em;margin-bottom:0;padding:.75em 0}
div.unbreakable{page-break-inside:avoid}
.big{font-size:larger}
.small{font-size:smaller}
.underline{text-decoration:underline}
.overline{text-decoration:overline}
.line-through{text-decoration:line-through}
.aqua{color:#00bfbf}
.aqua-background{background:#00fafa}
.black{color:#000}
.black-background{background:#000}
.blue{color:#0000bf}
.blue-background{background:#0000fa}
.fuchsia{color:#bf00bf}
.fuchsia-background{background:#fa00fa}
.gray{color:#606060}
.gray-background{background:#7d7d7d}
.green{color:#006000}
.green-background{background:#007d00}
.lime{color:#00bf00}
.lime-background{background:#00fa00}
.maroon{color:#600000}
.maroon-background{background:#7d0000}
.navy{color:#000060}
.navy-background{background:#00007d}
.olive{color:#606000}
.olive-background{background:#7d7d00}
.purple{color:#600060}
.purple-background{background:#7d007d}
.red{color:#bf0000}
.red-background{background:#fa0000}
.silver{color:#909090}
.silver-background{background:#bcbcbc}
.teal{color:#006060}
.teal-background{background:#007d7d}
.white{color:#bfbfbf}
.white-background{background:#fafafa}
.yellow{color:#bfbf00}
.yellow-background{background:#fafa00}
span.icon>.fa{cursor:default}
a span.icon>.fa{cursor:inherit}
.admonitionblock td.icon [class^="fa icon-"]{font-size:2.5em;text-shadow:1px 1px 2px rgba(0,0,0,.5);cursor:default}
.admonitionblock td.icon .icon-note::before{content:"\f05a";color:#19407c}
.admonitionblock td.icon .icon-tip::before{content:"\f0eb";text-shadow:1px 1px 2px rgba(155,155,0,.8);color:#111}
.admonitionblock td.icon .icon-warning::before{content:"\f071";color:#bf6900}
.admonitionblock td.icon .icon-caution::before{content:"\f06d";color:#bf3400}
.admonitionblock td.icon .icon-important::before{content:"\f06a";color:#bf0000}
.conum[data-value]{display:inline-block;color:#fff!important;background:rgba(0,0,0,.8);border-radius:50%;text-align:center;font-size:.75em;width:1.67em;height:1.67em;line-height:1.67em;font-family:"Open Sans","DejaVu Sans",sans-serif;font-style:normal;font-weight:bold}
.conum[data-value] *{color:#fff!important}
.conum[data-value]+b{display:none}
.conum[data-value]::after{content:attr(data-value)}
pre .conum[data-value]{position:relative;top:-.125em}
b.conum *{color:inherit!important}
.conum:not([data-value]):empty{display:none}
dt,th.tableblock,td.content,div.footnote{text-rendering:optimizeLegibility}
h1,h2,p,td.content,span.alt,summary{letter-spacing:-.01em}
p strong,td.content strong,div.footnote strong{letter-spacing:-.005em}
p,blockquote,dt,td.content,td.hdlist1,span.alt,summary{font-size:1.0625rem}
p{margin-bottom:1.25rem}
.sidebarblock p,.sidebarblock dt,.sidebarblock td.content,p.tableblock{font-size:1em}
.exampleblock>.content{background:#fffef7;border-color:#e0e0dc;box-shadow:0 1px 4px #e0e0dc}
.print-only{display:none!important}
@page{margin:1.25cm .75cm}
@media print{*{box-shadow:none!important;text-shadow:none!important}
html{font-size:80%}
a{color:inherit!important;text-decoration:underline!important}
a.bare,a[href^="#"],a[href^="mailto:"]{text-decoration:none!important}
a[href^="http:"]:not(.bare)::after,a[href^="https:"]:not(.bare)::after{content:"(" attr(href) ")";display:inline-block;font-size:.875em;padding-left:.25em}
abbr[title]{border-bottom:1px dotted}
abbr[title]::after{content:" (" attr(title) ")"}
pre,blockquote,tr,img,object,svg{page-break-inside:avoid}
thead{display:table-header-group}
svg{max-width:100%}
p,blockquote,dt,td.content{font-size:1em;orphans:3;widows:3}
h2,h3,#toctitle,.sidebarblock>.content>.title{page-break-after:avoid}
#header,#content,#footnotes,#footer{max-width:none}
#toc,.sidebarblock,.exampleblock>.content{background:none!important}
#toc{border-bottom:1px solid #dddddf!important;padding-bottom:0!important}
body.book #header{text-align:center}
body.book #header>h1:first-child{border:0!important;margin:2.5em 0 1em}
body.book #header .details{border:0!important;display:block;padding:0!important}
body.book #header .details span:first-child{margin-left:0!important}
body.book #header .details br{display:block}
body.book #header .details br+span::before{content:none!important}
body.book #toc{border:0!important;text-align:left!important;padding:0!important;margin:0!important}
body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-break-before:always}
.listingblock code[data-lang]::before{display:block}
#footer{padding:0 .9375em}
.hide-on-print{display:none!important}
.print-only{display:block!important}
.hide-for-print{display:none!important}
.show-for-print{display:inherit!important}}
@media amzn-kf8,print{#header>h1:first-child{margin-top:1.25rem}
.sect1{padding:0!important}
.sect1+.sect1{border:0}
#footer{background:none}
#footer-text{color:rgba(0,0,0,.6);font-size:.9em}}
@media amzn-kf8{#header,#content,#footnotes,#footer{padding:0}}
</style>
</head>
<body class="article">
<div id="header">
<h1>Scalar</h1>
</div>
<div id="content">
<div id="preamble">
<div class="sectionbody">
<div class="paragraph">
<p>Scalar is a repository management tool that optimizes Git for use in large
repositories. It accomplishes this by helping users to take advantage of
advanced performance features in Git. Unlike most other Git built-in commands,
Scalar is not executed as a subcommand of <em>git</em>; rather, it is built as a
separate executable containing its own series of subcommands.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_background">Background</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Scalar was originally designed as an add-on to Git and implemented as a .NET
Core application. It was created based on the learnings from the VFS for Git
project (another application aimed at improving the experience of working with
large repositories). As part of its initial implementation, Scalar relied on
custom features in the Microsoft fork of Git that have since been integrated
into core Git:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>partial clone,</p>
</li>
<li>
<p>commit graphs,</p>
</li>
<li>
<p>multi-pack index,</p>
</li>
<li>
<p>sparse checkout (cone mode),</p>
</li>
<li>
<p>scheduled background maintenance,</p>
</li>
<li>
<p>etc</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>With the requisite Git functionality in place and a desire to bring the benefits
of Scalar to the larger Git community, the Scalar application itself was ported
from C# to C and integrated upstream.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_features">Features</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Scalar is comprised of two major pieces of functionality: automatically
configuring built-in Git performance features and managing repository
enlistments.</p>
</div>
<div class="paragraph">
<p>The Git performance features configured by Scalar (see "Background" for
examples) confer substantial performance benefits to large repositories, but are
either too experimental to enable for all of Git yet, or only benefit large
repositories. As new features are introduced, Scalar should be updated
accordingly to incorporate them. This will prevent the tool from becoming stale
while also providing a path for more easily bringing features to the appropriate
users.</p>
</div>
<div class="paragraph">
<p>Enlistments are how Scalar knows which repositories on a user&#8217;s system should
utilize Scalar-configured features. This allows it to update performance
settings when new ones are added to the tool, as well as centrally manage
repository maintenance. The enlistment structure - a root directory with a
<code>src/</code> subdirectory containing the cloned repository itself - is designed to
encourage users to route build outputs outside of the repository to avoid the
performance-limiting overhead of ignoring those files in Git.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_design">Design</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Scalar is implemented in C and interacts with Git via a mix of child process
invocations of Git and direct usage of <code>libgit.a</code>. Internally, it is structured
much like other built-ins with subcommands (e.g., <code>git</code> <code>stash</code>), containing a
<code>cmd_</code><em>&lt;subcommand&gt;</em>() function for each subcommand, routed through a <code>cmd_main</code>()
function. Most options are unique to each subcommand, with <code>scalar</code> respecting
some "global" <code>git</code> options (e.g., <code>-c</code> and <code>-C</code>).</p>
</div>
<div class="paragraph">
<p>Because <code>scalar</code> is not invoked as a Git subcommand (like <code>git</code> <code>scalar</code>), it is
built and installed as its own executable in the <code>bin/</code> directory, alongside
<code>git</code>, <code>git-gui</code>, etc.</p>
</div>
</div>
</div>
</div>
<div id="footer">
<div id="footer-text">
Last updated 2025-03-17 07:09:38 UTC
</div>
</div>
</body>
</html>

View File

@@ -0,0 +1,63 @@
Git-send-pack internals
=======================
Overall operation
-----------------
. Connects to the remote side and invokes git-receive-pack.
. Learns what refs the remote has and what commit they point at.
Matches them to the refspecs we are pushing.
. Checks if there are non-fast-forwards. Unlike fetch-pack,
the repository send-pack runs in is supposed to be a superset
of the recipient in fast-forward cases, so there is no need
for want/have exchanges, and fast-forward check can be done
locally. Tell the result to the other end.
. Calls pack_objects() which generates a packfile and sends it
over to the other end.
. If the remote side is new enough (v1.1.0 or later), wait for
the unpack and hook status from the other end.
. Exit with appropriate error codes.
Pack_objects pipeline
---------------------
This function gets one file descriptor (`fd`) which is either a
socket (over the network) or a pipe (local). What's written to
this fd goes to git-receive-pack to be unpacked.
send-pack ---> fd ---> receive-pack
The function pack_objects creates a pipe and then forks. The
forked child execs pack-objects with --revs to receive revision
parameters from its standard input. This process will write the
packfile to the other end.
send-pack
|
pack_objects() ---> fd ---> receive-pack
| ^ (pipe)
v |
(child)
The child dup2's to arrange its standard output to go back to
the other end, and read its standard input to come from the
pipe. After that it exec's pack-objects. On the other hand,
the parent process, before starting to feed the child pipeline,
closes the reading side of the pipe and fd to receive-pack.
send-pack
|
pack_objects(parent)
|
v [0]
pack-objects [0] ---> receive-pack
[jc: the pipeline was much more complex and needed documentation before
I understood an earlier bug, but now it is trivial and straightforward.]

View File

@@ -0,0 +1,535 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="en">
<head>
<meta charset="UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=edge"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<meta name="generator" content="Asciidoctor 2.0.23"/>
<title>Git-send-pack internals</title>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"/>
<style>
/*! Asciidoctor default stylesheet | MIT License | https://asciidoctor.org */
/* Uncomment the following line when using as a custom stylesheet */
/* @import "https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"; */
html{font-family:sans-serif;-webkit-text-size-adjust:100%}
a{background:none}
a:focus{outline:thin dotted}
a:active,a:hover{outline:0}
h1{font-size:2em;margin:.67em 0}
b,strong{font-weight:bold}
abbr{font-size:.9em}
abbr[title]{cursor:help;border-bottom:1px dotted #dddddf;text-decoration:none}
dfn{font-style:italic}
hr{height:0}
mark{background:#ff0;color:#000}
code,kbd,pre,samp{font-family:monospace;font-size:1em}
pre{white-space:pre-wrap}
q{quotes:"\201C" "\201D" "\2018" "\2019"}
small{font-size:80%}
sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline}
sup{top:-.5em}
sub{bottom:-.25em}
img{border:0}
svg:not(:root){overflow:hidden}
figure{margin:0}
audio,video{display:inline-block}
audio:not([controls]){display:none;height:0}
fieldset{border:1px solid silver;margin:0 2px;padding:.35em .625em .75em}
legend{border:0;padding:0}
button,input,select,textarea{font-family:inherit;font-size:100%;margin:0}
button,input{line-height:normal}
button,select{text-transform:none}
button,html input[type=button],input[type=reset],input[type=submit]{-webkit-appearance:button;cursor:pointer}
button[disabled],html input[disabled]{cursor:default}
input[type=checkbox],input[type=radio]{padding:0}
button::-moz-focus-inner,input::-moz-focus-inner{border:0;padding:0}
textarea{overflow:auto;vertical-align:top}
table{border-collapse:collapse;border-spacing:0}
*,::before,::after{box-sizing:border-box}
html,body{font-size:100%}
body{background:#fff;color:rgba(0,0,0,.8);padding:0;margin:0;font-family:"Noto Serif","DejaVu Serif",serif;line-height:1;position:relative;cursor:auto;-moz-tab-size:4;-o-tab-size:4;tab-size:4;word-wrap:anywhere;-moz-osx-font-smoothing:grayscale;-webkit-font-smoothing:antialiased}
a:hover{cursor:pointer}
img,object,embed{max-width:100%;height:auto}
object,embed{height:100%}
img{-ms-interpolation-mode:bicubic}
.left{float:left!important}
.right{float:right!important}
.text-left{text-align:left!important}
.text-right{text-align:right!important}
.text-center{text-align:center!important}
.text-justify{text-align:justify!important}
.hide{display:none}
img,object,svg{display:inline-block;vertical-align:middle}
textarea{height:auto;min-height:50px}
select{width:100%}
.subheader,.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{line-height:1.45;color:#7a2518;font-weight:400;margin-top:0;margin-bottom:.25em}
div,dl,dt,dd,ul,ol,li,h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6,pre,form,p,blockquote,th,td{margin:0;padding:0}
a{color:#2156a5;text-decoration:underline;line-height:inherit}
a:hover,a:focus{color:#1d4b8f}
a img{border:0}
p{line-height:1.6;margin-bottom:1.25em;text-rendering:optimizeLegibility}
p aside{font-size:.875em;line-height:1.35;font-style:italic}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{font-family:"Open Sans","DejaVu Sans",sans-serif;font-weight:300;font-style:normal;color:#ba3925;text-rendering:optimizeLegibility;margin-top:1em;margin-bottom:.5em;line-height:1.0125em}
h1 small,h2 small,h3 small,#toctitle small,.sidebarblock>.content>.title small,h4 small,h5 small,h6 small{font-size:60%;color:#e99b8f;line-height:0}
h1{font-size:2.125em}
h2{font-size:1.6875em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.375em}
h4,h5{font-size:1.125em}
h6{font-size:1em}
hr{border:solid #dddddf;border-width:1px 0 0;clear:both;margin:1.25em 0 1.1875em}
em,i{font-style:italic;line-height:inherit}
strong,b{font-weight:bold;line-height:inherit}
small{font-size:60%;line-height:inherit}
code{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:400;color:rgba(0,0,0,.9)}
ul,ol,dl{line-height:1.6;margin-bottom:1.25em;list-style-position:outside;font-family:inherit}
ul,ol{margin-left:1.5em}
ul li ul,ul li ol{margin-left:1.25em;margin-bottom:0}
ul.circle{list-style-type:circle}
ul.disc{list-style-type:disc}
ul.square{list-style-type:square}
ul.circle ul:not([class]),ul.disc ul:not([class]),ul.square ul:not([class]){list-style:inherit}
ol li ul,ol li ol{margin-left:1.25em;margin-bottom:0}
dl dt{margin-bottom:.3125em;font-weight:bold}
dl dd{margin-bottom:1.25em}
blockquote{margin:0 0 1.25em;padding:.5625em 1.25em 0 1.1875em;border-left:1px solid #ddd}
blockquote,blockquote p{line-height:1.6;color:rgba(0,0,0,.85)}
@media screen and (min-width:768px){h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2}
h1{font-size:2.75em}
h2{font-size:2.3125em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.6875em}
h4{font-size:1.4375em}}
table{background:#fff;margin-bottom:1.25em;border:1px solid #dedede;word-wrap:normal}
table thead,table tfoot{background:#f7f8f7}
table thead tr th,table thead tr td,table tfoot tr th,table tfoot tr td{padding:.5em .625em .625em;font-size:inherit;color:rgba(0,0,0,.8);text-align:left}
table tr th,table tr td{padding:.5625em .625em;font-size:inherit;color:rgba(0,0,0,.8)}
table tr.even,table tr.alt{background:#f8f8f7}
table thead tr th,table tfoot tr th,table tbody tr td,table tr td,table tfoot tr td{line-height:1.6}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2;word-spacing:-.05em}
h1 strong,h2 strong,h3 strong,#toctitle strong,.sidebarblock>.content>.title strong,h4 strong,h5 strong,h6 strong{font-weight:400}
.center{margin-left:auto;margin-right:auto}
.stretch{width:100%}
.clearfix::before,.clearfix::after,.float-group::before,.float-group::after{content:" ";display:table}
.clearfix::after,.float-group::after{clear:both}
:not(pre).nobreak{word-wrap:normal}
:not(pre).nowrap{white-space:nowrap}
:not(pre).pre-wrap{white-space:pre-wrap}
:not(pre):not([class^=L])>code{font-size:.9375em;font-style:normal!important;letter-spacing:0;padding:.1em .5ex;word-spacing:-.15em;background:#f7f7f8;border-radius:4px;line-height:1.45;text-rendering:optimizeSpeed}
pre{color:rgba(0,0,0,.9);font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;line-height:1.45;text-rendering:optimizeSpeed}
pre code,pre pre{color:inherit;font-size:inherit;line-height:inherit}
pre>code{display:block}
pre.nowrap,pre.nowrap pre{white-space:pre;word-wrap:normal}
em em{font-style:normal}
strong strong{font-weight:400}
.keyseq{color:rgba(51,51,51,.8)}
kbd{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;display:inline-block;color:rgba(0,0,0,.8);font-size:.65em;line-height:1.45;background:#f7f7f7;border:1px solid #ccc;border-radius:3px;box-shadow:0 1px 0 rgba(0,0,0,.2),inset 0 0 0 .1em #fff;margin:0 .15em;padding:.2em .5em;vertical-align:middle;position:relative;top:-.1em;white-space:nowrap}
.keyseq kbd:first-child{margin-left:0}
.keyseq kbd:last-child{margin-right:0}
.menuseq,.menuref{color:#000}
.menuseq b:not(.caret),.menuref{font-weight:inherit}
.menuseq{word-spacing:-.02em}
.menuseq b.caret{font-size:1.25em;line-height:.8}
.menuseq i.caret{font-weight:bold;text-align:center;width:.45em}
b.button::before,b.button::after{position:relative;top:-1px;font-weight:400}
b.button::before{content:"[";padding:0 3px 0 2px}
b.button::after{content:"]";padding:0 2px 0 3px}
p a>code:hover{color:rgba(0,0,0,.9)}
#header,#content,#footnotes,#footer{width:100%;margin:0 auto;max-width:62.5em;*zoom:1;position:relative;padding-left:.9375em;padding-right:.9375em}
#header::before,#header::after,#content::before,#content::after,#footnotes::before,#footnotes::after,#footer::before,#footer::after{content:" ";display:table}
#header::after,#content::after,#footnotes::after,#footer::after{clear:both}
#content{margin-top:1.25em}
#content::before{content:none}
#header>h1:first-child{color:rgba(0,0,0,.85);margin-top:2.25rem;margin-bottom:0}
#header>h1:first-child+#toc{margin-top:8px;border-top:1px solid #dddddf}
#header>h1:only-child{border-bottom:1px solid #dddddf;padding-bottom:8px}
#header .details{border-bottom:1px solid #dddddf;line-height:1.45;padding-top:.25em;padding-bottom:.25em;padding-left:.25em;color:rgba(0,0,0,.6);display:flex;flex-flow:row wrap}
#header .details span:first-child{margin-left:-.125em}
#header .details span.email a{color:rgba(0,0,0,.85)}
#header .details br{display:none}
#header .details br+span::before{content:"\00a0\2013\00a0"}
#header .details br+span.author::before{content:"\00a0\22c5\00a0";color:rgba(0,0,0,.85)}
#header .details br+span#revremark::before{content:"\00a0|\00a0"}
#header #revnumber{text-transform:capitalize}
#header #revnumber::after{content:"\00a0"}
#content>h1:first-child:not([class]){color:rgba(0,0,0,.85);border-bottom:1px solid #dddddf;padding-bottom:8px;margin-top:0;padding-top:1rem;margin-bottom:1.25rem}
#toc{border-bottom:1px solid #e7e7e9;padding-bottom:.5em}
#toc>ul{margin-left:.125em}
#toc ul.sectlevel0>li>a{font-style:italic}
#toc ul.sectlevel0 ul.sectlevel1{margin:.5em 0}
#toc ul{font-family:"Open Sans","DejaVu Sans",sans-serif;list-style-type:none}
#toc li{line-height:1.3334;margin-top:.3334em}
#toc a{text-decoration:none}
#toc a:active{text-decoration:underline}
#toctitle{color:#7a2518;font-size:1.2em}
@media screen and (min-width:768px){#toctitle{font-size:1.375em}
body.toc2{padding-left:15em;padding-right:0}
body.toc2 #header>h1:nth-last-child(2){border-bottom:1px solid #dddddf;padding-bottom:8px}
#toc.toc2{margin-top:0!important;background:#f8f8f7;position:fixed;width:15em;left:0;top:0;border-right:1px solid #e7e7e9;border-top-width:0!important;border-bottom-width:0!important;z-index:1000;padding:1.25em 1em;height:100%;overflow:auto}
#toc.toc2 #toctitle{margin-top:0;margin-bottom:.8rem;font-size:1.2em}
#toc.toc2>ul{font-size:.9em;margin-bottom:0}
#toc.toc2 ul ul{margin-left:0;padding-left:1em}
#toc.toc2 ul.sectlevel0 ul.sectlevel1{padding-left:0;margin-top:.5em;margin-bottom:.5em}
body.toc2.toc-right{padding-left:0;padding-right:15em}
body.toc2.toc-right #toc.toc2{border-right-width:0;border-left:1px solid #e7e7e9;left:auto;right:0}}
@media screen and (min-width:1280px){body.toc2{padding-left:20em;padding-right:0}
#toc.toc2{width:20em}
#toc.toc2 #toctitle{font-size:1.375em}
#toc.toc2>ul{font-size:.95em}
#toc.toc2 ul ul{padding-left:1.25em}
body.toc2.toc-right{padding-left:0;padding-right:20em}}
#content #toc{border:1px solid #e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;border-radius:4px}
#content #toc>:first-child{margin-top:0}
#content #toc>:last-child{margin-bottom:0}
#footer{max-width:none;background:rgba(0,0,0,.8);padding:1.25em}
#footer-text{color:hsla(0,0%,100%,.8);line-height:1.44}
#content{margin-bottom:.625em}
.sect1{padding-bottom:.625em}
@media screen and (min-width:768px){#content{margin-bottom:1.25em}
.sect1{padding-bottom:1.25em}}
.sect1:last-child{padding-bottom:0}
.sect1+.sect1{border-top:1px solid #e7e7e9}
#content h1>a.anchor,h2>a.anchor,h3>a.anchor,#toctitle>a.anchor,.sidebarblock>.content>.title>a.anchor,h4>a.anchor,h5>a.anchor,h6>a.anchor{position:absolute;z-index:1001;width:1.5ex;margin-left:-1.5ex;display:block;text-decoration:none!important;visibility:hidden;text-align:center;font-weight:400}
#content h1>a.anchor::before,h2>a.anchor::before,h3>a.anchor::before,#toctitle>a.anchor::before,.sidebarblock>.content>.title>a.anchor::before,h4>a.anchor::before,h5>a.anchor::before,h6>a.anchor::before{content:"\00A7";font-size:.85em;display:block;padding-top:.1em}
#content h1:hover>a.anchor,#content h1>a.anchor:hover,h2:hover>a.anchor,h2>a.anchor:hover,h3:hover>a.anchor,#toctitle:hover>a.anchor,.sidebarblock>.content>.title:hover>a.anchor,h3>a.anchor:hover,#toctitle>a.anchor:hover,.sidebarblock>.content>.title>a.anchor:hover,h4:hover>a.anchor,h4>a.anchor:hover,h5:hover>a.anchor,h5>a.anchor:hover,h6:hover>a.anchor,h6>a.anchor:hover{visibility:visible}
#content h1>a.link,h2>a.link,h3>a.link,#toctitle>a.link,.sidebarblock>.content>.title>a.link,h4>a.link,h5>a.link,h6>a.link{color:#ba3925;text-decoration:none}
#content h1>a.link:hover,h2>a.link:hover,h3>a.link:hover,#toctitle>a.link:hover,.sidebarblock>.content>.title>a.link:hover,h4>a.link:hover,h5>a.link:hover,h6>a.link:hover{color:#a53221}
details,.audioblock,.imageblock,.literalblock,.listingblock,.stemblock,.videoblock{margin-bottom:1.25em}
details{margin-left:1.25rem}
details>summary{cursor:pointer;display:block;position:relative;line-height:1.6;margin-bottom:.625rem;outline:none;-webkit-tap-highlight-color:transparent}
details>summary::-webkit-details-marker{display:none}
details>summary::before{content:"";border:solid transparent;border-left:solid;border-width:.3em 0 .3em .5em;position:absolute;top:.5em;left:-1.25rem;transform:translateX(15%)}
details[open]>summary::before{border:solid transparent;border-top:solid;border-width:.5em .3em 0;transform:translateY(15%)}
details>summary::after{content:"";width:1.25rem;height:1em;position:absolute;top:.3em;left:-1.25rem}
.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{text-rendering:optimizeLegibility;text-align:left;font-family:"Noto Serif","DejaVu Serif",serif;font-size:1rem;font-style:italic}
table.tableblock.fit-content>caption.title{white-space:nowrap;width:0}
.paragraph.lead>p,#preamble>.sectionbody>[class=paragraph]:first-of-type p{font-size:1.21875em;line-height:1.6;color:rgba(0,0,0,.85)}
.admonitionblock>table{border-collapse:separate;border:0;background:none;width:100%}
.admonitionblock>table td.icon{text-align:center;width:80px}
.admonitionblock>table td.icon img{max-width:none}
.admonitionblock>table td.icon .title{font-weight:bold;font-family:"Open Sans","DejaVu Sans",sans-serif;text-transform:uppercase}
.admonitionblock>table td.content{padding-left:1.125em;padding-right:1.25em;border-left:1px solid #dddddf;color:rgba(0,0,0,.6);word-wrap:anywhere}
.admonitionblock>table td.content>:last-child>:last-child{margin-bottom:0}
.exampleblock>.content{border:1px solid #e6e6e6;margin-bottom:1.25em;padding:1.25em;background:#fff;border-radius:4px}
.sidebarblock{border:1px solid #dbdbd6;margin-bottom:1.25em;padding:1.25em;background:#f3f3f2;border-radius:4px}
.sidebarblock>.content>.title{color:#7a2518;margin-top:0;text-align:center}
.exampleblock>.content>:first-child,.sidebarblock>.content>:first-child{margin-top:0}
.exampleblock>.content>:last-child,.exampleblock>.content>:last-child>:last-child,.exampleblock>.content .olist>ol>li:last-child>:last-child,.exampleblock>.content .ulist>ul>li:last-child>:last-child,.exampleblock>.content .qlist>ol>li:last-child>:last-child,.sidebarblock>.content>:last-child,.sidebarblock>.content>:last-child>:last-child,.sidebarblock>.content .olist>ol>li:last-child>:last-child,.sidebarblock>.content .ulist>ul>li:last-child>:last-child,.sidebarblock>.content .qlist>ol>li:last-child>:last-child{margin-bottom:0}
.literalblock pre,.listingblock>.content>pre{border-radius:4px;overflow-x:auto;padding:1em;font-size:.8125em}
@media screen and (min-width:768px){.literalblock pre,.listingblock>.content>pre{font-size:.90625em}}
@media screen and (min-width:1280px){.literalblock pre,.listingblock>.content>pre{font-size:1em}}
.literalblock pre,.listingblock>.content>pre:not(.highlight),.listingblock>.content>pre[class=highlight],.listingblock>.content>pre[class^="highlight "]{background:#f7f7f8}
.literalblock.output pre{color:#f7f7f8;background:rgba(0,0,0,.9)}
.listingblock>.content{position:relative}
.listingblock code[data-lang]::before{display:none;content:attr(data-lang);position:absolute;font-size:.75em;top:.425rem;right:.5rem;line-height:1;text-transform:uppercase;color:inherit;opacity:.5}
.listingblock:hover code[data-lang]::before{display:block}
.listingblock.terminal pre .command::before{content:attr(data-prompt);padding-right:.5em;color:inherit;opacity:.5}
.listingblock.terminal pre .command:not([data-prompt])::before{content:"$"}
.listingblock pre.highlightjs{padding:0}
.listingblock pre.highlightjs>code{padding:1em;border-radius:4px}
.listingblock pre.prettyprint{border-width:0}
.prettyprint{background:#f7f7f8}
pre.prettyprint .linenums{line-height:1.45;margin-left:2em}
pre.prettyprint li{background:none;list-style-type:inherit;padding-left:0}
pre.prettyprint li code[data-lang]::before{opacity:1}
pre.prettyprint li:not(:first-child) code[data-lang]::before{display:none}
table.linenotable{border-collapse:separate;border:0;margin-bottom:0;background:none}
table.linenotable td[class]{color:inherit;vertical-align:top;padding:0;line-height:inherit;white-space:normal}
table.linenotable td.code{padding-left:.75em}
table.linenotable td.linenos,pre.pygments .linenos{border-right:1px solid;opacity:.35;padding-right:.5em;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}
pre.pygments span.linenos{display:inline-block;margin-right:.75em}
.quoteblock{margin:0 1em 1.25em 1.5em;display:table}
.quoteblock:not(.excerpt)>.title{margin-left:-1.5em;margin-bottom:.75em}
.quoteblock blockquote,.quoteblock p{color:rgba(0,0,0,.85);font-size:1.15rem;line-height:1.75;word-spacing:.1em;letter-spacing:0;font-style:italic;text-align:justify}
.quoteblock blockquote{margin:0;padding:0;border:0}
.quoteblock blockquote::before{content:"\201c";float:left;font-size:2.75em;font-weight:bold;line-height:.6em;margin-left:-.6em;color:#7a2518;text-shadow:0 1px 2px rgba(0,0,0,.1)}
.quoteblock blockquote>.paragraph:last-child p{margin-bottom:0}
.quoteblock .attribution{margin-top:.75em;margin-right:.5ex;text-align:right}
.verseblock{margin:0 1em 1.25em}
.verseblock pre{font-family:"Open Sans","DejaVu Sans",sans-serif;font-size:1.15rem;color:rgba(0,0,0,.85);font-weight:300;text-rendering:optimizeLegibility}
.verseblock pre strong{font-weight:400}
.verseblock .attribution{margin-top:1.25rem;margin-left:.5ex}
.quoteblock .attribution,.verseblock .attribution{font-size:.9375em;line-height:1.45;font-style:italic}
.quoteblock .attribution br,.verseblock .attribution br{display:none}
.quoteblock .attribution cite,.verseblock .attribution cite{display:block;letter-spacing:-.025em;color:rgba(0,0,0,.6)}
.quoteblock.abstract blockquote::before,.quoteblock.excerpt blockquote::before,.quoteblock .quoteblock blockquote::before{display:none}
.quoteblock.abstract blockquote,.quoteblock.abstract p,.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{line-height:1.6;word-spacing:0}
.quoteblock.abstract{margin:0 1em 1.25em;display:block}
.quoteblock.abstract>.title{margin:0 0 .375em;font-size:1.15em;text-align:center}
.quoteblock.excerpt>blockquote,.quoteblock .quoteblock{padding:0 0 .25em 1em;border-left:.25em solid #dddddf}
.quoteblock.excerpt,.quoteblock .quoteblock{margin-left:0}
.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{color:inherit;font-size:1.0625rem}
.quoteblock.excerpt .attribution,.quoteblock .quoteblock .attribution{color:inherit;font-size:.85rem;text-align:left;margin-right:0}
p.tableblock:last-child{margin-bottom:0}
td.tableblock>.content{margin-bottom:1.25em;word-wrap:anywhere}
td.tableblock>.content>:last-child{margin-bottom:-1.25em}
table.tableblock,th.tableblock,td.tableblock{border:0 solid #dedede}
table.grid-all>*>tr>*{border-width:1px}
table.grid-cols>*>tr>*{border-width:0 1px}
table.grid-rows>*>tr>*{border-width:1px 0}
table.frame-all{border-width:1px}
table.frame-ends{border-width:1px 0}
table.frame-sides{border-width:0 1px}
table.frame-none>colgroup+*>:first-child>*,table.frame-sides>colgroup+*>:first-child>*{border-top-width:0}
table.frame-none>:last-child>:last-child>*,table.frame-sides>:last-child>:last-child>*{border-bottom-width:0}
table.frame-none>*>tr>:first-child,table.frame-ends>*>tr>:first-child{border-left-width:0}
table.frame-none>*>tr>:last-child,table.frame-ends>*>tr>:last-child{border-right-width:0}
table.stripes-all>*>tr,table.stripes-odd>*>tr:nth-of-type(odd),table.stripes-even>*>tr:nth-of-type(even),table.stripes-hover>*>tr:hover{background:#f8f8f7}
th.halign-left,td.halign-left{text-align:left}
th.halign-right,td.halign-right{text-align:right}
th.halign-center,td.halign-center{text-align:center}
th.valign-top,td.valign-top{vertical-align:top}
th.valign-bottom,td.valign-bottom{vertical-align:bottom}
th.valign-middle,td.valign-middle{vertical-align:middle}
table thead th,table tfoot th{font-weight:bold}
tbody tr th{background:#f7f8f7}
tbody tr th,tbody tr th p,tfoot tr th,tfoot tr th p{color:rgba(0,0,0,.8);font-weight:bold}
p.tableblock>code:only-child{background:none;padding:0}
p.tableblock{font-size:1em}
ol{margin-left:1.75em}
ul li ol{margin-left:1.5em}
dl dd{margin-left:1.125em}
dl dd:last-child,dl dd:last-child>:last-child{margin-bottom:0}
li p,ul dd,ol dd,.olist .olist,.ulist .ulist,.ulist .olist,.olist .ulist{margin-bottom:.625em}
ul.checklist,ul.none,ol.none,ul.no-bullet,ol.no-bullet,ol.unnumbered,ul.unstyled,ol.unstyled{list-style-type:none}
ul.no-bullet,ol.no-bullet,ol.unnumbered{margin-left:.625em}
ul.unstyled,ol.unstyled{margin-left:0}
li>p:empty:only-child::before{content:"";display:inline-block}
ul.checklist>li>p:first-child{margin-left:-1em}
ul.checklist>li>p:first-child>.fa-square-o:first-child,ul.checklist>li>p:first-child>.fa-check-square-o:first-child{width:1.25em;font-size:.8em;position:relative;bottom:.125em}
ul.checklist>li>p:first-child>input[type=checkbox]:first-child{margin-right:.25em}
ul.inline{display:flex;flex-flow:row wrap;list-style:none;margin:0 0 .625em -1.25em}
ul.inline>li{margin-left:1.25em}
.unstyled dl dt{font-weight:400;font-style:normal}
ol.arabic{list-style-type:decimal}
ol.decimal{list-style-type:decimal-leading-zero}
ol.loweralpha{list-style-type:lower-alpha}
ol.upperalpha{list-style-type:upper-alpha}
ol.lowerroman{list-style-type:lower-roman}
ol.upperroman{list-style-type:upper-roman}
ol.lowergreek{list-style-type:lower-greek}
.hdlist>table,.colist>table{border:0;background:none}
.hdlist>table>tbody>tr,.colist>table>tbody>tr{background:none}
td.hdlist1,td.hdlist2{vertical-align:top;padding:0 .625em}
td.hdlist1{font-weight:bold;padding-bottom:1.25em}
td.hdlist2{word-wrap:anywhere}
.literalblock+.colist,.listingblock+.colist{margin-top:-.5em}
.colist td:not([class]):first-child{padding:.4em .75em 0;line-height:1;vertical-align:top}
.colist td:not([class]):first-child img{max-width:none}
.colist td:not([class]):last-child{padding:.25em 0}
.thumb,.th{line-height:0;display:inline-block;border:4px solid #fff;box-shadow:0 0 0 1px #ddd}
.imageblock.left{margin:.25em .625em 1.25em 0}
.imageblock.right{margin:.25em 0 1.25em .625em}
.imageblock>.title{margin-bottom:0}
.imageblock.thumb,.imageblock.th{border-width:6px}
.imageblock.thumb>.title,.imageblock.th>.title{padding:0 .125em}
.image.left,.image.right{margin-top:.25em;margin-bottom:.25em;display:inline-block;line-height:0}
.image.left{margin-right:.625em}
.image.right{margin-left:.625em}
a.image{text-decoration:none;display:inline-block}
a.image object{pointer-events:none}
sup.footnote,sup.footnoteref{font-size:.875em;position:static;vertical-align:super}
sup.footnote a,sup.footnoteref a{text-decoration:none}
sup.footnote a:active,sup.footnoteref a:active,#footnotes .footnote a:first-of-type:active{text-decoration:underline}
#footnotes{padding-top:.75em;padding-bottom:.75em;margin-bottom:.625em}
#footnotes hr{width:20%;min-width:6.25em;margin:-.25em 0 .75em;border-width:1px 0 0}
#footnotes .footnote{padding:0 .375em 0 .225em;line-height:1.3334;font-size:.875em;margin-left:1.2em;margin-bottom:.2em}
#footnotes .footnote a:first-of-type{font-weight:bold;text-decoration:none;margin-left:-1.05em}
#footnotes .footnote:last-of-type{margin-bottom:0}
#content #footnotes{margin-top:-.625em;margin-bottom:0;padding:.75em 0}
div.unbreakable{page-break-inside:avoid}
.big{font-size:larger}
.small{font-size:smaller}
.underline{text-decoration:underline}
.overline{text-decoration:overline}
.line-through{text-decoration:line-through}
.aqua{color:#00bfbf}
.aqua-background{background:#00fafa}
.black{color:#000}
.black-background{background:#000}
.blue{color:#0000bf}
.blue-background{background:#0000fa}
.fuchsia{color:#bf00bf}
.fuchsia-background{background:#fa00fa}
.gray{color:#606060}
.gray-background{background:#7d7d7d}
.green{color:#006000}
.green-background{background:#007d00}
.lime{color:#00bf00}
.lime-background{background:#00fa00}
.maroon{color:#600000}
.maroon-background{background:#7d0000}
.navy{color:#000060}
.navy-background{background:#00007d}
.olive{color:#606000}
.olive-background{background:#7d7d00}
.purple{color:#600060}
.purple-background{background:#7d007d}
.red{color:#bf0000}
.red-background{background:#fa0000}
.silver{color:#909090}
.silver-background{background:#bcbcbc}
.teal{color:#006060}
.teal-background{background:#007d7d}
.white{color:#bfbfbf}
.white-background{background:#fafafa}
.yellow{color:#bfbf00}
.yellow-background{background:#fafa00}
span.icon>.fa{cursor:default}
a span.icon>.fa{cursor:inherit}
.admonitionblock td.icon [class^="fa icon-"]{font-size:2.5em;text-shadow:1px 1px 2px rgba(0,0,0,.5);cursor:default}
.admonitionblock td.icon .icon-note::before{content:"\f05a";color:#19407c}
.admonitionblock td.icon .icon-tip::before{content:"\f0eb";text-shadow:1px 1px 2px rgba(155,155,0,.8);color:#111}
.admonitionblock td.icon .icon-warning::before{content:"\f071";color:#bf6900}
.admonitionblock td.icon .icon-caution::before{content:"\f06d";color:#bf3400}
.admonitionblock td.icon .icon-important::before{content:"\f06a";color:#bf0000}
.conum[data-value]{display:inline-block;color:#fff!important;background:rgba(0,0,0,.8);border-radius:50%;text-align:center;font-size:.75em;width:1.67em;height:1.67em;line-height:1.67em;font-family:"Open Sans","DejaVu Sans",sans-serif;font-style:normal;font-weight:bold}
.conum[data-value] *{color:#fff!important}
.conum[data-value]+b{display:none}
.conum[data-value]::after{content:attr(data-value)}
pre .conum[data-value]{position:relative;top:-.125em}
b.conum *{color:inherit!important}
.conum:not([data-value]):empty{display:none}
dt,th.tableblock,td.content,div.footnote{text-rendering:optimizeLegibility}
h1,h2,p,td.content,span.alt,summary{letter-spacing:-.01em}
p strong,td.content strong,div.footnote strong{letter-spacing:-.005em}
p,blockquote,dt,td.content,td.hdlist1,span.alt,summary{font-size:1.0625rem}
p{margin-bottom:1.25rem}
.sidebarblock p,.sidebarblock dt,.sidebarblock td.content,p.tableblock{font-size:1em}
.exampleblock>.content{background:#fffef7;border-color:#e0e0dc;box-shadow:0 1px 4px #e0e0dc}
.print-only{display:none!important}
@page{margin:1.25cm .75cm}
@media print{*{box-shadow:none!important;text-shadow:none!important}
html{font-size:80%}
a{color:inherit!important;text-decoration:underline!important}
a.bare,a[href^="#"],a[href^="mailto:"]{text-decoration:none!important}
a[href^="http:"]:not(.bare)::after,a[href^="https:"]:not(.bare)::after{content:"(" attr(href) ")";display:inline-block;font-size:.875em;padding-left:.25em}
abbr[title]{border-bottom:1px dotted}
abbr[title]::after{content:" (" attr(title) ")"}
pre,blockquote,tr,img,object,svg{page-break-inside:avoid}
thead{display:table-header-group}
svg{max-width:100%}
p,blockquote,dt,td.content{font-size:1em;orphans:3;widows:3}
h2,h3,#toctitle,.sidebarblock>.content>.title{page-break-after:avoid}
#header,#content,#footnotes,#footer{max-width:none}
#toc,.sidebarblock,.exampleblock>.content{background:none!important}
#toc{border-bottom:1px solid #dddddf!important;padding-bottom:0!important}
body.book #header{text-align:center}
body.book #header>h1:first-child{border:0!important;margin:2.5em 0 1em}
body.book #header .details{border:0!important;display:block;padding:0!important}
body.book #header .details span:first-child{margin-left:0!important}
body.book #header .details br{display:block}
body.book #header .details br+span::before{content:none!important}
body.book #toc{border:0!important;text-align:left!important;padding:0!important;margin:0!important}
body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-break-before:always}
.listingblock code[data-lang]::before{display:block}
#footer{padding:0 .9375em}
.hide-on-print{display:none!important}
.print-only{display:block!important}
.hide-for-print{display:none!important}
.show-for-print{display:inherit!important}}
@media amzn-kf8,print{#header>h1:first-child{margin-top:1.25rem}
.sect1{padding:0!important}
.sect1+.sect1{border:0}
#footer{background:none}
#footer-text{color:rgba(0,0,0,.6);font-size:.9em}}
@media amzn-kf8{#header,#content,#footnotes,#footer{padding:0}}
</style>
</head>
<body class="article">
<div id="header">
<h1>Git-send-pack internals</h1>
</div>
<div id="content">
<div class="sect1">
<h2 id="_overall_operation">Overall operation</h2>
<div class="sectionbody">
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Connects to the remote side and invokes git-receive-pack.</p>
</li>
<li>
<p>Learns what refs the remote has and what commit they point at.
Matches them to the refspecs we are pushing.</p>
</li>
<li>
<p>Checks if there are non-fast-forwards. Unlike fetch-pack,
the repository send-pack runs in is supposed to be a superset
of the recipient in fast-forward cases, so there is no need
for want/have exchanges, and fast-forward check can be done
locally. Tell the result to the other end.</p>
</li>
<li>
<p>Calls pack_objects() which generates a packfile and sends it
over to the other end.</p>
</li>
<li>
<p>If the remote side is new enough (v1.1.0 or later), wait for
the unpack and hook status from the other end.</p>
</li>
<li>
<p>Exit with appropriate error codes.</p>
</li>
</ol>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_pack_objects_pipeline">Pack_objects pipeline</h2>
<div class="sectionbody">
<div class="paragraph">
<p>This function gets one file descriptor (<code>fd</code>) which is either a
socket (over the network) or a pipe (local). What&#8217;s written to
this fd goes to git-receive-pack to be unpacked.</p>
</div>
<div class="literalblock">
<div class="content">
<pre>send-pack ---&gt; fd ---&gt; receive-pack</pre>
</div>
</div>
<div class="paragraph">
<p>The function pack_objects creates a pipe and then forks. The
forked child execs pack-objects with --revs to receive revision
parameters from its standard input. This process will write the
packfile to the other end.</p>
</div>
<div class="literalblock">
<div class="content">
<pre> send-pack
|
pack_objects() ---&gt; fd ---&gt; receive-pack
| ^ (pipe)
v |
(child)</pre>
</div>
</div>
<div class="paragraph">
<p>The child dup2&#8217;s to arrange its standard output to go back to
the other end, and read its standard input to come from the
pipe. After that it exec&#8217;s pack-objects. On the other hand,
the parent process, before starting to feed the child pipeline,
closes the reading side of the pipe and fd to receive-pack.</p>
</div>
<div class="literalblock">
<div class="content">
<pre> send-pack
|
pack_objects(parent)
|
v [0]
pack-objects [0] ---&gt; receive-pack</pre>
</div>
</div>
<div class="paragraph">
<p>[jc: the pipeline was much more complex and needed documentation before
I understood an earlier bug, but now it is trivial and straightforward.]</p>
</div>
</div>
</div>
</div>
<div id="footer">
<div id="footer-text">
Last updated 2025-03-17 07:09:38 UTC
</div>
</div>
</body>
</html>

View File

@@ -0,0 +1,60 @@
Shallow commits
===============
.Definition
*********************************************************
Shallow commits do have parents, but not in the shallow
repo, and therefore grafts are introduced pretending that
these commits have no parents.
*********************************************************
$GIT_DIR/shallow lists commit object names and tells Git to
pretend as if they are root commits (e.g. "git log" traversal
stops after showing them; "git fsck" does not complain saying
the commits listed on their "parent" lines do not exist).
Each line contains exactly one object name. When read, a commit_graft
will be constructed, which has nr_parent < 0 to make it easier
to discern from user provided grafts.
Note that the shallow feature could not be changed easily to
use replace refs: a commit containing a `mergetag` is not allowed
to be replaced, not even by a root commit. Such a commit can be
made shallow, though. Also, having a `shallow` file explicitly
listing all the commits made shallow makes it a *lot* easier to
do shallow-specific things such as to deepen the history.
Since fsck-objects relies on the library to read the objects,
it honours shallow commits automatically.
There are some unfinished ends of the whole shallow business:
- maybe we have to force non-thin packs when fetching into a
shallow repo (ATM they are forced non-thin).
- A special handling of a shallow upstream is needed. At some
stage, upload-pack has to check if it sends a shallow commit,
and it should send that information early (or fail, if the
client does not support shallow repositories). There is no
support at all for this in this patch series.
- Instead of locking $GIT_DIR/shallow at the start, just
the timestamp of it is noted, and when it comes to writing it,
a check is performed if the mtime is still the same, dying if
it is not.
- It is unclear how "push into/from a shallow repo" should behave.
- If you deepen a history, you'd want to get the tags of the
newly stored (but older!) commits. This does not work right now.
To make a shallow clone, you can call "git-clone --depth 20 repo".
The result contains only commit chains with a length of at most 20.
It also writes an appropriate $GIT_DIR/shallow.
You can deepen a shallow repository with "git-fetch --depth 20
repo branch", which will fetch branch from repo, but stop at depth
20, updating $GIT_DIR/shallow.
The special depth 2147483647 (or 0x7fffffff, the largest positive
number a signed 32-bit integer can contain) means infinite depth.

View File

@@ -0,0 +1,527 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="en">
<head>
<meta charset="UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=edge"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<meta name="generator" content="Asciidoctor 2.0.23"/>
<title>Shallow commits</title>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"/>
<style>
/*! Asciidoctor default stylesheet | MIT License | https://asciidoctor.org */
/* Uncomment the following line when using as a custom stylesheet */
/* @import "https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"; */
html{font-family:sans-serif;-webkit-text-size-adjust:100%}
a{background:none}
a:focus{outline:thin dotted}
a:active,a:hover{outline:0}
h1{font-size:2em;margin:.67em 0}
b,strong{font-weight:bold}
abbr{font-size:.9em}
abbr[title]{cursor:help;border-bottom:1px dotted #dddddf;text-decoration:none}
dfn{font-style:italic}
hr{height:0}
mark{background:#ff0;color:#000}
code,kbd,pre,samp{font-family:monospace;font-size:1em}
pre{white-space:pre-wrap}
q{quotes:"\201C" "\201D" "\2018" "\2019"}
small{font-size:80%}
sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline}
sup{top:-.5em}
sub{bottom:-.25em}
img{border:0}
svg:not(:root){overflow:hidden}
figure{margin:0}
audio,video{display:inline-block}
audio:not([controls]){display:none;height:0}
fieldset{border:1px solid silver;margin:0 2px;padding:.35em .625em .75em}
legend{border:0;padding:0}
button,input,select,textarea{font-family:inherit;font-size:100%;margin:0}
button,input{line-height:normal}
button,select{text-transform:none}
button,html input[type=button],input[type=reset],input[type=submit]{-webkit-appearance:button;cursor:pointer}
button[disabled],html input[disabled]{cursor:default}
input[type=checkbox],input[type=radio]{padding:0}
button::-moz-focus-inner,input::-moz-focus-inner{border:0;padding:0}
textarea{overflow:auto;vertical-align:top}
table{border-collapse:collapse;border-spacing:0}
*,::before,::after{box-sizing:border-box}
html,body{font-size:100%}
body{background:#fff;color:rgba(0,0,0,.8);padding:0;margin:0;font-family:"Noto Serif","DejaVu Serif",serif;line-height:1;position:relative;cursor:auto;-moz-tab-size:4;-o-tab-size:4;tab-size:4;word-wrap:anywhere;-moz-osx-font-smoothing:grayscale;-webkit-font-smoothing:antialiased}
a:hover{cursor:pointer}
img,object,embed{max-width:100%;height:auto}
object,embed{height:100%}
img{-ms-interpolation-mode:bicubic}
.left{float:left!important}
.right{float:right!important}
.text-left{text-align:left!important}
.text-right{text-align:right!important}
.text-center{text-align:center!important}
.text-justify{text-align:justify!important}
.hide{display:none}
img,object,svg{display:inline-block;vertical-align:middle}
textarea{height:auto;min-height:50px}
select{width:100%}
.subheader,.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{line-height:1.45;color:#7a2518;font-weight:400;margin-top:0;margin-bottom:.25em}
div,dl,dt,dd,ul,ol,li,h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6,pre,form,p,blockquote,th,td{margin:0;padding:0}
a{color:#2156a5;text-decoration:underline;line-height:inherit}
a:hover,a:focus{color:#1d4b8f}
a img{border:0}
p{line-height:1.6;margin-bottom:1.25em;text-rendering:optimizeLegibility}
p aside{font-size:.875em;line-height:1.35;font-style:italic}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{font-family:"Open Sans","DejaVu Sans",sans-serif;font-weight:300;font-style:normal;color:#ba3925;text-rendering:optimizeLegibility;margin-top:1em;margin-bottom:.5em;line-height:1.0125em}
h1 small,h2 small,h3 small,#toctitle small,.sidebarblock>.content>.title small,h4 small,h5 small,h6 small{font-size:60%;color:#e99b8f;line-height:0}
h1{font-size:2.125em}
h2{font-size:1.6875em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.375em}
h4,h5{font-size:1.125em}
h6{font-size:1em}
hr{border:solid #dddddf;border-width:1px 0 0;clear:both;margin:1.25em 0 1.1875em}
em,i{font-style:italic;line-height:inherit}
strong,b{font-weight:bold;line-height:inherit}
small{font-size:60%;line-height:inherit}
code{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:400;color:rgba(0,0,0,.9)}
ul,ol,dl{line-height:1.6;margin-bottom:1.25em;list-style-position:outside;font-family:inherit}
ul,ol{margin-left:1.5em}
ul li ul,ul li ol{margin-left:1.25em;margin-bottom:0}
ul.circle{list-style-type:circle}
ul.disc{list-style-type:disc}
ul.square{list-style-type:square}
ul.circle ul:not([class]),ul.disc ul:not([class]),ul.square ul:not([class]){list-style:inherit}
ol li ul,ol li ol{margin-left:1.25em;margin-bottom:0}
dl dt{margin-bottom:.3125em;font-weight:bold}
dl dd{margin-bottom:1.25em}
blockquote{margin:0 0 1.25em;padding:.5625em 1.25em 0 1.1875em;border-left:1px solid #ddd}
blockquote,blockquote p{line-height:1.6;color:rgba(0,0,0,.85)}
@media screen and (min-width:768px){h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2}
h1{font-size:2.75em}
h2{font-size:2.3125em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.6875em}
h4{font-size:1.4375em}}
table{background:#fff;margin-bottom:1.25em;border:1px solid #dedede;word-wrap:normal}
table thead,table tfoot{background:#f7f8f7}
table thead tr th,table thead tr td,table tfoot tr th,table tfoot tr td{padding:.5em .625em .625em;font-size:inherit;color:rgba(0,0,0,.8);text-align:left}
table tr th,table tr td{padding:.5625em .625em;font-size:inherit;color:rgba(0,0,0,.8)}
table tr.even,table tr.alt{background:#f8f8f7}
table thead tr th,table tfoot tr th,table tbody tr td,table tr td,table tfoot tr td{line-height:1.6}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2;word-spacing:-.05em}
h1 strong,h2 strong,h3 strong,#toctitle strong,.sidebarblock>.content>.title strong,h4 strong,h5 strong,h6 strong{font-weight:400}
.center{margin-left:auto;margin-right:auto}
.stretch{width:100%}
.clearfix::before,.clearfix::after,.float-group::before,.float-group::after{content:" ";display:table}
.clearfix::after,.float-group::after{clear:both}
:not(pre).nobreak{word-wrap:normal}
:not(pre).nowrap{white-space:nowrap}
:not(pre).pre-wrap{white-space:pre-wrap}
:not(pre):not([class^=L])>code{font-size:.9375em;font-style:normal!important;letter-spacing:0;padding:.1em .5ex;word-spacing:-.15em;background:#f7f7f8;border-radius:4px;line-height:1.45;text-rendering:optimizeSpeed}
pre{color:rgba(0,0,0,.9);font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;line-height:1.45;text-rendering:optimizeSpeed}
pre code,pre pre{color:inherit;font-size:inherit;line-height:inherit}
pre>code{display:block}
pre.nowrap,pre.nowrap pre{white-space:pre;word-wrap:normal}
em em{font-style:normal}
strong strong{font-weight:400}
.keyseq{color:rgba(51,51,51,.8)}
kbd{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;display:inline-block;color:rgba(0,0,0,.8);font-size:.65em;line-height:1.45;background:#f7f7f7;border:1px solid #ccc;border-radius:3px;box-shadow:0 1px 0 rgba(0,0,0,.2),inset 0 0 0 .1em #fff;margin:0 .15em;padding:.2em .5em;vertical-align:middle;position:relative;top:-.1em;white-space:nowrap}
.keyseq kbd:first-child{margin-left:0}
.keyseq kbd:last-child{margin-right:0}
.menuseq,.menuref{color:#000}
.menuseq b:not(.caret),.menuref{font-weight:inherit}
.menuseq{word-spacing:-.02em}
.menuseq b.caret{font-size:1.25em;line-height:.8}
.menuseq i.caret{font-weight:bold;text-align:center;width:.45em}
b.button::before,b.button::after{position:relative;top:-1px;font-weight:400}
b.button::before{content:"[";padding:0 3px 0 2px}
b.button::after{content:"]";padding:0 2px 0 3px}
p a>code:hover{color:rgba(0,0,0,.9)}
#header,#content,#footnotes,#footer{width:100%;margin:0 auto;max-width:62.5em;*zoom:1;position:relative;padding-left:.9375em;padding-right:.9375em}
#header::before,#header::after,#content::before,#content::after,#footnotes::before,#footnotes::after,#footer::before,#footer::after{content:" ";display:table}
#header::after,#content::after,#footnotes::after,#footer::after{clear:both}
#content{margin-top:1.25em}
#content::before{content:none}
#header>h1:first-child{color:rgba(0,0,0,.85);margin-top:2.25rem;margin-bottom:0}
#header>h1:first-child+#toc{margin-top:8px;border-top:1px solid #dddddf}
#header>h1:only-child{border-bottom:1px solid #dddddf;padding-bottom:8px}
#header .details{border-bottom:1px solid #dddddf;line-height:1.45;padding-top:.25em;padding-bottom:.25em;padding-left:.25em;color:rgba(0,0,0,.6);display:flex;flex-flow:row wrap}
#header .details span:first-child{margin-left:-.125em}
#header .details span.email a{color:rgba(0,0,0,.85)}
#header .details br{display:none}
#header .details br+span::before{content:"\00a0\2013\00a0"}
#header .details br+span.author::before{content:"\00a0\22c5\00a0";color:rgba(0,0,0,.85)}
#header .details br+span#revremark::before{content:"\00a0|\00a0"}
#header #revnumber{text-transform:capitalize}
#header #revnumber::after{content:"\00a0"}
#content>h1:first-child:not([class]){color:rgba(0,0,0,.85);border-bottom:1px solid #dddddf;padding-bottom:8px;margin-top:0;padding-top:1rem;margin-bottom:1.25rem}
#toc{border-bottom:1px solid #e7e7e9;padding-bottom:.5em}
#toc>ul{margin-left:.125em}
#toc ul.sectlevel0>li>a{font-style:italic}
#toc ul.sectlevel0 ul.sectlevel1{margin:.5em 0}
#toc ul{font-family:"Open Sans","DejaVu Sans",sans-serif;list-style-type:none}
#toc li{line-height:1.3334;margin-top:.3334em}
#toc a{text-decoration:none}
#toc a:active{text-decoration:underline}
#toctitle{color:#7a2518;font-size:1.2em}
@media screen and (min-width:768px){#toctitle{font-size:1.375em}
body.toc2{padding-left:15em;padding-right:0}
body.toc2 #header>h1:nth-last-child(2){border-bottom:1px solid #dddddf;padding-bottom:8px}
#toc.toc2{margin-top:0!important;background:#f8f8f7;position:fixed;width:15em;left:0;top:0;border-right:1px solid #e7e7e9;border-top-width:0!important;border-bottom-width:0!important;z-index:1000;padding:1.25em 1em;height:100%;overflow:auto}
#toc.toc2 #toctitle{margin-top:0;margin-bottom:.8rem;font-size:1.2em}
#toc.toc2>ul{font-size:.9em;margin-bottom:0}
#toc.toc2 ul ul{margin-left:0;padding-left:1em}
#toc.toc2 ul.sectlevel0 ul.sectlevel1{padding-left:0;margin-top:.5em;margin-bottom:.5em}
body.toc2.toc-right{padding-left:0;padding-right:15em}
body.toc2.toc-right #toc.toc2{border-right-width:0;border-left:1px solid #e7e7e9;left:auto;right:0}}
@media screen and (min-width:1280px){body.toc2{padding-left:20em;padding-right:0}
#toc.toc2{width:20em}
#toc.toc2 #toctitle{font-size:1.375em}
#toc.toc2>ul{font-size:.95em}
#toc.toc2 ul ul{padding-left:1.25em}
body.toc2.toc-right{padding-left:0;padding-right:20em}}
#content #toc{border:1px solid #e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;border-radius:4px}
#content #toc>:first-child{margin-top:0}
#content #toc>:last-child{margin-bottom:0}
#footer{max-width:none;background:rgba(0,0,0,.8);padding:1.25em}
#footer-text{color:hsla(0,0%,100%,.8);line-height:1.44}
#content{margin-bottom:.625em}
.sect1{padding-bottom:.625em}
@media screen and (min-width:768px){#content{margin-bottom:1.25em}
.sect1{padding-bottom:1.25em}}
.sect1:last-child{padding-bottom:0}
.sect1+.sect1{border-top:1px solid #e7e7e9}
#content h1>a.anchor,h2>a.anchor,h3>a.anchor,#toctitle>a.anchor,.sidebarblock>.content>.title>a.anchor,h4>a.anchor,h5>a.anchor,h6>a.anchor{position:absolute;z-index:1001;width:1.5ex;margin-left:-1.5ex;display:block;text-decoration:none!important;visibility:hidden;text-align:center;font-weight:400}
#content h1>a.anchor::before,h2>a.anchor::before,h3>a.anchor::before,#toctitle>a.anchor::before,.sidebarblock>.content>.title>a.anchor::before,h4>a.anchor::before,h5>a.anchor::before,h6>a.anchor::before{content:"\00A7";font-size:.85em;display:block;padding-top:.1em}
#content h1:hover>a.anchor,#content h1>a.anchor:hover,h2:hover>a.anchor,h2>a.anchor:hover,h3:hover>a.anchor,#toctitle:hover>a.anchor,.sidebarblock>.content>.title:hover>a.anchor,h3>a.anchor:hover,#toctitle>a.anchor:hover,.sidebarblock>.content>.title>a.anchor:hover,h4:hover>a.anchor,h4>a.anchor:hover,h5:hover>a.anchor,h5>a.anchor:hover,h6:hover>a.anchor,h6>a.anchor:hover{visibility:visible}
#content h1>a.link,h2>a.link,h3>a.link,#toctitle>a.link,.sidebarblock>.content>.title>a.link,h4>a.link,h5>a.link,h6>a.link{color:#ba3925;text-decoration:none}
#content h1>a.link:hover,h2>a.link:hover,h3>a.link:hover,#toctitle>a.link:hover,.sidebarblock>.content>.title>a.link:hover,h4>a.link:hover,h5>a.link:hover,h6>a.link:hover{color:#a53221}
details,.audioblock,.imageblock,.literalblock,.listingblock,.stemblock,.videoblock{margin-bottom:1.25em}
details{margin-left:1.25rem}
details>summary{cursor:pointer;display:block;position:relative;line-height:1.6;margin-bottom:.625rem;outline:none;-webkit-tap-highlight-color:transparent}
details>summary::-webkit-details-marker{display:none}
details>summary::before{content:"";border:solid transparent;border-left:solid;border-width:.3em 0 .3em .5em;position:absolute;top:.5em;left:-1.25rem;transform:translateX(15%)}
details[open]>summary::before{border:solid transparent;border-top:solid;border-width:.5em .3em 0;transform:translateY(15%)}
details>summary::after{content:"";width:1.25rem;height:1em;position:absolute;top:.3em;left:-1.25rem}
.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{text-rendering:optimizeLegibility;text-align:left;font-family:"Noto Serif","DejaVu Serif",serif;font-size:1rem;font-style:italic}
table.tableblock.fit-content>caption.title{white-space:nowrap;width:0}
.paragraph.lead>p,#preamble>.sectionbody>[class=paragraph]:first-of-type p{font-size:1.21875em;line-height:1.6;color:rgba(0,0,0,.85)}
.admonitionblock>table{border-collapse:separate;border:0;background:none;width:100%}
.admonitionblock>table td.icon{text-align:center;width:80px}
.admonitionblock>table td.icon img{max-width:none}
.admonitionblock>table td.icon .title{font-weight:bold;font-family:"Open Sans","DejaVu Sans",sans-serif;text-transform:uppercase}
.admonitionblock>table td.content{padding-left:1.125em;padding-right:1.25em;border-left:1px solid #dddddf;color:rgba(0,0,0,.6);word-wrap:anywhere}
.admonitionblock>table td.content>:last-child>:last-child{margin-bottom:0}
.exampleblock>.content{border:1px solid #e6e6e6;margin-bottom:1.25em;padding:1.25em;background:#fff;border-radius:4px}
.sidebarblock{border:1px solid #dbdbd6;margin-bottom:1.25em;padding:1.25em;background:#f3f3f2;border-radius:4px}
.sidebarblock>.content>.title{color:#7a2518;margin-top:0;text-align:center}
.exampleblock>.content>:first-child,.sidebarblock>.content>:first-child{margin-top:0}
.exampleblock>.content>:last-child,.exampleblock>.content>:last-child>:last-child,.exampleblock>.content .olist>ol>li:last-child>:last-child,.exampleblock>.content .ulist>ul>li:last-child>:last-child,.exampleblock>.content .qlist>ol>li:last-child>:last-child,.sidebarblock>.content>:last-child,.sidebarblock>.content>:last-child>:last-child,.sidebarblock>.content .olist>ol>li:last-child>:last-child,.sidebarblock>.content .ulist>ul>li:last-child>:last-child,.sidebarblock>.content .qlist>ol>li:last-child>:last-child{margin-bottom:0}
.literalblock pre,.listingblock>.content>pre{border-radius:4px;overflow-x:auto;padding:1em;font-size:.8125em}
@media screen and (min-width:768px){.literalblock pre,.listingblock>.content>pre{font-size:.90625em}}
@media screen and (min-width:1280px){.literalblock pre,.listingblock>.content>pre{font-size:1em}}
.literalblock pre,.listingblock>.content>pre:not(.highlight),.listingblock>.content>pre[class=highlight],.listingblock>.content>pre[class^="highlight "]{background:#f7f7f8}
.literalblock.output pre{color:#f7f7f8;background:rgba(0,0,0,.9)}
.listingblock>.content{position:relative}
.listingblock code[data-lang]::before{display:none;content:attr(data-lang);position:absolute;font-size:.75em;top:.425rem;right:.5rem;line-height:1;text-transform:uppercase;color:inherit;opacity:.5}
.listingblock:hover code[data-lang]::before{display:block}
.listingblock.terminal pre .command::before{content:attr(data-prompt);padding-right:.5em;color:inherit;opacity:.5}
.listingblock.terminal pre .command:not([data-prompt])::before{content:"$"}
.listingblock pre.highlightjs{padding:0}
.listingblock pre.highlightjs>code{padding:1em;border-radius:4px}
.listingblock pre.prettyprint{border-width:0}
.prettyprint{background:#f7f7f8}
pre.prettyprint .linenums{line-height:1.45;margin-left:2em}
pre.prettyprint li{background:none;list-style-type:inherit;padding-left:0}
pre.prettyprint li code[data-lang]::before{opacity:1}
pre.prettyprint li:not(:first-child) code[data-lang]::before{display:none}
table.linenotable{border-collapse:separate;border:0;margin-bottom:0;background:none}
table.linenotable td[class]{color:inherit;vertical-align:top;padding:0;line-height:inherit;white-space:normal}
table.linenotable td.code{padding-left:.75em}
table.linenotable td.linenos,pre.pygments .linenos{border-right:1px solid;opacity:.35;padding-right:.5em;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}
pre.pygments span.linenos{display:inline-block;margin-right:.75em}
.quoteblock{margin:0 1em 1.25em 1.5em;display:table}
.quoteblock:not(.excerpt)>.title{margin-left:-1.5em;margin-bottom:.75em}
.quoteblock blockquote,.quoteblock p{color:rgba(0,0,0,.85);font-size:1.15rem;line-height:1.75;word-spacing:.1em;letter-spacing:0;font-style:italic;text-align:justify}
.quoteblock blockquote{margin:0;padding:0;border:0}
.quoteblock blockquote::before{content:"\201c";float:left;font-size:2.75em;font-weight:bold;line-height:.6em;margin-left:-.6em;color:#7a2518;text-shadow:0 1px 2px rgba(0,0,0,.1)}
.quoteblock blockquote>.paragraph:last-child p{margin-bottom:0}
.quoteblock .attribution{margin-top:.75em;margin-right:.5ex;text-align:right}
.verseblock{margin:0 1em 1.25em}
.verseblock pre{font-family:"Open Sans","DejaVu Sans",sans-serif;font-size:1.15rem;color:rgba(0,0,0,.85);font-weight:300;text-rendering:optimizeLegibility}
.verseblock pre strong{font-weight:400}
.verseblock .attribution{margin-top:1.25rem;margin-left:.5ex}
.quoteblock .attribution,.verseblock .attribution{font-size:.9375em;line-height:1.45;font-style:italic}
.quoteblock .attribution br,.verseblock .attribution br{display:none}
.quoteblock .attribution cite,.verseblock .attribution cite{display:block;letter-spacing:-.025em;color:rgba(0,0,0,.6)}
.quoteblock.abstract blockquote::before,.quoteblock.excerpt blockquote::before,.quoteblock .quoteblock blockquote::before{display:none}
.quoteblock.abstract blockquote,.quoteblock.abstract p,.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{line-height:1.6;word-spacing:0}
.quoteblock.abstract{margin:0 1em 1.25em;display:block}
.quoteblock.abstract>.title{margin:0 0 .375em;font-size:1.15em;text-align:center}
.quoteblock.excerpt>blockquote,.quoteblock .quoteblock{padding:0 0 .25em 1em;border-left:.25em solid #dddddf}
.quoteblock.excerpt,.quoteblock .quoteblock{margin-left:0}
.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{color:inherit;font-size:1.0625rem}
.quoteblock.excerpt .attribution,.quoteblock .quoteblock .attribution{color:inherit;font-size:.85rem;text-align:left;margin-right:0}
p.tableblock:last-child{margin-bottom:0}
td.tableblock>.content{margin-bottom:1.25em;word-wrap:anywhere}
td.tableblock>.content>:last-child{margin-bottom:-1.25em}
table.tableblock,th.tableblock,td.tableblock{border:0 solid #dedede}
table.grid-all>*>tr>*{border-width:1px}
table.grid-cols>*>tr>*{border-width:0 1px}
table.grid-rows>*>tr>*{border-width:1px 0}
table.frame-all{border-width:1px}
table.frame-ends{border-width:1px 0}
table.frame-sides{border-width:0 1px}
table.frame-none>colgroup+*>:first-child>*,table.frame-sides>colgroup+*>:first-child>*{border-top-width:0}
table.frame-none>:last-child>:last-child>*,table.frame-sides>:last-child>:last-child>*{border-bottom-width:0}
table.frame-none>*>tr>:first-child,table.frame-ends>*>tr>:first-child{border-left-width:0}
table.frame-none>*>tr>:last-child,table.frame-ends>*>tr>:last-child{border-right-width:0}
table.stripes-all>*>tr,table.stripes-odd>*>tr:nth-of-type(odd),table.stripes-even>*>tr:nth-of-type(even),table.stripes-hover>*>tr:hover{background:#f8f8f7}
th.halign-left,td.halign-left{text-align:left}
th.halign-right,td.halign-right{text-align:right}
th.halign-center,td.halign-center{text-align:center}
th.valign-top,td.valign-top{vertical-align:top}
th.valign-bottom,td.valign-bottom{vertical-align:bottom}
th.valign-middle,td.valign-middle{vertical-align:middle}
table thead th,table tfoot th{font-weight:bold}
tbody tr th{background:#f7f8f7}
tbody tr th,tbody tr th p,tfoot tr th,tfoot tr th p{color:rgba(0,0,0,.8);font-weight:bold}
p.tableblock>code:only-child{background:none;padding:0}
p.tableblock{font-size:1em}
ol{margin-left:1.75em}
ul li ol{margin-left:1.5em}
dl dd{margin-left:1.125em}
dl dd:last-child,dl dd:last-child>:last-child{margin-bottom:0}
li p,ul dd,ol dd,.olist .olist,.ulist .ulist,.ulist .olist,.olist .ulist{margin-bottom:.625em}
ul.checklist,ul.none,ol.none,ul.no-bullet,ol.no-bullet,ol.unnumbered,ul.unstyled,ol.unstyled{list-style-type:none}
ul.no-bullet,ol.no-bullet,ol.unnumbered{margin-left:.625em}
ul.unstyled,ol.unstyled{margin-left:0}
li>p:empty:only-child::before{content:"";display:inline-block}
ul.checklist>li>p:first-child{margin-left:-1em}
ul.checklist>li>p:first-child>.fa-square-o:first-child,ul.checklist>li>p:first-child>.fa-check-square-o:first-child{width:1.25em;font-size:.8em;position:relative;bottom:.125em}
ul.checklist>li>p:first-child>input[type=checkbox]:first-child{margin-right:.25em}
ul.inline{display:flex;flex-flow:row wrap;list-style:none;margin:0 0 .625em -1.25em}
ul.inline>li{margin-left:1.25em}
.unstyled dl dt{font-weight:400;font-style:normal}
ol.arabic{list-style-type:decimal}
ol.decimal{list-style-type:decimal-leading-zero}
ol.loweralpha{list-style-type:lower-alpha}
ol.upperalpha{list-style-type:upper-alpha}
ol.lowerroman{list-style-type:lower-roman}
ol.upperroman{list-style-type:upper-roman}
ol.lowergreek{list-style-type:lower-greek}
.hdlist>table,.colist>table{border:0;background:none}
.hdlist>table>tbody>tr,.colist>table>tbody>tr{background:none}
td.hdlist1,td.hdlist2{vertical-align:top;padding:0 .625em}
td.hdlist1{font-weight:bold;padding-bottom:1.25em}
td.hdlist2{word-wrap:anywhere}
.literalblock+.colist,.listingblock+.colist{margin-top:-.5em}
.colist td:not([class]):first-child{padding:.4em .75em 0;line-height:1;vertical-align:top}
.colist td:not([class]):first-child img{max-width:none}
.colist td:not([class]):last-child{padding:.25em 0}
.thumb,.th{line-height:0;display:inline-block;border:4px solid #fff;box-shadow:0 0 0 1px #ddd}
.imageblock.left{margin:.25em .625em 1.25em 0}
.imageblock.right{margin:.25em 0 1.25em .625em}
.imageblock>.title{margin-bottom:0}
.imageblock.thumb,.imageblock.th{border-width:6px}
.imageblock.thumb>.title,.imageblock.th>.title{padding:0 .125em}
.image.left,.image.right{margin-top:.25em;margin-bottom:.25em;display:inline-block;line-height:0}
.image.left{margin-right:.625em}
.image.right{margin-left:.625em}
a.image{text-decoration:none;display:inline-block}
a.image object{pointer-events:none}
sup.footnote,sup.footnoteref{font-size:.875em;position:static;vertical-align:super}
sup.footnote a,sup.footnoteref a{text-decoration:none}
sup.footnote a:active,sup.footnoteref a:active,#footnotes .footnote a:first-of-type:active{text-decoration:underline}
#footnotes{padding-top:.75em;padding-bottom:.75em;margin-bottom:.625em}
#footnotes hr{width:20%;min-width:6.25em;margin:-.25em 0 .75em;border-width:1px 0 0}
#footnotes .footnote{padding:0 .375em 0 .225em;line-height:1.3334;font-size:.875em;margin-left:1.2em;margin-bottom:.2em}
#footnotes .footnote a:first-of-type{font-weight:bold;text-decoration:none;margin-left:-1.05em}
#footnotes .footnote:last-of-type{margin-bottom:0}
#content #footnotes{margin-top:-.625em;margin-bottom:0;padding:.75em 0}
div.unbreakable{page-break-inside:avoid}
.big{font-size:larger}
.small{font-size:smaller}
.underline{text-decoration:underline}
.overline{text-decoration:overline}
.line-through{text-decoration:line-through}
.aqua{color:#00bfbf}
.aqua-background{background:#00fafa}
.black{color:#000}
.black-background{background:#000}
.blue{color:#0000bf}
.blue-background{background:#0000fa}
.fuchsia{color:#bf00bf}
.fuchsia-background{background:#fa00fa}
.gray{color:#606060}
.gray-background{background:#7d7d7d}
.green{color:#006000}
.green-background{background:#007d00}
.lime{color:#00bf00}
.lime-background{background:#00fa00}
.maroon{color:#600000}
.maroon-background{background:#7d0000}
.navy{color:#000060}
.navy-background{background:#00007d}
.olive{color:#606000}
.olive-background{background:#7d7d00}
.purple{color:#600060}
.purple-background{background:#7d007d}
.red{color:#bf0000}
.red-background{background:#fa0000}
.silver{color:#909090}
.silver-background{background:#bcbcbc}
.teal{color:#006060}
.teal-background{background:#007d7d}
.white{color:#bfbfbf}
.white-background{background:#fafafa}
.yellow{color:#bfbf00}
.yellow-background{background:#fafa00}
span.icon>.fa{cursor:default}
a span.icon>.fa{cursor:inherit}
.admonitionblock td.icon [class^="fa icon-"]{font-size:2.5em;text-shadow:1px 1px 2px rgba(0,0,0,.5);cursor:default}
.admonitionblock td.icon .icon-note::before{content:"\f05a";color:#19407c}
.admonitionblock td.icon .icon-tip::before{content:"\f0eb";text-shadow:1px 1px 2px rgba(155,155,0,.8);color:#111}
.admonitionblock td.icon .icon-warning::before{content:"\f071";color:#bf6900}
.admonitionblock td.icon .icon-caution::before{content:"\f06d";color:#bf3400}
.admonitionblock td.icon .icon-important::before{content:"\f06a";color:#bf0000}
.conum[data-value]{display:inline-block;color:#fff!important;background:rgba(0,0,0,.8);border-radius:50%;text-align:center;font-size:.75em;width:1.67em;height:1.67em;line-height:1.67em;font-family:"Open Sans","DejaVu Sans",sans-serif;font-style:normal;font-weight:bold}
.conum[data-value] *{color:#fff!important}
.conum[data-value]+b{display:none}
.conum[data-value]::after{content:attr(data-value)}
pre .conum[data-value]{position:relative;top:-.125em}
b.conum *{color:inherit!important}
.conum:not([data-value]):empty{display:none}
dt,th.tableblock,td.content,div.footnote{text-rendering:optimizeLegibility}
h1,h2,p,td.content,span.alt,summary{letter-spacing:-.01em}
p strong,td.content strong,div.footnote strong{letter-spacing:-.005em}
p,blockquote,dt,td.content,td.hdlist1,span.alt,summary{font-size:1.0625rem}
p{margin-bottom:1.25rem}
.sidebarblock p,.sidebarblock dt,.sidebarblock td.content,p.tableblock{font-size:1em}
.exampleblock>.content{background:#fffef7;border-color:#e0e0dc;box-shadow:0 1px 4px #e0e0dc}
.print-only{display:none!important}
@page{margin:1.25cm .75cm}
@media print{*{box-shadow:none!important;text-shadow:none!important}
html{font-size:80%}
a{color:inherit!important;text-decoration:underline!important}
a.bare,a[href^="#"],a[href^="mailto:"]{text-decoration:none!important}
a[href^="http:"]:not(.bare)::after,a[href^="https:"]:not(.bare)::after{content:"(" attr(href) ")";display:inline-block;font-size:.875em;padding-left:.25em}
abbr[title]{border-bottom:1px dotted}
abbr[title]::after{content:" (" attr(title) ")"}
pre,blockquote,tr,img,object,svg{page-break-inside:avoid}
thead{display:table-header-group}
svg{max-width:100%}
p,blockquote,dt,td.content{font-size:1em;orphans:3;widows:3}
h2,h3,#toctitle,.sidebarblock>.content>.title{page-break-after:avoid}
#header,#content,#footnotes,#footer{max-width:none}
#toc,.sidebarblock,.exampleblock>.content{background:none!important}
#toc{border-bottom:1px solid #dddddf!important;padding-bottom:0!important}
body.book #header{text-align:center}
body.book #header>h1:first-child{border:0!important;margin:2.5em 0 1em}
body.book #header .details{border:0!important;display:block;padding:0!important}
body.book #header .details span:first-child{margin-left:0!important}
body.book #header .details br{display:block}
body.book #header .details br+span::before{content:none!important}
body.book #toc{border:0!important;text-align:left!important;padding:0!important;margin:0!important}
body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-break-before:always}
.listingblock code[data-lang]::before{display:block}
#footer{padding:0 .9375em}
.hide-on-print{display:none!important}
.print-only{display:block!important}
.hide-for-print{display:none!important}
.show-for-print{display:inherit!important}}
@media amzn-kf8,print{#header>h1:first-child{margin-top:1.25rem}
.sect1{padding:0!important}
.sect1+.sect1{border:0}
#footer{background:none}
#footer-text{color:rgba(0,0,0,.6);font-size:.9em}}
@media amzn-kf8{#header,#content,#footnotes,#footer{padding:0}}
</style>
</head>
<body class="article">
<div id="header">
<h1>Shallow commits</h1>
</div>
<div id="content">
<div class="sidebarblock">
<div class="content">
<div class="title">Definition</div>
<div class="paragraph">
<p>Shallow commits do have parents, but not in the shallow
repo, and therefore grafts are introduced pretending that
these commits have no parents.</p>
</div>
</div>
</div>
<div class="paragraph">
<p>$GIT_DIR/shallow lists commit object names and tells Git to
pretend as if they are root commits (e.g. "git log" traversal
stops after showing them; "git fsck" does not complain saying
the commits listed on their "parent" lines do not exist).</p>
</div>
<div class="paragraph">
<p>Each line contains exactly one object name. When read, a commit_graft
will be constructed, which has nr_parent &lt; 0 to make it easier
to discern from user provided grafts.</p>
</div>
<div class="paragraph">
<p>Note that the shallow feature could not be changed easily to
use replace refs: a commit containing a <code>mergetag</code> is not allowed
to be replaced, not even by a root commit. Such a commit can be
made shallow, though. Also, having a <code>shallow</code> file explicitly
listing all the commits made shallow makes it a <strong>lot</strong> easier to
do shallow-specific things such as to deepen the history.</p>
</div>
<div class="paragraph">
<p>Since fsck-objects relies on the library to read the objects,
it honours shallow commits automatically.</p>
</div>
<div class="paragraph">
<p>There are some unfinished ends of the whole shallow business:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>maybe we have to force non-thin packs when fetching into a
shallow repo (ATM they are forced non-thin).</p>
</li>
<li>
<p>A special handling of a shallow upstream is needed. At some
stage, upload-pack has to check if it sends a shallow commit,
and it should send that information early (or fail, if the
client does not support shallow repositories). There is no
support at all for this in this patch series.</p>
</li>
<li>
<p>Instead of locking $GIT_DIR/shallow at the start, just
the timestamp of it is noted, and when it comes to writing it,
a check is performed if the mtime is still the same, dying if
it is not.</p>
</li>
<li>
<p>It is unclear how "push into/from a shallow repo" should behave.</p>
</li>
<li>
<p>If you deepen a history, you&#8217;d want to get the tags of the
newly stored (but older!) commits. This does not work right now.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>To make a shallow clone, you can call "git-clone --depth 20 repo".
The result contains only commit chains with a length of at most 20.
It also writes an appropriate $GIT_DIR/shallow.</p>
</div>
<div class="paragraph">
<p>You can deepen a shallow repository with "git-fetch --depth 20
repo branch", which will fetch branch from repo, but stop at depth
20, updating $GIT_DIR/shallow.</p>
</div>
<div class="paragraph">
<p>The special depth 2147483647 (or 0x7fffffff, the largest positive
number a signed 32-bit integer can contain) means infinite depth.</p>
</div>
</div>
<div id="footer">
<div id="footer-text">
Last updated 2025-03-17 07:09:38 UTC
</div>
</div>
</body>
</html>

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,208 @@
Git Sparse-Index Design Document
================================
The sparse-checkout feature allows users to focus a working directory on
a subset of the files at HEAD. The cone mode patterns, enabled by
`core.sparseCheckoutCone`, allow for very fast pattern matching to
discover which files at HEAD belong in the sparse-checkout cone.
Three important scale dimensions for a Git working directory are:
* `HEAD`: How many files are present at `HEAD`?
* Populated: How many files are within the sparse-checkout cone.
* Modified: How many files has the user modified in the working directory?
We will use big-O notation -- O(X) -- to denote how expensive certain
operations are in terms of these dimensions.
These dimensions are ordered by their magnitude: users (typically) modify
fewer files than are populated, and we can only populate files at `HEAD`.
Problems occur if there is an extreme imbalance in these dimensions. For
example, if `HEAD` contains millions of paths but the populated set has
only tens of thousands, then commands like `git status` and `git add` can
be dominated by operations that require O(`HEAD`) operations instead of
O(Populated). Primarily, the cost is in parsing and rewriting the index,
which is filled primarily with files at `HEAD` that are marked with the
`SKIP_WORKTREE` bit.
The sparse-index intends to take these commands that read and modify the
index from O(`HEAD`) to O(Populated). To do this, we need to modify the
index format in a significant way: add "sparse directory" entries.
With cone mode patterns, it is possible to detect when an entire
directory will have its contents outside of the sparse-checkout definition.
Instead of listing all of the files it contains as individual entries, a
sparse-index contains an entry with the directory name, referencing the
object ID of the tree at `HEAD` and marked with the `SKIP_WORKTREE` bit.
If we need to discover the details for paths within that directory, we
can parse trees to find that list.
At time of writing, sparse-directory entries violate expectations about the
index format and its in-memory data structure. There are many consumers in
the codebase that expect to iterate through all of the index entries and
see only files. In fact, these loops expect to see a reference to every
staged file. One way to handle this is to parse trees to replace a
sparse-directory entry with all of the files within that tree as the index
is loaded. However, parsing trees is slower than parsing the index format,
so that is a slower operation than if we left the index alone. The plan is
to make all of these integrations "sparse aware" so this expansion through
tree parsing is unnecessary and they use fewer resources than when using a
full index.
The implementation plan below follows four phases to slowly integrate with
the sparse-index. The intention is to incrementally update Git commands to
interact safely with the sparse-index without significant slowdowns. This
may not always be possible, but the hope is that the primary commands that
users need in their daily work are dramatically improved.
Phase I: Format and initial speedups
------------------------------------
During this phase, Git learns to enable the sparse-index and safely parse
one. Protections are put in place so that every consumer of the in-memory
data structure can operate with its current assumption of every file at
`HEAD`.
At first, every index parse will call a helper method,
`ensure_full_index()`, which scans the index for sparse-directory entries
(pointing to trees) and replaces them with the full list of paths (with
blob contents) by parsing tree objects. This will be slower in all cases.
The only noticeable change in behavior will be that the serialized index
file contains sparse-directory entries.
To start, we use a new required index extension, `sdir`, to allow
inserting sparse-directory entries into indexes with file format
versions 2, 3, and 4. This prevents Git versions that do not understand
the sparse-index from operating on one, while allowing tools that do not
understand the sparse-index to operate on repositories as long as they do
not interact with the index. A new format, index v5, will be introduced
that includes sparse-directory entries by default. It might also
introduce other features that have been considered for improving the
index, as well.
Next, consumers of the index will be guarded against operating on a
sparse-index by inserting calls to `ensure_full_index()` or
`expand_index_to_path()`. If a specific path is requested, then those will
be protected from within the `index_file_exists()` and `index_name_pos()`
API calls: they will call `ensure_full_index()` if necessary. The
intention here is to preserve existing behavior when interacting with a
sparse-checkout. We don't want a change to happen by accident, without
tests. Many of these locations may not need any change before removing the
guards, but we should not do so without tests to ensure the expected
behavior happens.
It may be desirable to _change_ the behavior of some commands in the
presence of a sparse index or more generally in any sparse-checkout
scenario. In such cases, these should be carefully communicated and
tested. No such behavior changes are intended during this phase.
During a scan of the codebase, not every iteration of the cache entries
needs an `ensure_full_index()` check. The basic reasons include:
1. The loop is scanning for entries with non-zero stage. These entries
are not collapsed into a sparse-directory entry.
2. The loop is scanning for submodules. These entries are not collapsed
into a sparse-directory entry.
3. The loop is part of the index API, especially around reading or
writing the format.
4. The loop is checking for correct order of cache entries and that is
correct if and only if the sparse-directory entries are in the correct
location.
5. The loop ignores entries with the `SKIP_WORKTREE` bit set, or is
otherwise already aware of sparse directory entries.
6. The sparse-index is disabled at this point when using the split-index
feature, so no effort is made to protect the split-index API.
Even after inserting these guards, we will keep expanding sparse-indexes
for most Git commands using the `command_requires_full_index` repository
setting. This setting will be on by default and disabled one builtin at a
time until we have sufficient confidence that all of the index operations
are properly guarded.
To complete this phase, the commands `git status` and `git add` will be
integrated with the sparse-index so that they operate with O(Populated)
performance. They will be carefully tested for operations within and
outside the sparse-checkout definition.
Phase II: Careful integrations
------------------------------
This phase focuses on ensuring that all index extensions and APIs work
well with a sparse-index. This requires significant increases to our test
coverage, especially for operations that interact with the working
directory outside of the sparse-checkout definition. Some of these
behaviors may not be the desirable ones, such as some tests already
marked for failure in `t1092-sparse-checkout-compatibility.sh`.
The index extensions that may require special integrations are:
* FS Monitor
* Untracked cache
While integrating with these features, we should look for patterns that
might lead to better APIs for interacting with the index. Coalescing
common usage patterns into an API call can reduce the number of places
where sparse-directories need to be handled carefully.
Phase III: Important command speedups
-------------------------------------
At this point, the patterns for testing and implementing sparse-directory
logic should be relatively stable. This phase focuses on updating some of
the most common builtins that use the index to operate as O(Populated).
Here is a potential list of commands that could be valuable to integrate
at this point:
* `git commit`
* `git checkout`
* `git merge`
* `git rebase`
Hopefully, commands such as `git merge` and `git rebase` can benefit
instead from merge algorithms that do not use the index as a data
structure, such as the merge-ORT strategy. As these topics mature, we
may enable the ORT strategy by default for repositories using the
sparse-index feature.
Along with `git status` and `git add`, these commands cover the majority
of users' interactions with the working directory. In addition, we can
integrate with these commands:
* `git grep`
* `git rm`
These have been proposed as some whose behavior could change when in a
repo with a sparse-checkout definition. It would be good to include this
behavior automatically when using a sparse-index. Some clarity is needed
to make the behavior switch clear to the user.
This phase is the first where parallel work might be possible without too
much conflicts between topics.
Phase IV: The long tail
-----------------------
This last phase is less a "phase" and more "the new normal" after all of
the previous work.
To start, the `command_requires_full_index` option could be removed in
favor of expanding only when hitting an API guard.
There are many Git commands that could use special attention to operate as
O(Populated), while some might be so rare that it is acceptable to leave
them with additional overhead when a sparse-index is present.
Here are some commands that might be useful to update:
* `git sparse-checkout set`
* `git am`
* `git clean`
* `git stash`

View File

@@ -0,0 +1,121 @@
Trivial merge rules
===================
This document describes the outcomes of the trivial merge logic in read-tree.
One-way merge
-------------
This replaces the index with a different tree, keeping the stat info
for entries that don't change, and allowing -u to make the minimum
required changes to the working tree to have it match.
Entries marked '+' have stat information. Spaces marked '*' don't
affect the result.
index tree result
-----------------------
* (empty) (empty)
(empty) tree tree
index+ tree tree
index+ index index+
Two-way merge
-------------
It is permitted for the index to lack an entry; this does not prevent
any case from applying.
If the index exists, it is an error for it not to match either the old
or the result.
If multiple cases apply, the one used is listed first.
A result which changes the index is an error if the index is not empty
and not up to date.
Entries marked '+' have stat information. Spaces marked '*' don't
affect the result.
case index old new result
-------------------------------------
0/2 (empty) * (empty) (empty)
1/3 (empty) * new new
4/5 index+ (empty) (empty) index+
6/7 index+ (empty) index index+
10 index+ index (empty) (empty)
14/15 index+ old old index+
18/19 index+ old index index+
20 index+ index new new
Three-way merge
---------------
It is permitted for the index to lack an entry; this does not prevent
any case from applying.
If the index exists, it is an error for it not to match either the
head or (if the merge is trivial) the result.
If multiple cases apply, the one used is listed first.
A result of "no merge" means that index is left in stage 0, ancest in
stage 1, head in stage 2, and remote in stage 3 (if any of these are
empty, no entry is left for that stage). Otherwise, the given entry is
left in stage 0, and there are no other entries.
A result of "no merge" is an error if the index is not empty and not
up to date.
*empty* means that the tree must not have a directory-file conflict
with the entry.
For multiple ancestors, a '+' means that this case applies even if
only one ancestor or remote fits; a '^' means all of the ancestors
must be the same.
case ancest head remote result
----------------------------------------
1 (empty)+ (empty) (empty) (empty)
2ALT (empty)+ *empty* remote remote
2 (empty)^ (empty) remote no merge
3ALT (empty)+ head *empty* head
3 (empty)^ head (empty) no merge
4 (empty)^ head remote no merge
5ALT * head head head
6 ancest+ (empty) (empty) no merge
8 ancest^ (empty) ancest no merge
7 ancest+ (empty) remote no merge
10 ancest^ ancest (empty) no merge
9 ancest+ head (empty) no merge
16 anc1/anc2 anc1 anc2 no merge
13 ancest+ head ancest head
14 ancest+ ancest remote remote
11 ancest+ head remote no merge
Only #2ALT and #3ALT use *empty*, because these are the only cases
where there can be conflicts that didn't exist before. Note that we
allow directory-file conflicts between things in different stages
after the trivial merge.
A possible alternative for #6 is (empty), which would make it like
#1. This is not used, due to the likelihood that it arises due to
moving the file to multiple different locations or moving and deleting
it in different branches.
Case #1 is included for completeness, and also in case we decide to
put on '+' markings; any path that is never mentioned at all isn't
handled.
Note that #16 is when both #13 and #14 apply; in this case, we refuse
the trivial merge, because we can't tell from this data which is
right. This is a case of a reverted patch (in some direction, maybe
multiple times), and the right answer depends on looking at crossings
of history or common ancestors of the ancestors.
Note that, between #6, #7, #9, and #11, all cases not otherwise
covered are handled in this table.
For #8 and #10, there is alternative behavior, not currently
implemented, where the result is (empty). As currently implemented,
the automatic merge will generally give this effect.

View File

@@ -0,0 +1,608 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="en">
<head>
<meta charset="UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=edge"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<meta name="generator" content="Asciidoctor 2.0.23"/>
<title>Trivial merge rules</title>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"/>
<style>
/*! Asciidoctor default stylesheet | MIT License | https://asciidoctor.org */
/* Uncomment the following line when using as a custom stylesheet */
/* @import "https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"; */
html{font-family:sans-serif;-webkit-text-size-adjust:100%}
a{background:none}
a:focus{outline:thin dotted}
a:active,a:hover{outline:0}
h1{font-size:2em;margin:.67em 0}
b,strong{font-weight:bold}
abbr{font-size:.9em}
abbr[title]{cursor:help;border-bottom:1px dotted #dddddf;text-decoration:none}
dfn{font-style:italic}
hr{height:0}
mark{background:#ff0;color:#000}
code,kbd,pre,samp{font-family:monospace;font-size:1em}
pre{white-space:pre-wrap}
q{quotes:"\201C" "\201D" "\2018" "\2019"}
small{font-size:80%}
sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline}
sup{top:-.5em}
sub{bottom:-.25em}
img{border:0}
svg:not(:root){overflow:hidden}
figure{margin:0}
audio,video{display:inline-block}
audio:not([controls]){display:none;height:0}
fieldset{border:1px solid silver;margin:0 2px;padding:.35em .625em .75em}
legend{border:0;padding:0}
button,input,select,textarea{font-family:inherit;font-size:100%;margin:0}
button,input{line-height:normal}
button,select{text-transform:none}
button,html input[type=button],input[type=reset],input[type=submit]{-webkit-appearance:button;cursor:pointer}
button[disabled],html input[disabled]{cursor:default}
input[type=checkbox],input[type=radio]{padding:0}
button::-moz-focus-inner,input::-moz-focus-inner{border:0;padding:0}
textarea{overflow:auto;vertical-align:top}
table{border-collapse:collapse;border-spacing:0}
*,::before,::after{box-sizing:border-box}
html,body{font-size:100%}
body{background:#fff;color:rgba(0,0,0,.8);padding:0;margin:0;font-family:"Noto Serif","DejaVu Serif",serif;line-height:1;position:relative;cursor:auto;-moz-tab-size:4;-o-tab-size:4;tab-size:4;word-wrap:anywhere;-moz-osx-font-smoothing:grayscale;-webkit-font-smoothing:antialiased}
a:hover{cursor:pointer}
img,object,embed{max-width:100%;height:auto}
object,embed{height:100%}
img{-ms-interpolation-mode:bicubic}
.left{float:left!important}
.right{float:right!important}
.text-left{text-align:left!important}
.text-right{text-align:right!important}
.text-center{text-align:center!important}
.text-justify{text-align:justify!important}
.hide{display:none}
img,object,svg{display:inline-block;vertical-align:middle}
textarea{height:auto;min-height:50px}
select{width:100%}
.subheader,.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{line-height:1.45;color:#7a2518;font-weight:400;margin-top:0;margin-bottom:.25em}
div,dl,dt,dd,ul,ol,li,h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6,pre,form,p,blockquote,th,td{margin:0;padding:0}
a{color:#2156a5;text-decoration:underline;line-height:inherit}
a:hover,a:focus{color:#1d4b8f}
a img{border:0}
p{line-height:1.6;margin-bottom:1.25em;text-rendering:optimizeLegibility}
p aside{font-size:.875em;line-height:1.35;font-style:italic}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{font-family:"Open Sans","DejaVu Sans",sans-serif;font-weight:300;font-style:normal;color:#ba3925;text-rendering:optimizeLegibility;margin-top:1em;margin-bottom:.5em;line-height:1.0125em}
h1 small,h2 small,h3 small,#toctitle small,.sidebarblock>.content>.title small,h4 small,h5 small,h6 small{font-size:60%;color:#e99b8f;line-height:0}
h1{font-size:2.125em}
h2{font-size:1.6875em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.375em}
h4,h5{font-size:1.125em}
h6{font-size:1em}
hr{border:solid #dddddf;border-width:1px 0 0;clear:both;margin:1.25em 0 1.1875em}
em,i{font-style:italic;line-height:inherit}
strong,b{font-weight:bold;line-height:inherit}
small{font-size:60%;line-height:inherit}
code{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:400;color:rgba(0,0,0,.9)}
ul,ol,dl{line-height:1.6;margin-bottom:1.25em;list-style-position:outside;font-family:inherit}
ul,ol{margin-left:1.5em}
ul li ul,ul li ol{margin-left:1.25em;margin-bottom:0}
ul.circle{list-style-type:circle}
ul.disc{list-style-type:disc}
ul.square{list-style-type:square}
ul.circle ul:not([class]),ul.disc ul:not([class]),ul.square ul:not([class]){list-style:inherit}
ol li ul,ol li ol{margin-left:1.25em;margin-bottom:0}
dl dt{margin-bottom:.3125em;font-weight:bold}
dl dd{margin-bottom:1.25em}
blockquote{margin:0 0 1.25em;padding:.5625em 1.25em 0 1.1875em;border-left:1px solid #ddd}
blockquote,blockquote p{line-height:1.6;color:rgba(0,0,0,.85)}
@media screen and (min-width:768px){h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2}
h1{font-size:2.75em}
h2{font-size:2.3125em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.6875em}
h4{font-size:1.4375em}}
table{background:#fff;margin-bottom:1.25em;border:1px solid #dedede;word-wrap:normal}
table thead,table tfoot{background:#f7f8f7}
table thead tr th,table thead tr td,table tfoot tr th,table tfoot tr td{padding:.5em .625em .625em;font-size:inherit;color:rgba(0,0,0,.8);text-align:left}
table tr th,table tr td{padding:.5625em .625em;font-size:inherit;color:rgba(0,0,0,.8)}
table tr.even,table tr.alt{background:#f8f8f7}
table thead tr th,table tfoot tr th,table tbody tr td,table tr td,table tfoot tr td{line-height:1.6}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2;word-spacing:-.05em}
h1 strong,h2 strong,h3 strong,#toctitle strong,.sidebarblock>.content>.title strong,h4 strong,h5 strong,h6 strong{font-weight:400}
.center{margin-left:auto;margin-right:auto}
.stretch{width:100%}
.clearfix::before,.clearfix::after,.float-group::before,.float-group::after{content:" ";display:table}
.clearfix::after,.float-group::after{clear:both}
:not(pre).nobreak{word-wrap:normal}
:not(pre).nowrap{white-space:nowrap}
:not(pre).pre-wrap{white-space:pre-wrap}
:not(pre):not([class^=L])>code{font-size:.9375em;font-style:normal!important;letter-spacing:0;padding:.1em .5ex;word-spacing:-.15em;background:#f7f7f8;border-radius:4px;line-height:1.45;text-rendering:optimizeSpeed}
pre{color:rgba(0,0,0,.9);font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;line-height:1.45;text-rendering:optimizeSpeed}
pre code,pre pre{color:inherit;font-size:inherit;line-height:inherit}
pre>code{display:block}
pre.nowrap,pre.nowrap pre{white-space:pre;word-wrap:normal}
em em{font-style:normal}
strong strong{font-weight:400}
.keyseq{color:rgba(51,51,51,.8)}
kbd{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;display:inline-block;color:rgba(0,0,0,.8);font-size:.65em;line-height:1.45;background:#f7f7f7;border:1px solid #ccc;border-radius:3px;box-shadow:0 1px 0 rgba(0,0,0,.2),inset 0 0 0 .1em #fff;margin:0 .15em;padding:.2em .5em;vertical-align:middle;position:relative;top:-.1em;white-space:nowrap}
.keyseq kbd:first-child{margin-left:0}
.keyseq kbd:last-child{margin-right:0}
.menuseq,.menuref{color:#000}
.menuseq b:not(.caret),.menuref{font-weight:inherit}
.menuseq{word-spacing:-.02em}
.menuseq b.caret{font-size:1.25em;line-height:.8}
.menuseq i.caret{font-weight:bold;text-align:center;width:.45em}
b.button::before,b.button::after{position:relative;top:-1px;font-weight:400}
b.button::before{content:"[";padding:0 3px 0 2px}
b.button::after{content:"]";padding:0 2px 0 3px}
p a>code:hover{color:rgba(0,0,0,.9)}
#header,#content,#footnotes,#footer{width:100%;margin:0 auto;max-width:62.5em;*zoom:1;position:relative;padding-left:.9375em;padding-right:.9375em}
#header::before,#header::after,#content::before,#content::after,#footnotes::before,#footnotes::after,#footer::before,#footer::after{content:" ";display:table}
#header::after,#content::after,#footnotes::after,#footer::after{clear:both}
#content{margin-top:1.25em}
#content::before{content:none}
#header>h1:first-child{color:rgba(0,0,0,.85);margin-top:2.25rem;margin-bottom:0}
#header>h1:first-child+#toc{margin-top:8px;border-top:1px solid #dddddf}
#header>h1:only-child{border-bottom:1px solid #dddddf;padding-bottom:8px}
#header .details{border-bottom:1px solid #dddddf;line-height:1.45;padding-top:.25em;padding-bottom:.25em;padding-left:.25em;color:rgba(0,0,0,.6);display:flex;flex-flow:row wrap}
#header .details span:first-child{margin-left:-.125em}
#header .details span.email a{color:rgba(0,0,0,.85)}
#header .details br{display:none}
#header .details br+span::before{content:"\00a0\2013\00a0"}
#header .details br+span.author::before{content:"\00a0\22c5\00a0";color:rgba(0,0,0,.85)}
#header .details br+span#revremark::before{content:"\00a0|\00a0"}
#header #revnumber{text-transform:capitalize}
#header #revnumber::after{content:"\00a0"}
#content>h1:first-child:not([class]){color:rgba(0,0,0,.85);border-bottom:1px solid #dddddf;padding-bottom:8px;margin-top:0;padding-top:1rem;margin-bottom:1.25rem}
#toc{border-bottom:1px solid #e7e7e9;padding-bottom:.5em}
#toc>ul{margin-left:.125em}
#toc ul.sectlevel0>li>a{font-style:italic}
#toc ul.sectlevel0 ul.sectlevel1{margin:.5em 0}
#toc ul{font-family:"Open Sans","DejaVu Sans",sans-serif;list-style-type:none}
#toc li{line-height:1.3334;margin-top:.3334em}
#toc a{text-decoration:none}
#toc a:active{text-decoration:underline}
#toctitle{color:#7a2518;font-size:1.2em}
@media screen and (min-width:768px){#toctitle{font-size:1.375em}
body.toc2{padding-left:15em;padding-right:0}
body.toc2 #header>h1:nth-last-child(2){border-bottom:1px solid #dddddf;padding-bottom:8px}
#toc.toc2{margin-top:0!important;background:#f8f8f7;position:fixed;width:15em;left:0;top:0;border-right:1px solid #e7e7e9;border-top-width:0!important;border-bottom-width:0!important;z-index:1000;padding:1.25em 1em;height:100%;overflow:auto}
#toc.toc2 #toctitle{margin-top:0;margin-bottom:.8rem;font-size:1.2em}
#toc.toc2>ul{font-size:.9em;margin-bottom:0}
#toc.toc2 ul ul{margin-left:0;padding-left:1em}
#toc.toc2 ul.sectlevel0 ul.sectlevel1{padding-left:0;margin-top:.5em;margin-bottom:.5em}
body.toc2.toc-right{padding-left:0;padding-right:15em}
body.toc2.toc-right #toc.toc2{border-right-width:0;border-left:1px solid #e7e7e9;left:auto;right:0}}
@media screen and (min-width:1280px){body.toc2{padding-left:20em;padding-right:0}
#toc.toc2{width:20em}
#toc.toc2 #toctitle{font-size:1.375em}
#toc.toc2>ul{font-size:.95em}
#toc.toc2 ul ul{padding-left:1.25em}
body.toc2.toc-right{padding-left:0;padding-right:20em}}
#content #toc{border:1px solid #e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;border-radius:4px}
#content #toc>:first-child{margin-top:0}
#content #toc>:last-child{margin-bottom:0}
#footer{max-width:none;background:rgba(0,0,0,.8);padding:1.25em}
#footer-text{color:hsla(0,0%,100%,.8);line-height:1.44}
#content{margin-bottom:.625em}
.sect1{padding-bottom:.625em}
@media screen and (min-width:768px){#content{margin-bottom:1.25em}
.sect1{padding-bottom:1.25em}}
.sect1:last-child{padding-bottom:0}
.sect1+.sect1{border-top:1px solid #e7e7e9}
#content h1>a.anchor,h2>a.anchor,h3>a.anchor,#toctitle>a.anchor,.sidebarblock>.content>.title>a.anchor,h4>a.anchor,h5>a.anchor,h6>a.anchor{position:absolute;z-index:1001;width:1.5ex;margin-left:-1.5ex;display:block;text-decoration:none!important;visibility:hidden;text-align:center;font-weight:400}
#content h1>a.anchor::before,h2>a.anchor::before,h3>a.anchor::before,#toctitle>a.anchor::before,.sidebarblock>.content>.title>a.anchor::before,h4>a.anchor::before,h5>a.anchor::before,h6>a.anchor::before{content:"\00A7";font-size:.85em;display:block;padding-top:.1em}
#content h1:hover>a.anchor,#content h1>a.anchor:hover,h2:hover>a.anchor,h2>a.anchor:hover,h3:hover>a.anchor,#toctitle:hover>a.anchor,.sidebarblock>.content>.title:hover>a.anchor,h3>a.anchor:hover,#toctitle>a.anchor:hover,.sidebarblock>.content>.title>a.anchor:hover,h4:hover>a.anchor,h4>a.anchor:hover,h5:hover>a.anchor,h5>a.anchor:hover,h6:hover>a.anchor,h6>a.anchor:hover{visibility:visible}
#content h1>a.link,h2>a.link,h3>a.link,#toctitle>a.link,.sidebarblock>.content>.title>a.link,h4>a.link,h5>a.link,h6>a.link{color:#ba3925;text-decoration:none}
#content h1>a.link:hover,h2>a.link:hover,h3>a.link:hover,#toctitle>a.link:hover,.sidebarblock>.content>.title>a.link:hover,h4>a.link:hover,h5>a.link:hover,h6>a.link:hover{color:#a53221}
details,.audioblock,.imageblock,.literalblock,.listingblock,.stemblock,.videoblock{margin-bottom:1.25em}
details{margin-left:1.25rem}
details>summary{cursor:pointer;display:block;position:relative;line-height:1.6;margin-bottom:.625rem;outline:none;-webkit-tap-highlight-color:transparent}
details>summary::-webkit-details-marker{display:none}
details>summary::before{content:"";border:solid transparent;border-left:solid;border-width:.3em 0 .3em .5em;position:absolute;top:.5em;left:-1.25rem;transform:translateX(15%)}
details[open]>summary::before{border:solid transparent;border-top:solid;border-width:.5em .3em 0;transform:translateY(15%)}
details>summary::after{content:"";width:1.25rem;height:1em;position:absolute;top:.3em;left:-1.25rem}
.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{text-rendering:optimizeLegibility;text-align:left;font-family:"Noto Serif","DejaVu Serif",serif;font-size:1rem;font-style:italic}
table.tableblock.fit-content>caption.title{white-space:nowrap;width:0}
.paragraph.lead>p,#preamble>.sectionbody>[class=paragraph]:first-of-type p{font-size:1.21875em;line-height:1.6;color:rgba(0,0,0,.85)}
.admonitionblock>table{border-collapse:separate;border:0;background:none;width:100%}
.admonitionblock>table td.icon{text-align:center;width:80px}
.admonitionblock>table td.icon img{max-width:none}
.admonitionblock>table td.icon .title{font-weight:bold;font-family:"Open Sans","DejaVu Sans",sans-serif;text-transform:uppercase}
.admonitionblock>table td.content{padding-left:1.125em;padding-right:1.25em;border-left:1px solid #dddddf;color:rgba(0,0,0,.6);word-wrap:anywhere}
.admonitionblock>table td.content>:last-child>:last-child{margin-bottom:0}
.exampleblock>.content{border:1px solid #e6e6e6;margin-bottom:1.25em;padding:1.25em;background:#fff;border-radius:4px}
.sidebarblock{border:1px solid #dbdbd6;margin-bottom:1.25em;padding:1.25em;background:#f3f3f2;border-radius:4px}
.sidebarblock>.content>.title{color:#7a2518;margin-top:0;text-align:center}
.exampleblock>.content>:first-child,.sidebarblock>.content>:first-child{margin-top:0}
.exampleblock>.content>:last-child,.exampleblock>.content>:last-child>:last-child,.exampleblock>.content .olist>ol>li:last-child>:last-child,.exampleblock>.content .ulist>ul>li:last-child>:last-child,.exampleblock>.content .qlist>ol>li:last-child>:last-child,.sidebarblock>.content>:last-child,.sidebarblock>.content>:last-child>:last-child,.sidebarblock>.content .olist>ol>li:last-child>:last-child,.sidebarblock>.content .ulist>ul>li:last-child>:last-child,.sidebarblock>.content .qlist>ol>li:last-child>:last-child{margin-bottom:0}
.literalblock pre,.listingblock>.content>pre{border-radius:4px;overflow-x:auto;padding:1em;font-size:.8125em}
@media screen and (min-width:768px){.literalblock pre,.listingblock>.content>pre{font-size:.90625em}}
@media screen and (min-width:1280px){.literalblock pre,.listingblock>.content>pre{font-size:1em}}
.literalblock pre,.listingblock>.content>pre:not(.highlight),.listingblock>.content>pre[class=highlight],.listingblock>.content>pre[class^="highlight "]{background:#f7f7f8}
.literalblock.output pre{color:#f7f7f8;background:rgba(0,0,0,.9)}
.listingblock>.content{position:relative}
.listingblock code[data-lang]::before{display:none;content:attr(data-lang);position:absolute;font-size:.75em;top:.425rem;right:.5rem;line-height:1;text-transform:uppercase;color:inherit;opacity:.5}
.listingblock:hover code[data-lang]::before{display:block}
.listingblock.terminal pre .command::before{content:attr(data-prompt);padding-right:.5em;color:inherit;opacity:.5}
.listingblock.terminal pre .command:not([data-prompt])::before{content:"$"}
.listingblock pre.highlightjs{padding:0}
.listingblock pre.highlightjs>code{padding:1em;border-radius:4px}
.listingblock pre.prettyprint{border-width:0}
.prettyprint{background:#f7f7f8}
pre.prettyprint .linenums{line-height:1.45;margin-left:2em}
pre.prettyprint li{background:none;list-style-type:inherit;padding-left:0}
pre.prettyprint li code[data-lang]::before{opacity:1}
pre.prettyprint li:not(:first-child) code[data-lang]::before{display:none}
table.linenotable{border-collapse:separate;border:0;margin-bottom:0;background:none}
table.linenotable td[class]{color:inherit;vertical-align:top;padding:0;line-height:inherit;white-space:normal}
table.linenotable td.code{padding-left:.75em}
table.linenotable td.linenos,pre.pygments .linenos{border-right:1px solid;opacity:.35;padding-right:.5em;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}
pre.pygments span.linenos{display:inline-block;margin-right:.75em}
.quoteblock{margin:0 1em 1.25em 1.5em;display:table}
.quoteblock:not(.excerpt)>.title{margin-left:-1.5em;margin-bottom:.75em}
.quoteblock blockquote,.quoteblock p{color:rgba(0,0,0,.85);font-size:1.15rem;line-height:1.75;word-spacing:.1em;letter-spacing:0;font-style:italic;text-align:justify}
.quoteblock blockquote{margin:0;padding:0;border:0}
.quoteblock blockquote::before{content:"\201c";float:left;font-size:2.75em;font-weight:bold;line-height:.6em;margin-left:-.6em;color:#7a2518;text-shadow:0 1px 2px rgba(0,0,0,.1)}
.quoteblock blockquote>.paragraph:last-child p{margin-bottom:0}
.quoteblock .attribution{margin-top:.75em;margin-right:.5ex;text-align:right}
.verseblock{margin:0 1em 1.25em}
.verseblock pre{font-family:"Open Sans","DejaVu Sans",sans-serif;font-size:1.15rem;color:rgba(0,0,0,.85);font-weight:300;text-rendering:optimizeLegibility}
.verseblock pre strong{font-weight:400}
.verseblock .attribution{margin-top:1.25rem;margin-left:.5ex}
.quoteblock .attribution,.verseblock .attribution{font-size:.9375em;line-height:1.45;font-style:italic}
.quoteblock .attribution br,.verseblock .attribution br{display:none}
.quoteblock .attribution cite,.verseblock .attribution cite{display:block;letter-spacing:-.025em;color:rgba(0,0,0,.6)}
.quoteblock.abstract blockquote::before,.quoteblock.excerpt blockquote::before,.quoteblock .quoteblock blockquote::before{display:none}
.quoteblock.abstract blockquote,.quoteblock.abstract p,.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{line-height:1.6;word-spacing:0}
.quoteblock.abstract{margin:0 1em 1.25em;display:block}
.quoteblock.abstract>.title{margin:0 0 .375em;font-size:1.15em;text-align:center}
.quoteblock.excerpt>blockquote,.quoteblock .quoteblock{padding:0 0 .25em 1em;border-left:.25em solid #dddddf}
.quoteblock.excerpt,.quoteblock .quoteblock{margin-left:0}
.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{color:inherit;font-size:1.0625rem}
.quoteblock.excerpt .attribution,.quoteblock .quoteblock .attribution{color:inherit;font-size:.85rem;text-align:left;margin-right:0}
p.tableblock:last-child{margin-bottom:0}
td.tableblock>.content{margin-bottom:1.25em;word-wrap:anywhere}
td.tableblock>.content>:last-child{margin-bottom:-1.25em}
table.tableblock,th.tableblock,td.tableblock{border:0 solid #dedede}
table.grid-all>*>tr>*{border-width:1px}
table.grid-cols>*>tr>*{border-width:0 1px}
table.grid-rows>*>tr>*{border-width:1px 0}
table.frame-all{border-width:1px}
table.frame-ends{border-width:1px 0}
table.frame-sides{border-width:0 1px}
table.frame-none>colgroup+*>:first-child>*,table.frame-sides>colgroup+*>:first-child>*{border-top-width:0}
table.frame-none>:last-child>:last-child>*,table.frame-sides>:last-child>:last-child>*{border-bottom-width:0}
table.frame-none>*>tr>:first-child,table.frame-ends>*>tr>:first-child{border-left-width:0}
table.frame-none>*>tr>:last-child,table.frame-ends>*>tr>:last-child{border-right-width:0}
table.stripes-all>*>tr,table.stripes-odd>*>tr:nth-of-type(odd),table.stripes-even>*>tr:nth-of-type(even),table.stripes-hover>*>tr:hover{background:#f8f8f7}
th.halign-left,td.halign-left{text-align:left}
th.halign-right,td.halign-right{text-align:right}
th.halign-center,td.halign-center{text-align:center}
th.valign-top,td.valign-top{vertical-align:top}
th.valign-bottom,td.valign-bottom{vertical-align:bottom}
th.valign-middle,td.valign-middle{vertical-align:middle}
table thead th,table tfoot th{font-weight:bold}
tbody tr th{background:#f7f8f7}
tbody tr th,tbody tr th p,tfoot tr th,tfoot tr th p{color:rgba(0,0,0,.8);font-weight:bold}
p.tableblock>code:only-child{background:none;padding:0}
p.tableblock{font-size:1em}
ol{margin-left:1.75em}
ul li ol{margin-left:1.5em}
dl dd{margin-left:1.125em}
dl dd:last-child,dl dd:last-child>:last-child{margin-bottom:0}
li p,ul dd,ol dd,.olist .olist,.ulist .ulist,.ulist .olist,.olist .ulist{margin-bottom:.625em}
ul.checklist,ul.none,ol.none,ul.no-bullet,ol.no-bullet,ol.unnumbered,ul.unstyled,ol.unstyled{list-style-type:none}
ul.no-bullet,ol.no-bullet,ol.unnumbered{margin-left:.625em}
ul.unstyled,ol.unstyled{margin-left:0}
li>p:empty:only-child::before{content:"";display:inline-block}
ul.checklist>li>p:first-child{margin-left:-1em}
ul.checklist>li>p:first-child>.fa-square-o:first-child,ul.checklist>li>p:first-child>.fa-check-square-o:first-child{width:1.25em;font-size:.8em;position:relative;bottom:.125em}
ul.checklist>li>p:first-child>input[type=checkbox]:first-child{margin-right:.25em}
ul.inline{display:flex;flex-flow:row wrap;list-style:none;margin:0 0 .625em -1.25em}
ul.inline>li{margin-left:1.25em}
.unstyled dl dt{font-weight:400;font-style:normal}
ol.arabic{list-style-type:decimal}
ol.decimal{list-style-type:decimal-leading-zero}
ol.loweralpha{list-style-type:lower-alpha}
ol.upperalpha{list-style-type:upper-alpha}
ol.lowerroman{list-style-type:lower-roman}
ol.upperroman{list-style-type:upper-roman}
ol.lowergreek{list-style-type:lower-greek}
.hdlist>table,.colist>table{border:0;background:none}
.hdlist>table>tbody>tr,.colist>table>tbody>tr{background:none}
td.hdlist1,td.hdlist2{vertical-align:top;padding:0 .625em}
td.hdlist1{font-weight:bold;padding-bottom:1.25em}
td.hdlist2{word-wrap:anywhere}
.literalblock+.colist,.listingblock+.colist{margin-top:-.5em}
.colist td:not([class]):first-child{padding:.4em .75em 0;line-height:1;vertical-align:top}
.colist td:not([class]):first-child img{max-width:none}
.colist td:not([class]):last-child{padding:.25em 0}
.thumb,.th{line-height:0;display:inline-block;border:4px solid #fff;box-shadow:0 0 0 1px #ddd}
.imageblock.left{margin:.25em .625em 1.25em 0}
.imageblock.right{margin:.25em 0 1.25em .625em}
.imageblock>.title{margin-bottom:0}
.imageblock.thumb,.imageblock.th{border-width:6px}
.imageblock.thumb>.title,.imageblock.th>.title{padding:0 .125em}
.image.left,.image.right{margin-top:.25em;margin-bottom:.25em;display:inline-block;line-height:0}
.image.left{margin-right:.625em}
.image.right{margin-left:.625em}
a.image{text-decoration:none;display:inline-block}
a.image object{pointer-events:none}
sup.footnote,sup.footnoteref{font-size:.875em;position:static;vertical-align:super}
sup.footnote a,sup.footnoteref a{text-decoration:none}
sup.footnote a:active,sup.footnoteref a:active,#footnotes .footnote a:first-of-type:active{text-decoration:underline}
#footnotes{padding-top:.75em;padding-bottom:.75em;margin-bottom:.625em}
#footnotes hr{width:20%;min-width:6.25em;margin:-.25em 0 .75em;border-width:1px 0 0}
#footnotes .footnote{padding:0 .375em 0 .225em;line-height:1.3334;font-size:.875em;margin-left:1.2em;margin-bottom:.2em}
#footnotes .footnote a:first-of-type{font-weight:bold;text-decoration:none;margin-left:-1.05em}
#footnotes .footnote:last-of-type{margin-bottom:0}
#content #footnotes{margin-top:-.625em;margin-bottom:0;padding:.75em 0}
div.unbreakable{page-break-inside:avoid}
.big{font-size:larger}
.small{font-size:smaller}
.underline{text-decoration:underline}
.overline{text-decoration:overline}
.line-through{text-decoration:line-through}
.aqua{color:#00bfbf}
.aqua-background{background:#00fafa}
.black{color:#000}
.black-background{background:#000}
.blue{color:#0000bf}
.blue-background{background:#0000fa}
.fuchsia{color:#bf00bf}
.fuchsia-background{background:#fa00fa}
.gray{color:#606060}
.gray-background{background:#7d7d7d}
.green{color:#006000}
.green-background{background:#007d00}
.lime{color:#00bf00}
.lime-background{background:#00fa00}
.maroon{color:#600000}
.maroon-background{background:#7d0000}
.navy{color:#000060}
.navy-background{background:#00007d}
.olive{color:#606000}
.olive-background{background:#7d7d00}
.purple{color:#600060}
.purple-background{background:#7d007d}
.red{color:#bf0000}
.red-background{background:#fa0000}
.silver{color:#909090}
.silver-background{background:#bcbcbc}
.teal{color:#006060}
.teal-background{background:#007d7d}
.white{color:#bfbfbf}
.white-background{background:#fafafa}
.yellow{color:#bfbf00}
.yellow-background{background:#fafa00}
span.icon>.fa{cursor:default}
a span.icon>.fa{cursor:inherit}
.admonitionblock td.icon [class^="fa icon-"]{font-size:2.5em;text-shadow:1px 1px 2px rgba(0,0,0,.5);cursor:default}
.admonitionblock td.icon .icon-note::before{content:"\f05a";color:#19407c}
.admonitionblock td.icon .icon-tip::before{content:"\f0eb";text-shadow:1px 1px 2px rgba(155,155,0,.8);color:#111}
.admonitionblock td.icon .icon-warning::before{content:"\f071";color:#bf6900}
.admonitionblock td.icon .icon-caution::before{content:"\f06d";color:#bf3400}
.admonitionblock td.icon .icon-important::before{content:"\f06a";color:#bf0000}
.conum[data-value]{display:inline-block;color:#fff!important;background:rgba(0,0,0,.8);border-radius:50%;text-align:center;font-size:.75em;width:1.67em;height:1.67em;line-height:1.67em;font-family:"Open Sans","DejaVu Sans",sans-serif;font-style:normal;font-weight:bold}
.conum[data-value] *{color:#fff!important}
.conum[data-value]+b{display:none}
.conum[data-value]::after{content:attr(data-value)}
pre .conum[data-value]{position:relative;top:-.125em}
b.conum *{color:inherit!important}
.conum:not([data-value]):empty{display:none}
dt,th.tableblock,td.content,div.footnote{text-rendering:optimizeLegibility}
h1,h2,p,td.content,span.alt,summary{letter-spacing:-.01em}
p strong,td.content strong,div.footnote strong{letter-spacing:-.005em}
p,blockquote,dt,td.content,td.hdlist1,span.alt,summary{font-size:1.0625rem}
p{margin-bottom:1.25rem}
.sidebarblock p,.sidebarblock dt,.sidebarblock td.content,p.tableblock{font-size:1em}
.exampleblock>.content{background:#fffef7;border-color:#e0e0dc;box-shadow:0 1px 4px #e0e0dc}
.print-only{display:none!important}
@page{margin:1.25cm .75cm}
@media print{*{box-shadow:none!important;text-shadow:none!important}
html{font-size:80%}
a{color:inherit!important;text-decoration:underline!important}
a.bare,a[href^="#"],a[href^="mailto:"]{text-decoration:none!important}
a[href^="http:"]:not(.bare)::after,a[href^="https:"]:not(.bare)::after{content:"(" attr(href) ")";display:inline-block;font-size:.875em;padding-left:.25em}
abbr[title]{border-bottom:1px dotted}
abbr[title]::after{content:" (" attr(title) ")"}
pre,blockquote,tr,img,object,svg{page-break-inside:avoid}
thead{display:table-header-group}
svg{max-width:100%}
p,blockquote,dt,td.content{font-size:1em;orphans:3;widows:3}
h2,h3,#toctitle,.sidebarblock>.content>.title{page-break-after:avoid}
#header,#content,#footnotes,#footer{max-width:none}
#toc,.sidebarblock,.exampleblock>.content{background:none!important}
#toc{border-bottom:1px solid #dddddf!important;padding-bottom:0!important}
body.book #header{text-align:center}
body.book #header>h1:first-child{border:0!important;margin:2.5em 0 1em}
body.book #header .details{border:0!important;display:block;padding:0!important}
body.book #header .details span:first-child{margin-left:0!important}
body.book #header .details br{display:block}
body.book #header .details br+span::before{content:none!important}
body.book #toc{border:0!important;text-align:left!important;padding:0!important;margin:0!important}
body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-break-before:always}
.listingblock code[data-lang]::before{display:block}
#footer{padding:0 .9375em}
.hide-on-print{display:none!important}
.print-only{display:block!important}
.hide-for-print{display:none!important}
.show-for-print{display:inherit!important}}
@media amzn-kf8,print{#header>h1:first-child{margin-top:1.25rem}
.sect1{padding:0!important}
.sect1+.sect1{border:0}
#footer{background:none}
#footer-text{color:rgba(0,0,0,.6);font-size:.9em}}
@media amzn-kf8{#header,#content,#footnotes,#footer{padding:0}}
</style>
</head>
<body class="article">
<div id="header">
<h1>Trivial merge rules</h1>
</div>
<div id="content">
<div id="preamble">
<div class="sectionbody">
<div class="paragraph">
<p>This document describes the outcomes of the trivial merge logic in read-tree.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_one_way_merge">One-way merge</h2>
<div class="sectionbody">
<div class="paragraph">
<p>This replaces the index with a different tree, keeping the stat info
for entries that don&#8217;t change, and allowing -u to make the minimum
required changes to the working tree to have it match.</p>
</div>
<div class="paragraph">
<p>Entries marked <em>+</em> have stat information. Spaces marked <em>*</em> don&#8217;t
affect the result.</p>
</div>
<div class="literalblock">
<div class="content">
<pre>index tree result
-----------------------
* (empty) (empty)
(empty) tree tree
index+ tree tree
index+ index index+</pre>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_two_way_merge">Two-way merge</h2>
<div class="sectionbody">
<div class="paragraph">
<p>It is permitted for the index to lack an entry; this does not prevent
any case from applying.</p>
</div>
<div class="paragraph">
<p>If the index exists, it is an error for it not to match either the old
or the result.</p>
</div>
<div class="paragraph">
<p>If multiple cases apply, the one used is listed first.</p>
</div>
<div class="paragraph">
<p>A result which changes the index is an error if the index is not empty
and not up to date.</p>
</div>
<div class="paragraph">
<p>Entries marked <em>+</em> have stat information. Spaces marked <em>*</em> don&#8217;t
affect the result.</p>
</div>
<div class="literalblock">
<div class="content">
<pre>case index old new result
-------------------------------------
0/2 (empty) * (empty) (empty)
1/3 (empty) * new new
4/5 index+ (empty) (empty) index+
6/7 index+ (empty) index index+
10 index+ index (empty) (empty)
14/15 index+ old old index+
18/19 index+ old index index+
20 index+ index new new</pre>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_three_way_merge">Three-way merge</h2>
<div class="sectionbody">
<div class="paragraph">
<p>It is permitted for the index to lack an entry; this does not prevent
any case from applying.</p>
</div>
<div class="paragraph">
<p>If the index exists, it is an error for it not to match either the
head or (if the merge is trivial) the result.</p>
</div>
<div class="paragraph">
<p>If multiple cases apply, the one used is listed first.</p>
</div>
<div class="paragraph">
<p>A result of "no merge" means that index is left in stage 0, ancest in
stage 1, head in stage 2, and remote in stage 3 (if any of these are
empty, no entry is left for that stage). Otherwise, the given entry is
left in stage 0, and there are no other entries.</p>
</div>
<div class="paragraph">
<p>A result of "no merge" is an error if the index is not empty and not
up to date.</p>
</div>
<div class="paragraph">
<p><strong>empty</strong> means that the tree must not have a directory-file conflict
with the entry.</p>
</div>
<div class="paragraph">
<p>For multiple ancestors, a <em>+</em> means that this case applies even if
only one ancestor or remote fits; a <em>^</em> means all of the ancestors
must be the same.</p>
</div>
<div class="literalblock">
<div class="content">
<pre>case ancest head remote result
----------------------------------------
1 (empty)+ (empty) (empty) (empty)
2ALT (empty)+ *empty* remote remote
2 (empty)^ (empty) remote no merge
3ALT (empty)+ head *empty* head
3 (empty)^ head (empty) no merge
4 (empty)^ head remote no merge
5ALT * head head head
6 ancest+ (empty) (empty) no merge
8 ancest^ (empty) ancest no merge
7 ancest+ (empty) remote no merge
10 ancest^ ancest (empty) no merge
9 ancest+ head (empty) no merge
16 anc1/anc2 anc1 anc2 no merge
13 ancest+ head ancest head
14 ancest+ ancest remote remote
11 ancest+ head remote no merge</pre>
</div>
</div>
<div class="paragraph">
<p>Only #2ALT and #3ALT use <strong>empty</strong>, because these are the only cases
where there can be conflicts that didn&#8217;t exist before. Note that we
allow directory-file conflicts between things in different stages
after the trivial merge.</p>
</div>
<div class="paragraph">
<p>A possible alternative for #6 is (empty), which would make it like
#1. This is not used, due to the likelihood that it arises due to
moving the file to multiple different locations or moving and deleting
it in different branches.</p>
</div>
<div class="paragraph">
<p>Case #1 is included for completeness, and also in case we decide to
put on <em>+</em> markings; any path that is never mentioned at all isn&#8217;t
handled.</p>
</div>
<div class="paragraph">
<p>Note that #16 is when both #13 and #14 apply; in this case, we refuse
the trivial merge, because we can&#8217;t tell from this data which is
right. This is a case of a reverted patch (in some direction, maybe
multiple times), and the right answer depends on looking at crossings
of history or common ancestors of the ancestors.</p>
</div>
<div class="paragraph">
<p>Note that, between #6, #7, #9, and #11, all cases not otherwise
covered are handled in this table.</p>
</div>
<div class="paragraph">
<p>For #8 and #10, there is alternative behavior, not currently
implemented, where the result is (empty). As currently implemented,
the automatic merge will generally give this effect.</p>
</div>
</div>
</div>
</div>
<div id="footer">
<div id="footer-text">
Last updated 2025-03-17 07:09:38 UTC
</div>
</div>
</body>
</html>

View File

@@ -0,0 +1,242 @@
= Unit Testing
In our current testing environment, we spend a significant amount of effort
crafting end-to-end tests for error conditions that could easily be captured by
unit tests (or we simply forgo some hard-to-setup and rare error conditions).
Unit tests additionally provide stability to the codebase and can simplify
debugging through isolation. Writing unit tests in pure C, rather than with our
current shell/test-tool helper setup, simplifies test setup, simplifies passing
data around (no shell-isms required), and reduces testing runtime by not
spawning a separate process for every test invocation.
We believe that a large body of unit tests, living alongside the existing test
suite, will improve code quality for the Git project.
== Definitions
For the purposes of this document, we'll use *test framework* to refer to
projects that support writing test cases and running tests within the context
of a single executable. *Test harness* will refer to projects that manage
running multiple executables (each of which may contain multiple test cases) and
aggregating their results.
In reality, these terms are not strictly defined, and many of the projects
discussed below contain features from both categories.
For now, we will evaluate projects solely on their framework features. Since we
are relying on having TAP output (see below), we can assume that any framework
can be made to work with a harness that we can choose later.
== Summary
We believe the best way forward is to implement a custom TAP framework for the
Git project. We use a version of the framework originally proposed in
https://lore.kernel.org/git/c902a166-98ce-afba-93f2-ea6027557176@gmail.com/[1].
See the <<framework-selection,Framework Selection>> section below for the
rationale behind this decision.
== Choosing a test harness
During upstream discussion, it was occasionally noted that `prove` provides many
convenient features, such as scheduling slower tests first, or re-running
previously failed tests.
While we already support the use of `prove` as a test harness for the shell
tests, it is not strictly required. The t/Makefile allows running shell tests
directly (though with interleaved output if parallelism is enabled). Git
developers who wish to use `prove` as a more advanced harness can do so by
setting DEFAULT_TEST_TARGET=prove in their config.mak.
We will follow a similar approach for unit tests: by default the test
executables will be run directly from the t/Makefile, but `prove` can be
configured with DEFAULT_UNIT_TEST_TARGET=prove.
[[framework-selection]]
== Framework selection
There are a variety of features we can use to rank the candidate frameworks, and
those features have different priorities:
* Critical features: we probably won't consider a framework without these
** Can we legally / easily use the project?
*** <<license,License>>
*** <<vendorable-or-ubiquitous,Vendorable or ubiquitous>>
*** <<maintainable-extensible,Maintainable / extensible>>
*** <<major-platform-support,Major platform support>>
** Does the project support our bare-minimum needs?
*** <<tap-support,TAP support>>
*** <<diagnostic-output,Diagnostic output>>
*** <<runtime-skippable-tests,Runtime-skippable tests>>
* Nice-to-have features:
** <<parallel-execution,Parallel execution>>
** <<mock-support,Mock support>>
** <<signal-error-handling,Signal & error-handling>>
* Tie-breaker stats
** <<project-kloc,Project KLOC>>
** <<adoption,Adoption>>
[[license]]
=== License
We must be able to legally use the framework in connection with Git. As Git is
licensed only under GPLv2, we must eliminate any LGPLv3, GPLv3, or Apache 2.0
projects.
[[vendorable-or-ubiquitous]]
=== Vendorable or ubiquitous
We want to avoid forcing Git developers to install new tools just to run unit
tests. Any prospective frameworks and harnesses must either be vendorable
(meaning, we can copy their source directly into Git's repository), or so
ubiquitous that it is reasonable to expect that most developers will have the
tools installed already.
[[maintainable-extensible]]
=== Maintainable / extensible
It is unlikely that any pre-existing project perfectly fits our needs, so any
project we select will need to be actively maintained and open to accepting
changes. Alternatively, assuming we are vendoring the source into our repo, it
must be simple enough that Git developers can feel comfortable making changes as
needed to our version.
In the comparison table below, "True" means that the framework seems to have
active developers, that it is simple enough that Git developers can make changes
to it, and that the project seems open to accepting external contributions (or
that it is vendorable). "Partial" means that at least one of the above
conditions holds.
[[major-platform-support]]
=== Major platform support
At a bare minimum, unit-testing must work on Linux, MacOS, and Windows.
In the comparison table below, "True" means that it works on all three major
platforms with no issues. "Partial" means that there may be annoyances on one or
more platforms, but it is still usable in principle.
[[tap-support]]
=== TAP support
The https://testanything.org/[Test Anything Protocol] is a text-based interface
that allows tests to communicate with a test harness. It is already used by
Git's integration test suite. Supporting TAP output is a mandatory feature for
any prospective test framework.
In the comparison table below, "True" means this is natively supported.
"Partial" means TAP output must be generated by post-processing the native
output.
Frameworks that do not have at least Partial support will not be evaluated
further.
[[diagnostic-output]]
=== Diagnostic output
When a test case fails, the framework must generate enough diagnostic output to
help developers find the appropriate test case in source code in order to debug
the failure.
[[runtime-skippable-tests]]
=== Runtime-skippable tests
Test authors may wish to skip certain test cases based on runtime circumstances,
so the framework should support this.
[[parallel-execution]]
=== Parallel execution
Ideally, we will build up a significant collection of unit test cases, most
likely split across multiple executables. It will be necessary to run these
tests in parallel to enable fast develop-test-debug cycles.
In the comparison table below, "True" means that individual test cases within a
single test executable can be run in parallel. We assume that executable-level
parallelism can be handled by the test harness.
[[mock-support]]
=== Mock support
Unit test authors may wish to test code that interacts with objects that may be
inconvenient to handle in a test (e.g. interacting with a network service).
Mocking allows test authors to provide a fake implementation of these objects
for more convenient tests.
[[signal-error-handling]]
=== Signal & error handling
The test framework should fail gracefully when test cases are themselves buggy
or when they are interrupted by signals during runtime.
[[project-kloc]]
=== Project KLOC
The size of the project, in thousands of lines of code as measured by
https://dwheeler.com/sloccount/[sloccount] (rounded up to the next multiple of
1,000). As a tie-breaker, we probably prefer a project with fewer LOC.
[[adoption]]
=== Adoption
As a tie-breaker, we prefer a more widely-used project. We use the number of
GitHub / GitLab stars to estimate this.
=== Comparison
:true: [lime-background]#True#
:false: [red-background]#False#
:partial: [yellow-background]#Partial#
:gpl: [lime-background]#GPL v2#
:isc: [lime-background]#ISC#
:mit: [lime-background]#MIT#
:expat: [lime-background]#Expat#
:lgpl: [lime-background]#LGPL v2.1#
:custom-impl: https://lore.kernel.org/git/c902a166-98ce-afba-93f2-ea6027557176@gmail.com/[Custom Git impl.]
:greatest: https://github.com/silentbicycle/greatest[Greatest]
:criterion: https://github.com/Snaipe/Criterion[Criterion]
:c-tap: https://github.com/rra/c-tap-harness/[C TAP]
:check: https://libcheck.github.io/check/[Check]
:clar: https://github.com/clar-test/clar[Clar]
[format="csv",options="header",width="33%",subs="specialcharacters,attributes,quotes,macros"]
|=====
Framework,"<<license,License>>","<<vendorable-or-ubiquitous,Vendorable or ubiquitous>>","<<maintainable-extensible,Maintainable / extensible>>","<<major-platform-support,Major platform support>>","<<tap-support,TAP support>>","<<diagnostic-output,Diagnostic output>>","<<runtime--skippable-tests,Runtime- skippable tests>>","<<parallel-execution,Parallel execution>>","<<mock-support,Mock support>>","<<signal-error-handling,Signal & error handling>>","<<project-kloc,Project KLOC>>","<<adoption,Adoption>>"
{custom-impl},{gpl},{true},{true},{true},{true},{true},{true},{false},{false},{false},1,0
{greatest},{isc},{true},{partial},{true},{partial},{true},{true},{false},{false},{false},3,1400
{criterion},{mit},{false},{partial},{true},{true},{true},{true},{true},{false},{true},19,1800
{c-tap},{expat},{true},{partial},{partial},{true},{false},{true},{false},{false},{false},4,33
{check},{lgpl},{false},{partial},{true},{true},{true},{false},{false},{false},{true},17,973
{clar},{isc},{false},{partial},{true},{true},{true},{true},{false},{false},{true},1,192
|=====
=== Additional framework candidates
Several suggested frameworks have been eliminated from consideration:
* Incompatible licenses:
** https://github.com/zorgnax/libtap[libtap] (LGPL v3)
** https://cmocka.org/[cmocka] (Apache 2.0)
* Missing source: https://www.kindahl.net/mytap/doc/index.html[MyTap]
* No TAP support:
** https://nemequ.github.io/munit/[µnit]
** https://github.com/google/cmockery[cmockery]
** https://github.com/lpabon/cmockery2[cmockery2]
** https://github.com/ThrowTheSwitch/Unity[Unity]
** https://github.com/siu/minunit[minunit]
** https://cunit.sourceforge.net/[CUnit]
== Milestones
* Add useful tests of library-like code
* Integrate with
https://lore.kernel.org/git/20230502211454.1673000-1-calvinwan@google.com/[stdlib
work]
* Run alongside regular `make test` target

View File

@@ -0,0 +1,931 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="en">
<head>
<meta charset="UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=edge"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<meta name="generator" content="Asciidoctor 2.0.23"/>
<title>Unit Testing</title>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"/>
<style>
/*! Asciidoctor default stylesheet | MIT License | https://asciidoctor.org */
/* Uncomment the following line when using as a custom stylesheet */
/* @import "https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"; */
html{font-family:sans-serif;-webkit-text-size-adjust:100%}
a{background:none}
a:focus{outline:thin dotted}
a:active,a:hover{outline:0}
h1{font-size:2em;margin:.67em 0}
b,strong{font-weight:bold}
abbr{font-size:.9em}
abbr[title]{cursor:help;border-bottom:1px dotted #dddddf;text-decoration:none}
dfn{font-style:italic}
hr{height:0}
mark{background:#ff0;color:#000}
code,kbd,pre,samp{font-family:monospace;font-size:1em}
pre{white-space:pre-wrap}
q{quotes:"\201C" "\201D" "\2018" "\2019"}
small{font-size:80%}
sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline}
sup{top:-.5em}
sub{bottom:-.25em}
img{border:0}
svg:not(:root){overflow:hidden}
figure{margin:0}
audio,video{display:inline-block}
audio:not([controls]){display:none;height:0}
fieldset{border:1px solid silver;margin:0 2px;padding:.35em .625em .75em}
legend{border:0;padding:0}
button,input,select,textarea{font-family:inherit;font-size:100%;margin:0}
button,input{line-height:normal}
button,select{text-transform:none}
button,html input[type=button],input[type=reset],input[type=submit]{-webkit-appearance:button;cursor:pointer}
button[disabled],html input[disabled]{cursor:default}
input[type=checkbox],input[type=radio]{padding:0}
button::-moz-focus-inner,input::-moz-focus-inner{border:0;padding:0}
textarea{overflow:auto;vertical-align:top}
table{border-collapse:collapse;border-spacing:0}
*,::before,::after{box-sizing:border-box}
html,body{font-size:100%}
body{background:#fff;color:rgba(0,0,0,.8);padding:0;margin:0;font-family:"Noto Serif","DejaVu Serif",serif;line-height:1;position:relative;cursor:auto;-moz-tab-size:4;-o-tab-size:4;tab-size:4;word-wrap:anywhere;-moz-osx-font-smoothing:grayscale;-webkit-font-smoothing:antialiased}
a:hover{cursor:pointer}
img,object,embed{max-width:100%;height:auto}
object,embed{height:100%}
img{-ms-interpolation-mode:bicubic}
.left{float:left!important}
.right{float:right!important}
.text-left{text-align:left!important}
.text-right{text-align:right!important}
.text-center{text-align:center!important}
.text-justify{text-align:justify!important}
.hide{display:none}
img,object,svg{display:inline-block;vertical-align:middle}
textarea{height:auto;min-height:50px}
select{width:100%}
.subheader,.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{line-height:1.45;color:#7a2518;font-weight:400;margin-top:0;margin-bottom:.25em}
div,dl,dt,dd,ul,ol,li,h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6,pre,form,p,blockquote,th,td{margin:0;padding:0}
a{color:#2156a5;text-decoration:underline;line-height:inherit}
a:hover,a:focus{color:#1d4b8f}
a img{border:0}
p{line-height:1.6;margin-bottom:1.25em;text-rendering:optimizeLegibility}
p aside{font-size:.875em;line-height:1.35;font-style:italic}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{font-family:"Open Sans","DejaVu Sans",sans-serif;font-weight:300;font-style:normal;color:#ba3925;text-rendering:optimizeLegibility;margin-top:1em;margin-bottom:.5em;line-height:1.0125em}
h1 small,h2 small,h3 small,#toctitle small,.sidebarblock>.content>.title small,h4 small,h5 small,h6 small{font-size:60%;color:#e99b8f;line-height:0}
h1{font-size:2.125em}
h2{font-size:1.6875em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.375em}
h4,h5{font-size:1.125em}
h6{font-size:1em}
hr{border:solid #dddddf;border-width:1px 0 0;clear:both;margin:1.25em 0 1.1875em}
em,i{font-style:italic;line-height:inherit}
strong,b{font-weight:bold;line-height:inherit}
small{font-size:60%;line-height:inherit}
code{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:400;color:rgba(0,0,0,.9)}
ul,ol,dl{line-height:1.6;margin-bottom:1.25em;list-style-position:outside;font-family:inherit}
ul,ol{margin-left:1.5em}
ul li ul,ul li ol{margin-left:1.25em;margin-bottom:0}
ul.circle{list-style-type:circle}
ul.disc{list-style-type:disc}
ul.square{list-style-type:square}
ul.circle ul:not([class]),ul.disc ul:not([class]),ul.square ul:not([class]){list-style:inherit}
ol li ul,ol li ol{margin-left:1.25em;margin-bottom:0}
dl dt{margin-bottom:.3125em;font-weight:bold}
dl dd{margin-bottom:1.25em}
blockquote{margin:0 0 1.25em;padding:.5625em 1.25em 0 1.1875em;border-left:1px solid #ddd}
blockquote,blockquote p{line-height:1.6;color:rgba(0,0,0,.85)}
@media screen and (min-width:768px){h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2}
h1{font-size:2.75em}
h2{font-size:2.3125em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.6875em}
h4{font-size:1.4375em}}
table{background:#fff;margin-bottom:1.25em;border:1px solid #dedede;word-wrap:normal}
table thead,table tfoot{background:#f7f8f7}
table thead tr th,table thead tr td,table tfoot tr th,table tfoot tr td{padding:.5em .625em .625em;font-size:inherit;color:rgba(0,0,0,.8);text-align:left}
table tr th,table tr td{padding:.5625em .625em;font-size:inherit;color:rgba(0,0,0,.8)}
table tr.even,table tr.alt{background:#f8f8f7}
table thead tr th,table tfoot tr th,table tbody tr td,table tr td,table tfoot tr td{line-height:1.6}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2;word-spacing:-.05em}
h1 strong,h2 strong,h3 strong,#toctitle strong,.sidebarblock>.content>.title strong,h4 strong,h5 strong,h6 strong{font-weight:400}
.center{margin-left:auto;margin-right:auto}
.stretch{width:100%}
.clearfix::before,.clearfix::after,.float-group::before,.float-group::after{content:" ";display:table}
.clearfix::after,.float-group::after{clear:both}
:not(pre).nobreak{word-wrap:normal}
:not(pre).nowrap{white-space:nowrap}
:not(pre).pre-wrap{white-space:pre-wrap}
:not(pre):not([class^=L])>code{font-size:.9375em;font-style:normal!important;letter-spacing:0;padding:.1em .5ex;word-spacing:-.15em;background:#f7f7f8;border-radius:4px;line-height:1.45;text-rendering:optimizeSpeed}
pre{color:rgba(0,0,0,.9);font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;line-height:1.45;text-rendering:optimizeSpeed}
pre code,pre pre{color:inherit;font-size:inherit;line-height:inherit}
pre>code{display:block}
pre.nowrap,pre.nowrap pre{white-space:pre;word-wrap:normal}
em em{font-style:normal}
strong strong{font-weight:400}
.keyseq{color:rgba(51,51,51,.8)}
kbd{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;display:inline-block;color:rgba(0,0,0,.8);font-size:.65em;line-height:1.45;background:#f7f7f7;border:1px solid #ccc;border-radius:3px;box-shadow:0 1px 0 rgba(0,0,0,.2),inset 0 0 0 .1em #fff;margin:0 .15em;padding:.2em .5em;vertical-align:middle;position:relative;top:-.1em;white-space:nowrap}
.keyseq kbd:first-child{margin-left:0}
.keyseq kbd:last-child{margin-right:0}
.menuseq,.menuref{color:#000}
.menuseq b:not(.caret),.menuref{font-weight:inherit}
.menuseq{word-spacing:-.02em}
.menuseq b.caret{font-size:1.25em;line-height:.8}
.menuseq i.caret{font-weight:bold;text-align:center;width:.45em}
b.button::before,b.button::after{position:relative;top:-1px;font-weight:400}
b.button::before{content:"[";padding:0 3px 0 2px}
b.button::after{content:"]";padding:0 2px 0 3px}
p a>code:hover{color:rgba(0,0,0,.9)}
#header,#content,#footnotes,#footer{width:100%;margin:0 auto;max-width:62.5em;*zoom:1;position:relative;padding-left:.9375em;padding-right:.9375em}
#header::before,#header::after,#content::before,#content::after,#footnotes::before,#footnotes::after,#footer::before,#footer::after{content:" ";display:table}
#header::after,#content::after,#footnotes::after,#footer::after{clear:both}
#content{margin-top:1.25em}
#content::before{content:none}
#header>h1:first-child{color:rgba(0,0,0,.85);margin-top:2.25rem;margin-bottom:0}
#header>h1:first-child+#toc{margin-top:8px;border-top:1px solid #dddddf}
#header>h1:only-child{border-bottom:1px solid #dddddf;padding-bottom:8px}
#header .details{border-bottom:1px solid #dddddf;line-height:1.45;padding-top:.25em;padding-bottom:.25em;padding-left:.25em;color:rgba(0,0,0,.6);display:flex;flex-flow:row wrap}
#header .details span:first-child{margin-left:-.125em}
#header .details span.email a{color:rgba(0,0,0,.85)}
#header .details br{display:none}
#header .details br+span::before{content:"\00a0\2013\00a0"}
#header .details br+span.author::before{content:"\00a0\22c5\00a0";color:rgba(0,0,0,.85)}
#header .details br+span#revremark::before{content:"\00a0|\00a0"}
#header #revnumber{text-transform:capitalize}
#header #revnumber::after{content:"\00a0"}
#content>h1:first-child:not([class]){color:rgba(0,0,0,.85);border-bottom:1px solid #dddddf;padding-bottom:8px;margin-top:0;padding-top:1rem;margin-bottom:1.25rem}
#toc{border-bottom:1px solid #e7e7e9;padding-bottom:.5em}
#toc>ul{margin-left:.125em}
#toc ul.sectlevel0>li>a{font-style:italic}
#toc ul.sectlevel0 ul.sectlevel1{margin:.5em 0}
#toc ul{font-family:"Open Sans","DejaVu Sans",sans-serif;list-style-type:none}
#toc li{line-height:1.3334;margin-top:.3334em}
#toc a{text-decoration:none}
#toc a:active{text-decoration:underline}
#toctitle{color:#7a2518;font-size:1.2em}
@media screen and (min-width:768px){#toctitle{font-size:1.375em}
body.toc2{padding-left:15em;padding-right:0}
body.toc2 #header>h1:nth-last-child(2){border-bottom:1px solid #dddddf;padding-bottom:8px}
#toc.toc2{margin-top:0!important;background:#f8f8f7;position:fixed;width:15em;left:0;top:0;border-right:1px solid #e7e7e9;border-top-width:0!important;border-bottom-width:0!important;z-index:1000;padding:1.25em 1em;height:100%;overflow:auto}
#toc.toc2 #toctitle{margin-top:0;margin-bottom:.8rem;font-size:1.2em}
#toc.toc2>ul{font-size:.9em;margin-bottom:0}
#toc.toc2 ul ul{margin-left:0;padding-left:1em}
#toc.toc2 ul.sectlevel0 ul.sectlevel1{padding-left:0;margin-top:.5em;margin-bottom:.5em}
body.toc2.toc-right{padding-left:0;padding-right:15em}
body.toc2.toc-right #toc.toc2{border-right-width:0;border-left:1px solid #e7e7e9;left:auto;right:0}}
@media screen and (min-width:1280px){body.toc2{padding-left:20em;padding-right:0}
#toc.toc2{width:20em}
#toc.toc2 #toctitle{font-size:1.375em}
#toc.toc2>ul{font-size:.95em}
#toc.toc2 ul ul{padding-left:1.25em}
body.toc2.toc-right{padding-left:0;padding-right:20em}}
#content #toc{border:1px solid #e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;border-radius:4px}
#content #toc>:first-child{margin-top:0}
#content #toc>:last-child{margin-bottom:0}
#footer{max-width:none;background:rgba(0,0,0,.8);padding:1.25em}
#footer-text{color:hsla(0,0%,100%,.8);line-height:1.44}
#content{margin-bottom:.625em}
.sect1{padding-bottom:.625em}
@media screen and (min-width:768px){#content{margin-bottom:1.25em}
.sect1{padding-bottom:1.25em}}
.sect1:last-child{padding-bottom:0}
.sect1+.sect1{border-top:1px solid #e7e7e9}
#content h1>a.anchor,h2>a.anchor,h3>a.anchor,#toctitle>a.anchor,.sidebarblock>.content>.title>a.anchor,h4>a.anchor,h5>a.anchor,h6>a.anchor{position:absolute;z-index:1001;width:1.5ex;margin-left:-1.5ex;display:block;text-decoration:none!important;visibility:hidden;text-align:center;font-weight:400}
#content h1>a.anchor::before,h2>a.anchor::before,h3>a.anchor::before,#toctitle>a.anchor::before,.sidebarblock>.content>.title>a.anchor::before,h4>a.anchor::before,h5>a.anchor::before,h6>a.anchor::before{content:"\00A7";font-size:.85em;display:block;padding-top:.1em}
#content h1:hover>a.anchor,#content h1>a.anchor:hover,h2:hover>a.anchor,h2>a.anchor:hover,h3:hover>a.anchor,#toctitle:hover>a.anchor,.sidebarblock>.content>.title:hover>a.anchor,h3>a.anchor:hover,#toctitle>a.anchor:hover,.sidebarblock>.content>.title>a.anchor:hover,h4:hover>a.anchor,h4>a.anchor:hover,h5:hover>a.anchor,h5>a.anchor:hover,h6:hover>a.anchor,h6>a.anchor:hover{visibility:visible}
#content h1>a.link,h2>a.link,h3>a.link,#toctitle>a.link,.sidebarblock>.content>.title>a.link,h4>a.link,h5>a.link,h6>a.link{color:#ba3925;text-decoration:none}
#content h1>a.link:hover,h2>a.link:hover,h3>a.link:hover,#toctitle>a.link:hover,.sidebarblock>.content>.title>a.link:hover,h4>a.link:hover,h5>a.link:hover,h6>a.link:hover{color:#a53221}
details,.audioblock,.imageblock,.literalblock,.listingblock,.stemblock,.videoblock{margin-bottom:1.25em}
details{margin-left:1.25rem}
details>summary{cursor:pointer;display:block;position:relative;line-height:1.6;margin-bottom:.625rem;outline:none;-webkit-tap-highlight-color:transparent}
details>summary::-webkit-details-marker{display:none}
details>summary::before{content:"";border:solid transparent;border-left:solid;border-width:.3em 0 .3em .5em;position:absolute;top:.5em;left:-1.25rem;transform:translateX(15%)}
details[open]>summary::before{border:solid transparent;border-top:solid;border-width:.5em .3em 0;transform:translateY(15%)}
details>summary::after{content:"";width:1.25rem;height:1em;position:absolute;top:.3em;left:-1.25rem}
.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{text-rendering:optimizeLegibility;text-align:left;font-family:"Noto Serif","DejaVu Serif",serif;font-size:1rem;font-style:italic}
table.tableblock.fit-content>caption.title{white-space:nowrap;width:0}
.paragraph.lead>p,#preamble>.sectionbody>[class=paragraph]:first-of-type p{font-size:1.21875em;line-height:1.6;color:rgba(0,0,0,.85)}
.admonitionblock>table{border-collapse:separate;border:0;background:none;width:100%}
.admonitionblock>table td.icon{text-align:center;width:80px}
.admonitionblock>table td.icon img{max-width:none}
.admonitionblock>table td.icon .title{font-weight:bold;font-family:"Open Sans","DejaVu Sans",sans-serif;text-transform:uppercase}
.admonitionblock>table td.content{padding-left:1.125em;padding-right:1.25em;border-left:1px solid #dddddf;color:rgba(0,0,0,.6);word-wrap:anywhere}
.admonitionblock>table td.content>:last-child>:last-child{margin-bottom:0}
.exampleblock>.content{border:1px solid #e6e6e6;margin-bottom:1.25em;padding:1.25em;background:#fff;border-radius:4px}
.sidebarblock{border:1px solid #dbdbd6;margin-bottom:1.25em;padding:1.25em;background:#f3f3f2;border-radius:4px}
.sidebarblock>.content>.title{color:#7a2518;margin-top:0;text-align:center}
.exampleblock>.content>:first-child,.sidebarblock>.content>:first-child{margin-top:0}
.exampleblock>.content>:last-child,.exampleblock>.content>:last-child>:last-child,.exampleblock>.content .olist>ol>li:last-child>:last-child,.exampleblock>.content .ulist>ul>li:last-child>:last-child,.exampleblock>.content .qlist>ol>li:last-child>:last-child,.sidebarblock>.content>:last-child,.sidebarblock>.content>:last-child>:last-child,.sidebarblock>.content .olist>ol>li:last-child>:last-child,.sidebarblock>.content .ulist>ul>li:last-child>:last-child,.sidebarblock>.content .qlist>ol>li:last-child>:last-child{margin-bottom:0}
.literalblock pre,.listingblock>.content>pre{border-radius:4px;overflow-x:auto;padding:1em;font-size:.8125em}
@media screen and (min-width:768px){.literalblock pre,.listingblock>.content>pre{font-size:.90625em}}
@media screen and (min-width:1280px){.literalblock pre,.listingblock>.content>pre{font-size:1em}}
.literalblock pre,.listingblock>.content>pre:not(.highlight),.listingblock>.content>pre[class=highlight],.listingblock>.content>pre[class^="highlight "]{background:#f7f7f8}
.literalblock.output pre{color:#f7f7f8;background:rgba(0,0,0,.9)}
.listingblock>.content{position:relative}
.listingblock code[data-lang]::before{display:none;content:attr(data-lang);position:absolute;font-size:.75em;top:.425rem;right:.5rem;line-height:1;text-transform:uppercase;color:inherit;opacity:.5}
.listingblock:hover code[data-lang]::before{display:block}
.listingblock.terminal pre .command::before{content:attr(data-prompt);padding-right:.5em;color:inherit;opacity:.5}
.listingblock.terminal pre .command:not([data-prompt])::before{content:"$"}
.listingblock pre.highlightjs{padding:0}
.listingblock pre.highlightjs>code{padding:1em;border-radius:4px}
.listingblock pre.prettyprint{border-width:0}
.prettyprint{background:#f7f7f8}
pre.prettyprint .linenums{line-height:1.45;margin-left:2em}
pre.prettyprint li{background:none;list-style-type:inherit;padding-left:0}
pre.prettyprint li code[data-lang]::before{opacity:1}
pre.prettyprint li:not(:first-child) code[data-lang]::before{display:none}
table.linenotable{border-collapse:separate;border:0;margin-bottom:0;background:none}
table.linenotable td[class]{color:inherit;vertical-align:top;padding:0;line-height:inherit;white-space:normal}
table.linenotable td.code{padding-left:.75em}
table.linenotable td.linenos,pre.pygments .linenos{border-right:1px solid;opacity:.35;padding-right:.5em;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}
pre.pygments span.linenos{display:inline-block;margin-right:.75em}
.quoteblock{margin:0 1em 1.25em 1.5em;display:table}
.quoteblock:not(.excerpt)>.title{margin-left:-1.5em;margin-bottom:.75em}
.quoteblock blockquote,.quoteblock p{color:rgba(0,0,0,.85);font-size:1.15rem;line-height:1.75;word-spacing:.1em;letter-spacing:0;font-style:italic;text-align:justify}
.quoteblock blockquote{margin:0;padding:0;border:0}
.quoteblock blockquote::before{content:"\201c";float:left;font-size:2.75em;font-weight:bold;line-height:.6em;margin-left:-.6em;color:#7a2518;text-shadow:0 1px 2px rgba(0,0,0,.1)}
.quoteblock blockquote>.paragraph:last-child p{margin-bottom:0}
.quoteblock .attribution{margin-top:.75em;margin-right:.5ex;text-align:right}
.verseblock{margin:0 1em 1.25em}
.verseblock pre{font-family:"Open Sans","DejaVu Sans",sans-serif;font-size:1.15rem;color:rgba(0,0,0,.85);font-weight:300;text-rendering:optimizeLegibility}
.verseblock pre strong{font-weight:400}
.verseblock .attribution{margin-top:1.25rem;margin-left:.5ex}
.quoteblock .attribution,.verseblock .attribution{font-size:.9375em;line-height:1.45;font-style:italic}
.quoteblock .attribution br,.verseblock .attribution br{display:none}
.quoteblock .attribution cite,.verseblock .attribution cite{display:block;letter-spacing:-.025em;color:rgba(0,0,0,.6)}
.quoteblock.abstract blockquote::before,.quoteblock.excerpt blockquote::before,.quoteblock .quoteblock blockquote::before{display:none}
.quoteblock.abstract blockquote,.quoteblock.abstract p,.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{line-height:1.6;word-spacing:0}
.quoteblock.abstract{margin:0 1em 1.25em;display:block}
.quoteblock.abstract>.title{margin:0 0 .375em;font-size:1.15em;text-align:center}
.quoteblock.excerpt>blockquote,.quoteblock .quoteblock{padding:0 0 .25em 1em;border-left:.25em solid #dddddf}
.quoteblock.excerpt,.quoteblock .quoteblock{margin-left:0}
.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{color:inherit;font-size:1.0625rem}
.quoteblock.excerpt .attribution,.quoteblock .quoteblock .attribution{color:inherit;font-size:.85rem;text-align:left;margin-right:0}
p.tableblock:last-child{margin-bottom:0}
td.tableblock>.content{margin-bottom:1.25em;word-wrap:anywhere}
td.tableblock>.content>:last-child{margin-bottom:-1.25em}
table.tableblock,th.tableblock,td.tableblock{border:0 solid #dedede}
table.grid-all>*>tr>*{border-width:1px}
table.grid-cols>*>tr>*{border-width:0 1px}
table.grid-rows>*>tr>*{border-width:1px 0}
table.frame-all{border-width:1px}
table.frame-ends{border-width:1px 0}
table.frame-sides{border-width:0 1px}
table.frame-none>colgroup+*>:first-child>*,table.frame-sides>colgroup+*>:first-child>*{border-top-width:0}
table.frame-none>:last-child>:last-child>*,table.frame-sides>:last-child>:last-child>*{border-bottom-width:0}
table.frame-none>*>tr>:first-child,table.frame-ends>*>tr>:first-child{border-left-width:0}
table.frame-none>*>tr>:last-child,table.frame-ends>*>tr>:last-child{border-right-width:0}
table.stripes-all>*>tr,table.stripes-odd>*>tr:nth-of-type(odd),table.stripes-even>*>tr:nth-of-type(even),table.stripes-hover>*>tr:hover{background:#f8f8f7}
th.halign-left,td.halign-left{text-align:left}
th.halign-right,td.halign-right{text-align:right}
th.halign-center,td.halign-center{text-align:center}
th.valign-top,td.valign-top{vertical-align:top}
th.valign-bottom,td.valign-bottom{vertical-align:bottom}
th.valign-middle,td.valign-middle{vertical-align:middle}
table thead th,table tfoot th{font-weight:bold}
tbody tr th{background:#f7f8f7}
tbody tr th,tbody tr th p,tfoot tr th,tfoot tr th p{color:rgba(0,0,0,.8);font-weight:bold}
p.tableblock>code:only-child{background:none;padding:0}
p.tableblock{font-size:1em}
ol{margin-left:1.75em}
ul li ol{margin-left:1.5em}
dl dd{margin-left:1.125em}
dl dd:last-child,dl dd:last-child>:last-child{margin-bottom:0}
li p,ul dd,ol dd,.olist .olist,.ulist .ulist,.ulist .olist,.olist .ulist{margin-bottom:.625em}
ul.checklist,ul.none,ol.none,ul.no-bullet,ol.no-bullet,ol.unnumbered,ul.unstyled,ol.unstyled{list-style-type:none}
ul.no-bullet,ol.no-bullet,ol.unnumbered{margin-left:.625em}
ul.unstyled,ol.unstyled{margin-left:0}
li>p:empty:only-child::before{content:"";display:inline-block}
ul.checklist>li>p:first-child{margin-left:-1em}
ul.checklist>li>p:first-child>.fa-square-o:first-child,ul.checklist>li>p:first-child>.fa-check-square-o:first-child{width:1.25em;font-size:.8em;position:relative;bottom:.125em}
ul.checklist>li>p:first-child>input[type=checkbox]:first-child{margin-right:.25em}
ul.inline{display:flex;flex-flow:row wrap;list-style:none;margin:0 0 .625em -1.25em}
ul.inline>li{margin-left:1.25em}
.unstyled dl dt{font-weight:400;font-style:normal}
ol.arabic{list-style-type:decimal}
ol.decimal{list-style-type:decimal-leading-zero}
ol.loweralpha{list-style-type:lower-alpha}
ol.upperalpha{list-style-type:upper-alpha}
ol.lowerroman{list-style-type:lower-roman}
ol.upperroman{list-style-type:upper-roman}
ol.lowergreek{list-style-type:lower-greek}
.hdlist>table,.colist>table{border:0;background:none}
.hdlist>table>tbody>tr,.colist>table>tbody>tr{background:none}
td.hdlist1,td.hdlist2{vertical-align:top;padding:0 .625em}
td.hdlist1{font-weight:bold;padding-bottom:1.25em}
td.hdlist2{word-wrap:anywhere}
.literalblock+.colist,.listingblock+.colist{margin-top:-.5em}
.colist td:not([class]):first-child{padding:.4em .75em 0;line-height:1;vertical-align:top}
.colist td:not([class]):first-child img{max-width:none}
.colist td:not([class]):last-child{padding:.25em 0}
.thumb,.th{line-height:0;display:inline-block;border:4px solid #fff;box-shadow:0 0 0 1px #ddd}
.imageblock.left{margin:.25em .625em 1.25em 0}
.imageblock.right{margin:.25em 0 1.25em .625em}
.imageblock>.title{margin-bottom:0}
.imageblock.thumb,.imageblock.th{border-width:6px}
.imageblock.thumb>.title,.imageblock.th>.title{padding:0 .125em}
.image.left,.image.right{margin-top:.25em;margin-bottom:.25em;display:inline-block;line-height:0}
.image.left{margin-right:.625em}
.image.right{margin-left:.625em}
a.image{text-decoration:none;display:inline-block}
a.image object{pointer-events:none}
sup.footnote,sup.footnoteref{font-size:.875em;position:static;vertical-align:super}
sup.footnote a,sup.footnoteref a{text-decoration:none}
sup.footnote a:active,sup.footnoteref a:active,#footnotes .footnote a:first-of-type:active{text-decoration:underline}
#footnotes{padding-top:.75em;padding-bottom:.75em;margin-bottom:.625em}
#footnotes hr{width:20%;min-width:6.25em;margin:-.25em 0 .75em;border-width:1px 0 0}
#footnotes .footnote{padding:0 .375em 0 .225em;line-height:1.3334;font-size:.875em;margin-left:1.2em;margin-bottom:.2em}
#footnotes .footnote a:first-of-type{font-weight:bold;text-decoration:none;margin-left:-1.05em}
#footnotes .footnote:last-of-type{margin-bottom:0}
#content #footnotes{margin-top:-.625em;margin-bottom:0;padding:.75em 0}
div.unbreakable{page-break-inside:avoid}
.big{font-size:larger}
.small{font-size:smaller}
.underline{text-decoration:underline}
.overline{text-decoration:overline}
.line-through{text-decoration:line-through}
.aqua{color:#00bfbf}
.aqua-background{background:#00fafa}
.black{color:#000}
.black-background{background:#000}
.blue{color:#0000bf}
.blue-background{background:#0000fa}
.fuchsia{color:#bf00bf}
.fuchsia-background{background:#fa00fa}
.gray{color:#606060}
.gray-background{background:#7d7d7d}
.green{color:#006000}
.green-background{background:#007d00}
.lime{color:#00bf00}
.lime-background{background:#00fa00}
.maroon{color:#600000}
.maroon-background{background:#7d0000}
.navy{color:#000060}
.navy-background{background:#00007d}
.olive{color:#606000}
.olive-background{background:#7d7d00}
.purple{color:#600060}
.purple-background{background:#7d007d}
.red{color:#bf0000}
.red-background{background:#fa0000}
.silver{color:#909090}
.silver-background{background:#bcbcbc}
.teal{color:#006060}
.teal-background{background:#007d7d}
.white{color:#bfbfbf}
.white-background{background:#fafafa}
.yellow{color:#bfbf00}
.yellow-background{background:#fafa00}
span.icon>.fa{cursor:default}
a span.icon>.fa{cursor:inherit}
.admonitionblock td.icon [class^="fa icon-"]{font-size:2.5em;text-shadow:1px 1px 2px rgba(0,0,0,.5);cursor:default}
.admonitionblock td.icon .icon-note::before{content:"\f05a";color:#19407c}
.admonitionblock td.icon .icon-tip::before{content:"\f0eb";text-shadow:1px 1px 2px rgba(155,155,0,.8);color:#111}
.admonitionblock td.icon .icon-warning::before{content:"\f071";color:#bf6900}
.admonitionblock td.icon .icon-caution::before{content:"\f06d";color:#bf3400}
.admonitionblock td.icon .icon-important::before{content:"\f06a";color:#bf0000}
.conum[data-value]{display:inline-block;color:#fff!important;background:rgba(0,0,0,.8);border-radius:50%;text-align:center;font-size:.75em;width:1.67em;height:1.67em;line-height:1.67em;font-family:"Open Sans","DejaVu Sans",sans-serif;font-style:normal;font-weight:bold}
.conum[data-value] *{color:#fff!important}
.conum[data-value]+b{display:none}
.conum[data-value]::after{content:attr(data-value)}
pre .conum[data-value]{position:relative;top:-.125em}
b.conum *{color:inherit!important}
.conum:not([data-value]):empty{display:none}
dt,th.tableblock,td.content,div.footnote{text-rendering:optimizeLegibility}
h1,h2,p,td.content,span.alt,summary{letter-spacing:-.01em}
p strong,td.content strong,div.footnote strong{letter-spacing:-.005em}
p,blockquote,dt,td.content,td.hdlist1,span.alt,summary{font-size:1.0625rem}
p{margin-bottom:1.25rem}
.sidebarblock p,.sidebarblock dt,.sidebarblock td.content,p.tableblock{font-size:1em}
.exampleblock>.content{background:#fffef7;border-color:#e0e0dc;box-shadow:0 1px 4px #e0e0dc}
.print-only{display:none!important}
@page{margin:1.25cm .75cm}
@media print{*{box-shadow:none!important;text-shadow:none!important}
html{font-size:80%}
a{color:inherit!important;text-decoration:underline!important}
a.bare,a[href^="#"],a[href^="mailto:"]{text-decoration:none!important}
a[href^="http:"]:not(.bare)::after,a[href^="https:"]:not(.bare)::after{content:"(" attr(href) ")";display:inline-block;font-size:.875em;padding-left:.25em}
abbr[title]{border-bottom:1px dotted}
abbr[title]::after{content:" (" attr(title) ")"}
pre,blockquote,tr,img,object,svg{page-break-inside:avoid}
thead{display:table-header-group}
svg{max-width:100%}
p,blockquote,dt,td.content{font-size:1em;orphans:3;widows:3}
h2,h3,#toctitle,.sidebarblock>.content>.title{page-break-after:avoid}
#header,#content,#footnotes,#footer{max-width:none}
#toc,.sidebarblock,.exampleblock>.content{background:none!important}
#toc{border-bottom:1px solid #dddddf!important;padding-bottom:0!important}
body.book #header{text-align:center}
body.book #header>h1:first-child{border:0!important;margin:2.5em 0 1em}
body.book #header .details{border:0!important;display:block;padding:0!important}
body.book #header .details span:first-child{margin-left:0!important}
body.book #header .details br{display:block}
body.book #header .details br+span::before{content:none!important}
body.book #toc{border:0!important;text-align:left!important;padding:0!important;margin:0!important}
body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-break-before:always}
.listingblock code[data-lang]::before{display:block}
#footer{padding:0 .9375em}
.hide-on-print{display:none!important}
.print-only{display:block!important}
.hide-for-print{display:none!important}
.show-for-print{display:inherit!important}}
@media amzn-kf8,print{#header>h1:first-child{margin-top:1.25rem}
.sect1{padding:0!important}
.sect1+.sect1{border:0}
#footer{background:none}
#footer-text{color:rgba(0,0,0,.6);font-size:.9em}}
@media amzn-kf8{#header,#content,#footnotes,#footer{padding:0}}
</style>
</head>
<body class="article">
<div id="header">
<h1>Unit Testing</h1>
</div>
<div id="content">
<div id="preamble">
<div class="sectionbody">
<div class="paragraph">
<p>In our current testing environment, we spend a significant amount of effort
crafting end-to-end tests for error conditions that could easily be captured by
unit tests (or we simply forgo some hard-to-setup and rare error conditions).
Unit tests additionally provide stability to the codebase and can simplify
debugging through isolation. Writing unit tests in pure C, rather than with our
current shell/test-tool helper setup, simplifies test setup, simplifies passing
data around (no shell-isms required), and reduces testing runtime by not
spawning a separate process for every test invocation.</p>
</div>
<div class="paragraph">
<p>We believe that a large body of unit tests, living alongside the existing test
suite, will improve code quality for the Git project.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_definitions">Definitions</h2>
<div class="sectionbody">
<div class="paragraph">
<p>For the purposes of this document, we&#8217;ll use <strong>test framework</strong> to refer to
projects that support writing test cases and running tests within the context
of a single executable. <strong>Test harness</strong> will refer to projects that manage
running multiple executables (each of which may contain multiple test cases) and
aggregating their results.</p>
</div>
<div class="paragraph">
<p>In reality, these terms are not strictly defined, and many of the projects
discussed below contain features from both categories.</p>
</div>
<div class="paragraph">
<p>For now, we will evaluate projects solely on their framework features. Since we
are relying on having TAP output (see below), we can assume that any framework
can be made to work with a harness that we can choose later.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_summary">Summary</h2>
<div class="sectionbody">
<div class="paragraph">
<p>We believe the best way forward is to implement a custom TAP framework for the
Git project. We use a version of the framework originally proposed in
<a href="https://lore.kernel.org/git/c902a166-98ce-afba-93f2-ea6027557176@gmail.com/">1</a>.</p>
</div>
<div class="paragraph">
<p>See the <a href="#framework-selection">Framework Selection</a> section below for the
rationale behind this decision.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_choosing_a_test_harness">Choosing a test harness</h2>
<div class="sectionbody">
<div class="paragraph">
<p>During upstream discussion, it was occasionally noted that <code>prove</code> provides many
convenient features, such as scheduling slower tests first, or re-running
previously failed tests.</p>
</div>
<div class="paragraph">
<p>While we already support the use of <code>prove</code> as a test harness for the shell
tests, it is not strictly required. The t/Makefile allows running shell tests
directly (though with interleaved output if parallelism is enabled). Git
developers who wish to use <code>prove</code> as a more advanced harness can do so by
setting DEFAULT_TEST_TARGET=prove in their config.mak.</p>
</div>
<div class="paragraph">
<p>We will follow a similar approach for unit tests: by default the test
executables will be run directly from the t/Makefile, but <code>prove</code> can be
configured with DEFAULT_UNIT_TEST_TARGET=prove.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="framework-selection">Framework selection</h2>
<div class="sectionbody">
<div class="paragraph">
<p>There are a variety of features we can use to rank the candidate frameworks, and
those features have different priorities:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Critical features: we probably won&#8217;t consider a framework without these</p>
<div class="ulist">
<ul>
<li>
<p>Can we legally / easily use the project?</p>
<div class="ulist">
<ul>
<li>
<p><a href="#license">License</a></p>
</li>
<li>
<p><a href="#vendorable-or-ubiquitous">Vendorable or ubiquitous</a></p>
</li>
<li>
<p><a href="#maintainable-extensible">Maintainable / extensible</a></p>
</li>
<li>
<p><a href="#major-platform-support">Major platform support</a></p>
</li>
</ul>
</div>
</li>
<li>
<p>Does the project support our bare-minimum needs?</p>
<div class="ulist">
<ul>
<li>
<p><a href="#tap-support">TAP support</a></p>
</li>
<li>
<p><a href="#diagnostic-output">Diagnostic output</a></p>
</li>
<li>
<p><a href="#runtime-skippable-tests">Runtime-skippable tests</a></p>
</li>
</ul>
</div>
</li>
</ul>
</div>
</li>
<li>
<p>Nice-to-have features:</p>
<div class="ulist">
<ul>
<li>
<p><a href="#parallel-execution">Parallel execution</a></p>
</li>
<li>
<p><a href="#mock-support">Mock support</a></p>
</li>
<li>
<p><a href="#signal-error-handling">Signal &amp; error-handling</a></p>
</li>
</ul>
</div>
</li>
<li>
<p>Tie-breaker stats</p>
<div class="ulist">
<ul>
<li>
<p><a href="#project-kloc">Project KLOC</a></p>
</li>
<li>
<p><a href="#adoption">Adoption</a></p>
</li>
</ul>
</div>
</li>
</ul>
</div>
<div class="sect2">
<h3 id="license">License</h3>
<div class="paragraph">
<p>We must be able to legally use the framework in connection with Git. As Git is
licensed only under GPLv2, we must eliminate any LGPLv3, GPLv3, or Apache 2.0
projects.</p>
</div>
</div>
<div class="sect2">
<h3 id="vendorable-or-ubiquitous">Vendorable or ubiquitous</h3>
<div class="paragraph">
<p>We want to avoid forcing Git developers to install new tools just to run unit
tests. Any prospective frameworks and harnesses must either be vendorable
(meaning, we can copy their source directly into Git&#8217;s repository), or so
ubiquitous that it is reasonable to expect that most developers will have the
tools installed already.</p>
</div>
</div>
<div class="sect2">
<h3 id="maintainable-extensible">Maintainable / extensible</h3>
<div class="paragraph">
<p>It is unlikely that any pre-existing project perfectly fits our needs, so any
project we select will need to be actively maintained and open to accepting
changes. Alternatively, assuming we are vendoring the source into our repo, it
must be simple enough that Git developers can feel comfortable making changes as
needed to our version.</p>
</div>
<div class="paragraph">
<p>In the comparison table below, "True" means that the framework seems to have
active developers, that it is simple enough that Git developers can make changes
to it, and that the project seems open to accepting external contributions (or
that it is vendorable). "Partial" means that at least one of the above
conditions holds.</p>
</div>
</div>
<div class="sect2">
<h3 id="major-platform-support">Major platform support</h3>
<div class="paragraph">
<p>At a bare minimum, unit-testing must work on Linux, MacOS, and Windows.</p>
</div>
<div class="paragraph">
<p>In the comparison table below, "True" means that it works on all three major
platforms with no issues. "Partial" means that there may be annoyances on one or
more platforms, but it is still usable in principle.</p>
</div>
</div>
<div class="sect2">
<h3 id="tap-support">TAP support</h3>
<div class="paragraph">
<p>The <a href="https://testanything.org/">Test Anything Protocol</a> is a text-based interface
that allows tests to communicate with a test harness. It is already used by
Git&#8217;s integration test suite. Supporting TAP output is a mandatory feature for
any prospective test framework.</p>
</div>
<div class="paragraph">
<p>In the comparison table below, "True" means this is natively supported.
"Partial" means TAP output must be generated by post-processing the native
output.</p>
</div>
<div class="paragraph">
<p>Frameworks that do not have at least Partial support will not be evaluated
further.</p>
</div>
</div>
<div class="sect2">
<h3 id="diagnostic-output">Diagnostic output</h3>
<div class="paragraph">
<p>When a test case fails, the framework must generate enough diagnostic output to
help developers find the appropriate test case in source code in order to debug
the failure.</p>
</div>
</div>
<div class="sect2">
<h3 id="runtime-skippable-tests">Runtime-skippable tests</h3>
<div class="paragraph">
<p>Test authors may wish to skip certain test cases based on runtime circumstances,
so the framework should support this.</p>
</div>
</div>
<div class="sect2">
<h3 id="parallel-execution">Parallel execution</h3>
<div class="paragraph">
<p>Ideally, we will build up a significant collection of unit test cases, most
likely split across multiple executables. It will be necessary to run these
tests in parallel to enable fast develop-test-debug cycles.</p>
</div>
<div class="paragraph">
<p>In the comparison table below, "True" means that individual test cases within a
single test executable can be run in parallel. We assume that executable-level
parallelism can be handled by the test harness.</p>
</div>
</div>
<div class="sect2">
<h3 id="mock-support">Mock support</h3>
<div class="paragraph">
<p>Unit test authors may wish to test code that interacts with objects that may be
inconvenient to handle in a test (e.g. interacting with a network service).
Mocking allows test authors to provide a fake implementation of these objects
for more convenient tests.</p>
</div>
</div>
<div class="sect2">
<h3 id="signal-error-handling">Signal &amp; error handling</h3>
<div class="paragraph">
<p>The test framework should fail gracefully when test cases are themselves buggy
or when they are interrupted by signals during runtime.</p>
</div>
</div>
<div class="sect2">
<h3 id="project-kloc">Project KLOC</h3>
<div class="paragraph">
<p>The size of the project, in thousands of lines of code as measured by
<a href="https://dwheeler.com/sloccount/">sloccount</a> (rounded up to the next multiple of
1,000). As a tie-breaker, we probably prefer a project with fewer LOC.</p>
</div>
</div>
<div class="sect2">
<h3 id="adoption">Adoption</h3>
<div class="paragraph">
<p>As a tie-breaker, we prefer a more widely-used project. We use the number of
GitHub / GitLab stars to estimate this.</p>
</div>
</div>
<div class="sect2">
<h3 id="_comparison">Comparison</h3>
<table class="tableblock frame-all grid-all" style="width: 33%;">
<colgroup>
<col style="width: 7.6923%;"/>
<col style="width: 7.6923%;"/>
<col style="width: 7.6923%;"/>
<col style="width: 7.6923%;"/>
<col style="width: 7.6923%;"/>
<col style="width: 7.6923%;"/>
<col style="width: 7.6923%;"/>
<col style="width: 7.6923%;"/>
<col style="width: 7.6923%;"/>
<col style="width: 7.6923%;"/>
<col style="width: 7.6923%;"/>
<col style="width: 7.6923%;"/>
<col style="width: 7.6924%;"/>
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Framework</th>
<th class="tableblock halign-left valign-top"><a href="#license">License</a></th>
<th class="tableblock halign-left valign-top"><a href="#vendorable-or-ubiquitous">Vendorable or ubiquitous</a></th>
<th class="tableblock halign-left valign-top"><a href="#maintainable-extensible">Maintainable / extensible</a></th>
<th class="tableblock halign-left valign-top"><a href="#major-platform-support">Major platform support</a></th>
<th class="tableblock halign-left valign-top"><a href="#tap-support">TAP support</a></th>
<th class="tableblock halign-left valign-top"><a href="#diagnostic-output">Diagnostic output</a></th>
<th class="tableblock halign-left valign-top"><a href="#runtime&#8212;&#8203;skippable-tests">Runtime- skippable tests</a></th>
<th class="tableblock halign-left valign-top"><a href="#parallel-execution">Parallel execution</a></th>
<th class="tableblock halign-left valign-top"><a href="#mock-support">Mock support</a></th>
<th class="tableblock halign-left valign-top"><a href="#signal-error-handling">Signal &amp; error handling</a></th>
<th class="tableblock halign-left valign-top"><a href="#project-kloc">Project KLOC</a></th>
<th class="tableblock halign-left valign-top"><a href="#adoption">Adoption</a></th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><a href="https://lore.kernel.org/git/c902a166-98ce-afba-93f2-ea6027557176@gmail.com/">Custom Git impl.</a></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#GPL v2#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#True#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#True#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#True#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#True#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#True#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#True#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[red-background]#False#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[red-background]#False#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[red-background]#False#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">1</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">0</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><a href="https://github.com/silentbicycle/greatest">Greatest</a></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#ISC#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#True#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[yellow-background]#Partial#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#True#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[yellow-background]#Partial#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#True#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#True#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[red-background]#False#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[red-background]#False#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[red-background]#False#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">3</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">1400</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><a href="https://github.com/Snaipe/Criterion">Criterion</a></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#MIT#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[red-background]#False#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[yellow-background]#Partial#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#True#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#True#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#True#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#True#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#True#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[red-background]#False#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#True#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">19</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">1800</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><a href="https://github.com/rra/c-tap-harness/">C TAP</a></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#Expat#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#True#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[yellow-background]#Partial#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[yellow-background]#Partial#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#True#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[red-background]#False#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#True#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[red-background]#False#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[red-background]#False#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[red-background]#False#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">4</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">33</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><a href="https://libcheck.github.io/check/">Check</a></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#LGPL v2.1#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[red-background]#False#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[yellow-background]#Partial#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#True#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#True#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#True#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[red-background]#False#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[red-background]#False#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[red-background]#False#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#True#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">17</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">973</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><a href="https://github.com/clar-test/clar">Clar</a></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#ISC#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[red-background]#False#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[yellow-background]#Partial#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#True#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#True#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#True#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#True#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[red-background]#False#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[red-background]#False#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">[lime-background]#True#</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">1</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">192</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="_additional_framework_candidates">Additional framework candidates</h3>
<div class="paragraph">
<p>Several suggested frameworks have been eliminated from consideration:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Incompatible licenses:</p>
<div class="ulist">
<ul>
<li>
<p><a href="https://github.com/zorgnax/libtap">libtap</a> (LGPL v3)</p>
</li>
<li>
<p><a href="https://cmocka.org/">cmocka</a> (Apache 2.0)</p>
</li>
</ul>
</div>
</li>
<li>
<p>Missing source: <a href="https://www.kindahl.net/mytap/doc/index.html">MyTap</a></p>
</li>
<li>
<p>No TAP support:</p>
<div class="ulist">
<ul>
<li>
<p><a href="https://nemequ.github.io/munit/">µnit</a></p>
</li>
<li>
<p><a href="https://github.com/google/cmockery">cmockery</a></p>
</li>
<li>
<p><a href="https://github.com/lpabon/cmockery2">cmockery2</a></p>
</li>
<li>
<p><a href="https://github.com/ThrowTheSwitch/Unity">Unity</a></p>
</li>
<li>
<p><a href="https://github.com/siu/minunit">minunit</a></p>
</li>
<li>
<p><a href="https://cunit.sourceforge.net/">CUnit</a></p>
</li>
</ul>
</div>
</li>
</ul>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_milestones">Milestones</h2>
<div class="sectionbody">
<div class="ulist">
<ul>
<li>
<p>Add useful tests of library-like code</p>
</li>
<li>
<p>Integrate with
<a href="https://lore.kernel.org/git/20230502211454.1673000-1-calvinwan@google.com/">stdlib
work</a></p>
</li>
<li>
<p>Run alongside regular <code>make</code> <code>test</code> target</p>
</li>
</ul>
</div>
</div>
</div>
</div>
<div id="footer">
<div id="footer-text">
Last updated 2025-03-17 07:09:38 UTC
</div>
</div>
</body>
</html>