github
/
git-sizer
зеркало из
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
git-sizer/sizes/sizes.go

305 строки
10 KiB
Go
Исходник Постоянная ссылка Обычный вид История

Implement the main command, `git-sizer` It can compute various dimensions about a tree object in a Git repository, in a way that is robust against git bombs. For now it must be passed the path to a Git repository and one or more tree object specifications (like `HEAD^{tree}` or `HEAD:src`). That is likely to change. To avoid starting up lots of processes, we use `git cat-file --batch` to read trees. But for blobs we only need to know the size, so use `git cat-file --batch-check` for blobs. This unfortunately means that we need to keep two `git` processes running, which means paying the git startup time twice.
2017-10-14 11:00:38 +03:00
package sizes
import (
"fmt"
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
"github.com/github/git-sizer/counts"
Separate out a `git` package
2018-02-26 08:30:45 +03:00
"github.com/github/git-sizer/git"
Implement the main command, `git-sizer` It can compute various dimensions about a tree object in a Git repository, in a way that is robust against git bombs. For now it must be passed the path to a Git repository and one or more tree object specifications (like `HEAD^{tree}` or `HEAD:src`). That is likely to change. To avoid starting up lots of processes, we use `git cat-file --batch` to read trees. But for blobs we only need to know the size, so use `git cat-file --batch-check` for blobs. This unfortunately means that we need to keep two `git` processes running, which means paying the git startup time twice.
2017-10-14 11:00:38 +03:00
)
type Size interface {
fmt.Stringer
}
Change `BlobSize` into a struct
2017-10-29 11:07:11 +03:00
type BlobSize struct {
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
Size counts.Count32
Change `BlobSize` into a struct
2017-10-29 11:07:11 +03:00
}
Implement the main command, `git-sizer` It can compute various dimensions about a tree object in a Git repository, in a way that is robust against git bombs. For now it must be passed the path to a Git repository and one or more tree object specifications (like `HEAD^{tree}` or `HEAD:src`). That is likely to change. To avoid starting up lots of processes, we use `git cat-file --batch` to read trees. But for blobs we only need to know the size, so use `git cat-file --batch-check` for blobs. This unfortunately means that we need to keep two `git` processes running, which means paying the git startup time twice.
2017-10-14 11:00:38 +03:00
type TreeSize struct {
Rename a bunch of things to make way for more things
2017-10-29 07:22:38 +03:00
// The maximum depth of trees and blobs starting at this object
Don't include the root tree in `MaxPathDepth` It is more natural to define the length of a relative path `foo/bar/baz.txt` as 3 rather than 4, so tweak the code to do so.
2018-03-11 12:50:20 +03:00
// (not including this object).
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
MaxPathDepth counts.Count32 `json:"max_path_depth"`
Implement the main command, `git-sizer` It can compute various dimensions about a tree object in a Git repository, in a way that is robust against git bombs. For now it must be passed the path to a Git repository and one or more tree object specifications (like `HEAD^{tree}` or `HEAD:src`). That is likely to change. To avoid starting up lots of processes, we use `git cat-file --batch` to read trees. But for blobs we only need to know the size, so use `git cat-file --batch-check` for blobs. This unfortunately means that we need to keep two `git` processes running, which means paying the git startup time twice.
2017-10-14 11:00:38 +03:00
Rename a bunch of things to make way for more things
2017-10-29 07:22:38 +03:00
// The maximum length of any path relative to this object, in
// characters.
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
MaxPathLength counts.Count32 `json:"max_path_length"`
Implement the main command, `git-sizer` It can compute various dimensions about a tree object in a Git repository, in a way that is robust against git bombs. For now it must be passed the path to a Git repository and one or more tree object specifications (like `HEAD^{tree}` or `HEAD:src`). That is likely to change. To avoid starting up lots of processes, we use `git cat-file --batch` to read trees. But for blobs we only need to know the size, so use `git cat-file --batch-check` for blobs. This unfortunately means that we need to keep two `git` processes running, which means paying the git startup time twice.
2017-10-14 11:00:38 +03:00
Distinguish two `Count` sizes: `Count32` and `Count64` This is mostly interesting because it shrinks `TreeSize` from 56 bytes to 32 bytes, and we might have a lot of these.
2017-10-30 09:56:23 +03:00
// The total number of trees, including duplicates.
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
ExpandedTreeCount counts.Count32 `json:"expanded_tree_count"`
Implement the main command, `git-sizer` It can compute various dimensions about a tree object in a Git repository, in a way that is robust against git bombs. For now it must be passed the path to a Git repository and one or more tree object specifications (like `HEAD^{tree}` or `HEAD:src`). That is likely to change. To avoid starting up lots of processes, we use `git cat-file --batch` to read trees. But for blobs we only need to know the size, so use `git cat-file --batch-check` for blobs. This unfortunately means that we need to keep two `git` processes running, which means paying the git startup time twice.
2017-10-14 11:00:38 +03:00
Distinguish two `Count` sizes: `Count32` and `Count64` This is mostly interesting because it shrinks `TreeSize` from 56 bytes to 32 bytes, and we might have a lot of these.
2017-10-30 09:56:23 +03:00
// The total number of blobs, including duplicates.
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
ExpandedBlobCount counts.Count32 `json:"expanded_blob_count"`
Implement the main command, `git-sizer` It can compute various dimensions about a tree object in a Git repository, in a way that is robust against git bombs. For now it must be passed the path to a Git repository and one or more tree object specifications (like `HEAD^{tree}` or `HEAD:src`). That is likely to change. To avoid starting up lots of processes, we use `git cat-file --batch` to read trees. But for blobs we only need to know the size, so use `git cat-file --batch-check` for blobs. This unfortunately means that we need to keep two `git` processes running, which means paying the git startup time twice.
2017-10-14 11:00:38 +03:00
Distinguish two `Count` sizes: `Count32` and `Count64` This is mostly interesting because it shrinks `TreeSize` from 56 bytes to 32 bytes, and we might have a lot of these.
2017-10-30 09:56:23 +03:00
// The total size of all blobs, including duplicates.
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
ExpandedBlobSize counts.Count64 `json:"expanded_blob_size"`
Implement the main command, `git-sizer` It can compute various dimensions about a tree object in a Git repository, in a way that is robust against git bombs. For now it must be passed the path to a Git repository and one or more tree object specifications (like `HEAD^{tree}` or `HEAD:src`). That is likely to change. To avoid starting up lots of processes, we use `git cat-file --batch` to read trees. But for blobs we only need to know the size, so use `git cat-file --batch-check` for blobs. This unfortunately means that we need to keep two `git` processes running, which means paying the git startup time twice.
2017-10-14 11:00:38 +03:00
Distinguish two `Count` sizes: `Count32` and `Count64` This is mostly interesting because it shrinks `TreeSize` from 56 bytes to 32 bytes, and we might have a lot of these.
2017-10-30 09:56:23 +03:00
// The total number of symbolic links, including duplicates.
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
ExpandedLinkCount counts.Count32 `json:"expanded_link_count"`
Implement the main command, `git-sizer` It can compute various dimensions about a tree object in a Git repository, in a way that is robust against git bombs. For now it must be passed the path to a Git repository and one or more tree object specifications (like `HEAD^{tree}` or `HEAD:src`). That is likely to change. To avoid starting up lots of processes, we use `git cat-file --batch` to read trees. But for blobs we only need to know the size, so use `git cat-file --batch-check` for blobs. This unfortunately means that we need to keep two `git` processes running, which means paying the git startup time twice.
2017-10-14 11:00:38 +03:00
Distinguish two `Count` sizes: `Count32` and `Count64` This is mostly interesting because it shrinks `TreeSize` from 56 bytes to 32 bytes, and we might have a lot of these.
2017-10-30 09:56:23 +03:00
// The total number of submodules referenced, including duplicates.
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
ExpandedSubmoduleCount counts.Count32 `json:"expanded_submodule_count"`
Implement the main command, `git-sizer` It can compute various dimensions about a tree object in a Git repository, in a way that is robust against git bombs. For now it must be passed the path to a Git repository and one or more tree object specifications (like `HEAD^{tree}` or `HEAD:src`). That is likely to change. To avoid starting up lots of processes, we use `git cat-file --batch` to read trees. But for blobs we only need to know the size, so use `git cat-file --batch-check` for blobs. This unfortunately means that we need to keep two `git` processes running, which means paying the git startup time twice.
2017-10-14 11:00:38 +03:00
}
func (s *TreeSize) addDescendent(filename string, s2 TreeSize) {
Don't include the root tree in `MaxPathDepth` It is more natural to define the length of a relative path `foo/bar/baz.txt` as 3 rather than 4, so tweak the code to do so.
2018-03-11 12:50:20 +03:00
s.MaxPathDepth.AdjustMaxIfNecessary(s2.MaxPathDepth.Plus(1))
Implement the main command, `git-sizer` It can compute various dimensions about a tree object in a Git repository, in a way that is robust against git bombs. For now it must be passed the path to a Git repository and one or more tree object specifications (like `HEAD^{tree}` or `HEAD:src`). That is likely to change. To avoid starting up lots of processes, we use `git cat-file --batch` to read trees. But for blobs we only need to know the size, so use `git cat-file --batch-check` for blobs. This unfortunately means that we need to keep two `git` processes running, which means paying the git startup time twice.
2017-10-14 11:00:38 +03:00
if s2.MaxPathLength > 0 {
Systematically prefer to cite more recent objects Given two objects that are the same size, or multiple trees/commits that refer to the same large object, which one should we prefer to cite in the footnotes? I think that new objects are more interesting than old objects [1], because * It is interesting to know whether the "bad" objects are still in use. * Once you have a commit containing a "bad" object, it is easier to trace it backwards in time than forwards. * New objects are more likely to be at the tips of branches and thus more likely to have a reference pointing at them. The old code was somewhat inconsistent because blobs and trees are processed in roughly reverse-chronological order whereas commits are processed in roughly chronological order. Improve the situation by implementing two pairs of `Count.AdjustMax()` methods, `AdjustMaxIfNecessary()` and `AdjustMaxIfPossible()`, differing in the obvious way if the two values that they are comparing are equal. By choosing which variant to use in which situation, we can arrange to preferentially pick the new of equal-sized objects. Note that this still doesn't cause newer commits to be preferred in the paths of trees and blobs. [1] Note that there are also arguments that the old objects are more interesting, because they are more likely to indicate the commits where a large object was first introduced.
2017-12-11 19:05:52 +03:00
s.MaxPathLength.AdjustMaxIfNecessary(
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
(counts.NewCount32(uint64(len(filename))) + 1).Plus(s2.MaxPathLength),
Systematically prefer to cite more recent objects Given two objects that are the same size, or multiple trees/commits that refer to the same large object, which one should we prefer to cite in the footnotes? I think that new objects are more interesting than old objects [1], because * It is interesting to know whether the "bad" objects are still in use. * Once you have a commit containing a "bad" object, it is easier to trace it backwards in time than forwards. * New objects are more likely to be at the tips of branches and thus more likely to have a reference pointing at them. The old code was somewhat inconsistent because blobs and trees are processed in roughly reverse-chronological order whereas commits are processed in roughly chronological order. Improve the situation by implementing two pairs of `Count.AdjustMax()` methods, `AdjustMaxIfNecessary()` and `AdjustMaxIfPossible()`, differing in the obvious way if the two values that they are comparing are equal. By choosing which variant to use in which situation, we can arrange to preferentially pick the new of equal-sized objects. Note that this still doesn't cause newer commits to be preferred in the paths of trees and blobs. [1] Note that there are also arguments that the old objects are more interesting, because they are more likely to indicate the commits where a large object was first introduced.
2017-12-11 19:05:52 +03:00
)
Implement the main command, `git-sizer` It can compute various dimensions about a tree object in a Git repository, in a way that is robust against git bombs. For now it must be passed the path to a Git repository and one or more tree object specifications (like `HEAD^{tree}` or `HEAD:src`). That is likely to change. To avoid starting up lots of processes, we use `git cat-file --batch` to read trees. But for blobs we only need to know the size, so use `git cat-file --batch-check` for blobs. This unfortunately means that we need to keep two `git` processes running, which means paying the git startup time twice.
2017-10-14 11:00:38 +03:00
} else {
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
s.MaxPathLength.AdjustMaxIfNecessary(counts.NewCount32(uint64(len(filename))))
Implement the main command, `git-sizer` It can compute various dimensions about a tree object in a Git repository, in a way that is robust against git bombs. For now it must be passed the path to a Git repository and one or more tree object specifications (like `HEAD^{tree}` or `HEAD:src`). That is likely to change. To avoid starting up lots of processes, we use `git cat-file --batch` to read trees. But for blobs we only need to know the size, so use `git cat-file --batch-check` for blobs. This unfortunately means that we need to keep two `git` processes running, which means paying the git startup time twice.
2017-10-14 11:00:38 +03:00
}
Extract a method `Count.Increment()`
2017-10-29 09:42:40 +03:00
s.ExpandedTreeCount.Increment(s2.ExpandedTreeCount)
s.ExpandedBlobCount.Increment(s2.ExpandedBlobCount)
s.ExpandedBlobSize.Increment(s2.ExpandedBlobSize)
s.ExpandedLinkCount.Increment(s2.ExpandedLinkCount)
s.ExpandedSubmoduleCount.Increment(s2.ExpandedSubmoduleCount)
Implement the main command, `git-sizer` It can compute various dimensions about a tree object in a Git repository, in a way that is robust against git bombs. For now it must be passed the path to a Git repository and one or more tree object specifications (like `HEAD^{tree}` or `HEAD:src`). That is likely to change. To avoid starting up lots of processes, we use `git cat-file --batch` to read trees. But for blobs we only need to know the size, so use `git cat-file --batch-check` for blobs. This unfortunately means that we need to keep two `git` processes running, which means paying the git startup time twice.
2017-10-14 11:00:38 +03:00
}
// Record that the object has a blob of the specified `size` as a
// direct descendant.
func (s *TreeSize) addBlob(filename string, size BlobSize) {
Systematically prefer to cite more recent objects Given two objects that are the same size, or multiple trees/commits that refer to the same large object, which one should we prefer to cite in the footnotes? I think that new objects are more interesting than old objects [1], because * It is interesting to know whether the "bad" objects are still in use. * Once you have a commit containing a "bad" object, it is easier to trace it backwards in time than forwards. * New objects are more likely to be at the tips of branches and thus more likely to have a reference pointing at them. The old code was somewhat inconsistent because blobs and trees are processed in roughly reverse-chronological order whereas commits are processed in roughly chronological order. Improve the situation by implementing two pairs of `Count.AdjustMax()` methods, `AdjustMaxIfNecessary()` and `AdjustMaxIfPossible()`, differing in the obvious way if the two values that they are comparing are equal. By choosing which variant to use in which situation, we can arrange to preferentially pick the new of equal-sized objects. Note that this still doesn't cause newer commits to be preferred in the paths of trees and blobs. [1] Note that there are also arguments that the old objects are more interesting, because they are more likely to indicate the commits where a large object was first introduced.
2017-12-11 19:05:52 +03:00
s.MaxPathDepth.AdjustMaxIfNecessary(1)
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
s.MaxPathLength.AdjustMaxIfNecessary(counts.NewCount32(uint64(len(filename))))
s.ExpandedBlobSize.Increment(counts.Count64(size.Size))
Extract a method `Count.Increment()`
2017-10-29 09:42:40 +03:00
s.ExpandedBlobCount.Increment(1)
Implement the main command, `git-sizer` It can compute various dimensions about a tree object in a Git repository, in a way that is robust against git bombs. For now it must be passed the path to a Git repository and one or more tree object specifications (like `HEAD^{tree}` or `HEAD:src`). That is likely to change. To avoid starting up lots of processes, we use `git cat-file --batch` to read trees. But for blobs we only need to know the size, so use `git cat-file --batch-check` for blobs. This unfortunately means that we need to keep two `git` processes running, which means paying the git startup time twice.
2017-10-14 11:00:38 +03:00
}
// Record that the object has a link as a direct descendant.
func (s *TreeSize) addLink(filename string) {
Systematically prefer to cite more recent objects Given two objects that are the same size, or multiple trees/commits that refer to the same large object, which one should we prefer to cite in the footnotes? I think that new objects are more interesting than old objects [1], because * It is interesting to know whether the "bad" objects are still in use. * Once you have a commit containing a "bad" object, it is easier to trace it backwards in time than forwards. * New objects are more likely to be at the tips of branches and thus more likely to have a reference pointing at them. The old code was somewhat inconsistent because blobs and trees are processed in roughly reverse-chronological order whereas commits are processed in roughly chronological order. Improve the situation by implementing two pairs of `Count.AdjustMax()` methods, `AdjustMaxIfNecessary()` and `AdjustMaxIfPossible()`, differing in the obvious way if the two values that they are comparing are equal. By choosing which variant to use in which situation, we can arrange to preferentially pick the new of equal-sized objects. Note that this still doesn't cause newer commits to be preferred in the paths of trees and blobs. [1] Note that there are also arguments that the old objects are more interesting, because they are more likely to indicate the commits where a large object was first introduced.
2017-12-11 19:05:52 +03:00
s.MaxPathDepth.AdjustMaxIfNecessary(1)
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
s.MaxPathLength.AdjustMaxIfNecessary(counts.NewCount32(uint64(len(filename))))
Extract a method `Count.Increment()`
2017-10-29 09:42:40 +03:00
s.ExpandedLinkCount.Increment(1)
Implement the main command, `git-sizer` It can compute various dimensions about a tree object in a Git repository, in a way that is robust against git bombs. For now it must be passed the path to a Git repository and one or more tree object specifications (like `HEAD^{tree}` or `HEAD:src`). That is likely to change. To avoid starting up lots of processes, we use `git cat-file --batch` to read trees. But for blobs we only need to know the size, so use `git cat-file --batch-check` for blobs. This unfortunately means that we need to keep two `git` processes running, which means paying the git startup time twice.
2017-10-14 11:00:38 +03:00
}
// Record that the object has a submodule as a direct descendant.
func (s *TreeSize) addSubmodule(filename string) {
Systematically prefer to cite more recent objects Given two objects that are the same size, or multiple trees/commits that refer to the same large object, which one should we prefer to cite in the footnotes? I think that new objects are more interesting than old objects [1], because * It is interesting to know whether the "bad" objects are still in use. * Once you have a commit containing a "bad" object, it is easier to trace it backwards in time than forwards. * New objects are more likely to be at the tips of branches and thus more likely to have a reference pointing at them. The old code was somewhat inconsistent because blobs and trees are processed in roughly reverse-chronological order whereas commits are processed in roughly chronological order. Improve the situation by implementing two pairs of `Count.AdjustMax()` methods, `AdjustMaxIfNecessary()` and `AdjustMaxIfPossible()`, differing in the obvious way if the two values that they are comparing are equal. By choosing which variant to use in which situation, we can arrange to preferentially pick the new of equal-sized objects. Note that this still doesn't cause newer commits to be preferred in the paths of trees and blobs. [1] Note that there are also arguments that the old objects are more interesting, because they are more likely to indicate the commits where a large object was first introduced.
2017-12-11 19:05:52 +03:00
s.MaxPathDepth.AdjustMaxIfNecessary(1)
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
s.MaxPathLength.AdjustMaxIfNecessary(counts.NewCount32(uint64(len(filename))))
Extract a method `Count.Increment()`
2017-10-29 09:42:40 +03:00
s.ExpandedSubmoduleCount.Increment(1)
Implement the main command, `git-sizer` It can compute various dimensions about a tree object in a Git repository, in a way that is robust against git bombs. For now it must be passed the path to a Git repository and one or more tree object specifications (like `HEAD^{tree}` or `HEAD:src`). That is likely to change. To avoid starting up lots of processes, we use `git cat-file --batch` to read trees. But for blobs we only need to know the size, so use `git cat-file --batch-check` for blobs. This unfortunately means that we need to keep two `git` processes running, which means paying the git startup time twice.
2017-10-14 11:00:38 +03:00
}
Support gathering information about commits, too
2017-10-29 09:14:33 +03:00
type CommitSize struct {
// The height of the ancestor graph, including this commit.
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
MaxAncestorDepth counts.Count32 `json:"max_ancestor_depth"`
Support gathering information about commits, too
2017-10-29 09:14:33 +03:00
}
func (s *CommitSize) addParent(s2 CommitSize) {
Systematically prefer to cite more recent objects Given two objects that are the same size, or multiple trees/commits that refer to the same large object, which one should we prefer to cite in the footnotes? I think that new objects are more interesting than old objects [1], because * It is interesting to know whether the "bad" objects are still in use. * Once you have a commit containing a "bad" object, it is easier to trace it backwards in time than forwards. * New objects are more likely to be at the tips of branches and thus more likely to have a reference pointing at them. The old code was somewhat inconsistent because blobs and trees are processed in roughly reverse-chronological order whereas commits are processed in roughly chronological order. Improve the situation by implementing two pairs of `Count.AdjustMax()` methods, `AdjustMaxIfNecessary()` and `AdjustMaxIfPossible()`, differing in the obvious way if the two values that they are comparing are equal. By choosing which variant to use in which situation, we can arrange to preferentially pick the new of equal-sized objects. Note that this still doesn't cause newer commits to be preferred in the paths of trees and blobs. [1] Note that there are also arguments that the old objects are more interesting, because they are more likely to indicate the commits where a large object was first introduced.
2017-12-11 19:05:52 +03:00
s.MaxAncestorDepth.AdjustMaxIfNecessary(s2.MaxAncestorDepth)
Support gathering information about commits, too
2017-10-29 09:14:33 +03:00
}
func (s *CommitSize) addTree(s2 TreeSize) {
}
Handle tag objects
2017-10-30 09:09:40 +03:00
type TagSize struct {
// The number of tags that have to be traversed (including this
// one) to get to an object.
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
TagDepth counts.Count32
Handle tag objects
2017-10-30 09:09:40 +03:00
}
sizes.go: move some definitions around
2017-10-30 05:44:06 +03:00
type HistorySize struct {
// The total number of unique commits analyzed.
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
UniqueCommitCount counts.Count32 `json:"unique_commit_count"`
sizes.go: move some definitions around
2017-10-30 05:44:06 +03:00
// The total size of all commits analyzed.
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
UniqueCommitSize counts.Count64 `json:"unique_commit_size"`
sizes.go: move some definitions around
2017-10-30 05:44:06 +03:00
// The maximum size of any analyzed commit.
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
MaxCommitSize counts.Count32 `json:"max_commit_size"`
sizes.go: move some definitions around
2017-10-30 05:44:06 +03:00
Keep track of the worst objects in several dimensions
2017-12-08 02:32:40 +03:00
// The commit with the maximum size.
Report the paths to the worst objects It is frustrating to find out that there is, say, a gigantic blob in a repository with no easy way to find out where it is in the history. (Git's tooling for answering that question is almost nonexistent.) So... * Define a new class, `PathResolver`, that can compute the paths (i.e., the `git rev-parse` expression) that can be used to access an object with a specific OID. This class must be fed the objects and references in depth-first order, but that's OK because that's the order we need to process them to compute their expanded sizes. * Change `Sizes` to record `*Path` pointers, rather than OIDs, for the big objects that it has found. * If `Sizes` finds a new object that is bigger than the biggest that it has seen before, have it discard the (maybe only partly-computed) path of the old object and replace it with the new one. `PathResolver` knows how to free resources associated with the partly-computed paths, so basically the number of paths being computed at any time is at most one for each of the "largest" things that we want to track. * Output the paths (still in crude format) to the JSON output. This costs about 15% in runtime and hardly any additional memory consumption. Still to do: (1) Improve the format of the path output. (2) Emit better output for commits that are not referred to directly by references (perhaps using `git name-rev`). (3) Include the paths in non-JSON output, and probably change where they are presented in JSON format.
2017-12-08 19:14:47 +03:00
MaxCommitSizeCommit *Path `json:"max_commit,omitempty"`
Keep track of the worst objects in several dimensions
2017-12-08 02:32:40 +03:00
sizes.go: move some definitions around
2017-10-30 05:44:06 +03:00
// The maximum ancestor depth of any analyzed commit.
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
MaxHistoryDepth counts.Count32 `json:"max_history_depth"`
sizes.go: move some definitions around
2017-10-30 05:44:06 +03:00
// The maximum number of direct parents of any analyzed commit.
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
MaxParentCount counts.Count32 `json:"max_parent_count"`
sizes.go: move some definitions around
2017-10-30 05:44:06 +03:00
Add footnotes containing the paths to the tabular output For the non-JSON output, add footnote after the table listing the "biggest" objects referred to by the table, with bracketed citation numbers in the table indicating the appropriate footnote. If an object is "biggest" in more than one way, make the lines in the table refer to the same footnote.
2017-12-09 01:23:22 +03:00
// The commit with the maximum number of direct parents.
Add `omitempty` for JSON output of `MaxParentCountCommit`
2017-12-19 15:50:51 +03:00
MaxParentCountCommit *Path `json:"max_parent_count_commit,omitempty"`
Add footnotes containing the paths to the tabular output For the non-JSON output, add footnote after the table listing the "biggest" objects referred to by the table, with bracketed citation numbers in the table indicating the appropriate footnote. If an object is "biggest" in more than one way, make the lines in the table refer to the same footnote.
2017-12-09 01:23:22 +03:00
sizes.go: move some definitions around
2017-10-30 05:44:06 +03:00
// The total number of unique trees analyzed.
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
UniqueTreeCount counts.Count32 `json:"unique_tree_count"`
sizes.go: move some definitions around
2017-10-30 05:44:06 +03:00
// The total size of all trees analyzed.
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
UniqueTreeSize counts.Count64 `json:"unique_tree_size"`
sizes.go: move some definitions around
2017-10-30 05:44:06 +03:00
// The total number of tree entries in all unique trees analyzed.
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
UniqueTreeEntries counts.Count64 `json:"unique_tree_entries"`
sizes.go: move some definitions around
2017-10-30 05:44:06 +03:00
Move `MaxTreeEntries` from `TreeSize` to `HistorySize`
2017-10-30 05:57:46 +03:00
// The maximum number of entries an a tree.
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
MaxTreeEntries counts.Count32 `json:"max_tree_entries"`
Move `MaxTreeEntries` from `TreeSize` to `HistorySize`
2017-10-30 05:57:46 +03:00
Keep track of the worst objects in several dimensions
2017-12-08 02:32:40 +03:00
// The tree with the maximum number of entries.
Report the paths to the worst objects It is frustrating to find out that there is, say, a gigantic blob in a repository with no easy way to find out where it is in the history. (Git's tooling for answering that question is almost nonexistent.) So... * Define a new class, `PathResolver`, that can compute the paths (i.e., the `git rev-parse` expression) that can be used to access an object with a specific OID. This class must be fed the objects and references in depth-first order, but that's OK because that's the order we need to process them to compute their expanded sizes. * Change `Sizes` to record `*Path` pointers, rather than OIDs, for the big objects that it has found. * If `Sizes` finds a new object that is bigger than the biggest that it has seen before, have it discard the (maybe only partly-computed) path of the old object and replace it with the new one. `PathResolver` knows how to free resources associated with the partly-computed paths, so basically the number of paths being computed at any time is at most one for each of the "largest" things that we want to track. * Output the paths (still in crude format) to the JSON output. This costs about 15% in runtime and hardly any additional memory consumption. Still to do: (1) Improve the format of the path output. (2) Emit better output for commits that are not referred to directly by references (perhaps using `git name-rev`). (3) Include the paths in non-JSON output, and probably change where they are presented in JSON format.
2017-12-08 19:14:47 +03:00
MaxTreeEntriesTree *Path `json:"max_tree_entries_tree,omitempty"`
Keep track of the worst objects in several dimensions
2017-12-08 02:32:40 +03:00
sizes.go: move some definitions around
2017-10-30 05:44:06 +03:00
// The total number of unique blobs analyzed.
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
UniqueBlobCount counts.Count32 `json:"unique_blob_count"`
sizes.go: move some definitions around
2017-10-30 05:44:06 +03:00
// The total size of all of the unique blobs analyzed.
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
UniqueBlobSize counts.Count64 `json:"unique_blob_size"`
sizes.go: move some definitions around
2017-10-30 05:44:06 +03:00
HistorySize: keep track of the maximum blob size
2017-10-30 06:16:30 +03:00
// The maximum size of any analyzed blob.
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
MaxBlobSize counts.Count32 `json:"max_blob_size"`
HistorySize: keep track of the maximum blob size
2017-10-30 06:16:30 +03:00
Keep track of the worst objects in several dimensions
2017-12-08 02:32:40 +03:00
// The biggest blob found.
Report the paths to the worst objects It is frustrating to find out that there is, say, a gigantic blob in a repository with no easy way to find out where it is in the history. (Git's tooling for answering that question is almost nonexistent.) So... * Define a new class, `PathResolver`, that can compute the paths (i.e., the `git rev-parse` expression) that can be used to access an object with a specific OID. This class must be fed the objects and references in depth-first order, but that's OK because that's the order we need to process them to compute their expanded sizes. * Change `Sizes` to record `*Path` pointers, rather than OIDs, for the big objects that it has found. * If `Sizes` finds a new object that is bigger than the biggest that it has seen before, have it discard the (maybe only partly-computed) path of the old object and replace it with the new one. `PathResolver` knows how to free resources associated with the partly-computed paths, so basically the number of paths being computed at any time is at most one for each of the "largest" things that we want to track. * Output the paths (still in crude format) to the JSON output. This costs about 15% in runtime and hardly any additional memory consumption. Still to do: (1) Improve the format of the path output. (2) Emit better output for commits that are not referred to directly by references (perhaps using `git name-rev`). (3) Include the paths in non-JSON output, and probably change where they are presented in JSON format.
2017-12-08 19:14:47 +03:00
MaxBlobSizeBlob *Path `json:"max_blob_size_blob,omitempty"`
Keep track of the worst objects in several dimensions
2017-12-08 02:32:40 +03:00
sizes.go: move some definitions around
2017-10-30 05:44:06 +03:00
// The total number of unique tag objects analyzed.
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
UniqueTagCount counts.Count32 `json:"unique_tag_count"`
sizes.go: move some definitions around
2017-10-30 05:44:06 +03:00
Handle tag objects
2017-10-30 09:09:40 +03:00
// The maximum number of tags in a chain.
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
MaxTagDepth counts.Count32 `json:"max_tag_depth"`
Handle tag objects
2017-10-30 09:09:40 +03:00
Keep track of the worst objects in several dimensions
2017-12-08 02:32:40 +03:00
// The tag with the maximum tag depth.
Report the paths to the worst objects It is frustrating to find out that there is, say, a gigantic blob in a repository with no easy way to find out where it is in the history. (Git's tooling for answering that question is almost nonexistent.) So... * Define a new class, `PathResolver`, that can compute the paths (i.e., the `git rev-parse` expression) that can be used to access an object with a specific OID. This class must be fed the objects and references in depth-first order, but that's OK because that's the order we need to process them to compute their expanded sizes. * Change `Sizes` to record `*Path` pointers, rather than OIDs, for the big objects that it has found. * If `Sizes` finds a new object that is bigger than the biggest that it has seen before, have it discard the (maybe only partly-computed) path of the old object and replace it with the new one. `PathResolver` knows how to free resources associated with the partly-computed paths, so basically the number of paths being computed at any time is at most one for each of the "largest" things that we want to track. * Output the paths (still in crude format) to the JSON output. This costs about 15% in runtime and hardly any additional memory consumption. Still to do: (1) Improve the format of the path output. (2) Emit better output for commits that are not referred to directly by references (perhaps using `git name-rev`). (3) Include the paths in non-JSON output, and probably change where they are presented in JSON format.
2017-12-08 19:14:47 +03:00
MaxTagDepthTag *Path `json:"max_tag_depth_tag,omitempty"`
Keep track of the worst objects in several dimensions
2017-12-08 02:32:40 +03:00
Keep a count of the number of references analyzed
2017-10-30 09:11:38 +03:00
// The number of references analyzed. Note that we don't eliminate
// duplicates if the user passes the same reference more than
// once.
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
ReferenceCount counts.Count32 `json:"reference_count"`
Keep a count of the number of references analyzed
2017-10-30 09:11:38 +03:00
HistorySize: keep track of counts for each reference group
2021-10-13 15:29:39 +03:00
// ReferenceGroups keeps track of how many references in each
// reference group were scanned.
ReferenceGroups map[RefGroupSymbol]*counts.Count32 `json:"reference_groups"`
sizes.go: move some definitions around
2017-10-30 05:44:06 +03:00
// The maximum TreeSize in the analyzed history (where each
// attribute is maximized separately).
Inline the `TreeSize` into `HistorySize`
2017-12-08 02:07:38 +03:00
// The maximum depth of trees and blobs starting at this object
Don't include the root tree in `MaxPathDepth` It is more natural to define the length of a relative path `foo/bar/baz.txt` as 3 rather than 4, so tweak the code to do so.
2018-03-11 12:50:20 +03:00
// (not including this object).
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
MaxPathDepth counts.Count32 `json:"max_path_depth"`
Inline the `TreeSize` into `HistorySize`
2017-12-08 02:07:38 +03:00
Keep track of the worst objects in several dimensions
2017-12-08 02:32:40 +03:00
// The tree with the maximum path depth.
Report the paths to the worst objects It is frustrating to find out that there is, say, a gigantic blob in a repository with no easy way to find out where it is in the history. (Git's tooling for answering that question is almost nonexistent.) So... * Define a new class, `PathResolver`, that can compute the paths (i.e., the `git rev-parse` expression) that can be used to access an object with a specific OID. This class must be fed the objects and references in depth-first order, but that's OK because that's the order we need to process them to compute their expanded sizes. * Change `Sizes` to record `*Path` pointers, rather than OIDs, for the big objects that it has found. * If `Sizes` finds a new object that is bigger than the biggest that it has seen before, have it discard the (maybe only partly-computed) path of the old object and replace it with the new one. `PathResolver` knows how to free resources associated with the partly-computed paths, so basically the number of paths being computed at any time is at most one for each of the "largest" things that we want to track. * Output the paths (still in crude format) to the JSON output. This costs about 15% in runtime and hardly any additional memory consumption. Still to do: (1) Improve the format of the path output. (2) Emit better output for commits that are not referred to directly by references (perhaps using `git name-rev`). (3) Include the paths in non-JSON output, and probably change where they are presented in JSON format.
2017-12-08 19:14:47 +03:00
MaxPathDepthTree *Path `json:"max_path_depth_tree,omitempty"`
Keep track of the worst objects in several dimensions
2017-12-08 02:32:40 +03:00
Inline the `TreeSize` into `HistorySize`
2017-12-08 02:07:38 +03:00
// The maximum length of any path relative to this object, in
// characters.
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
MaxPathLength counts.Count32 `json:"max_path_length"`
Inline the `TreeSize` into `HistorySize`
2017-12-08 02:07:38 +03:00
Keep track of the worst objects in several dimensions
2017-12-08 02:32:40 +03:00
// The tree with the maximum path length.
Report the paths to the worst objects It is frustrating to find out that there is, say, a gigantic blob in a repository with no easy way to find out where it is in the history. (Git's tooling for answering that question is almost nonexistent.) So... * Define a new class, `PathResolver`, that can compute the paths (i.e., the `git rev-parse` expression) that can be used to access an object with a specific OID. This class must be fed the objects and references in depth-first order, but that's OK because that's the order we need to process them to compute their expanded sizes. * Change `Sizes` to record `*Path` pointers, rather than OIDs, for the big objects that it has found. * If `Sizes` finds a new object that is bigger than the biggest that it has seen before, have it discard the (maybe only partly-computed) path of the old object and replace it with the new one. `PathResolver` knows how to free resources associated with the partly-computed paths, so basically the number of paths being computed at any time is at most one for each of the "largest" things that we want to track. * Output the paths (still in crude format) to the JSON output. This costs about 15% in runtime and hardly any additional memory consumption. Still to do: (1) Improve the format of the path output. (2) Emit better output for commits that are not referred to directly by references (perhaps using `git name-rev`). (3) Include the paths in non-JSON output, and probably change where they are presented in JSON format.
2017-12-08 19:14:47 +03:00
MaxPathLengthTree *Path `json:"max_path_length_tree,omitempty"`
Keep track of the worst objects in several dimensions
2017-12-08 02:32:40 +03:00
Inline the `TreeSize` into `HistorySize`
2017-12-08 02:07:38 +03:00
// The total number of trees, including duplicates.
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
MaxExpandedTreeCount counts.Count32 `json:"max_expanded_tree_count"`
Inline the `TreeSize` into `HistorySize`
2017-12-08 02:07:38 +03:00
Keep track of the worst objects in several dimensions
2017-12-08 02:32:40 +03:00
// The tree with the maximum expanded tree count.
Report the paths to the worst objects It is frustrating to find out that there is, say, a gigantic blob in a repository with no easy way to find out where it is in the history. (Git's tooling for answering that question is almost nonexistent.) So... * Define a new class, `PathResolver`, that can compute the paths (i.e., the `git rev-parse` expression) that can be used to access an object with a specific OID. This class must be fed the objects and references in depth-first order, but that's OK because that's the order we need to process them to compute their expanded sizes. * Change `Sizes` to record `*Path` pointers, rather than OIDs, for the big objects that it has found. * If `Sizes` finds a new object that is bigger than the biggest that it has seen before, have it discard the (maybe only partly-computed) path of the old object and replace it with the new one. `PathResolver` knows how to free resources associated with the partly-computed paths, so basically the number of paths being computed at any time is at most one for each of the "largest" things that we want to track. * Output the paths (still in crude format) to the JSON output. This costs about 15% in runtime and hardly any additional memory consumption. Still to do: (1) Improve the format of the path output. (2) Emit better output for commits that are not referred to directly by references (perhaps using `git name-rev`). (3) Include the paths in non-JSON output, and probably change where they are presented in JSON format.
2017-12-08 19:14:47 +03:00
MaxExpandedTreeCountTree *Path `json:"max_expanded_tree_count_tree,omitempty"`
Keep track of the worst objects in several dimensions
2017-12-08 02:32:40 +03:00
Inline the `TreeSize` into `HistorySize`
2017-12-08 02:07:38 +03:00
// The total number of blobs, including duplicates.
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
MaxExpandedBlobCount counts.Count32 `json:"max_expanded_blob_count"`
Inline the `TreeSize` into `HistorySize`
2017-12-08 02:07:38 +03:00
Keep track of the worst objects in several dimensions
2017-12-08 02:32:40 +03:00
// The tree with the maximum expanded blob count.
Report the paths to the worst objects It is frustrating to find out that there is, say, a gigantic blob in a repository with no easy way to find out where it is in the history. (Git's tooling for answering that question is almost nonexistent.) So... * Define a new class, `PathResolver`, that can compute the paths (i.e., the `git rev-parse` expression) that can be used to access an object with a specific OID. This class must be fed the objects and references in depth-first order, but that's OK because that's the order we need to process them to compute their expanded sizes. * Change `Sizes` to record `*Path` pointers, rather than OIDs, for the big objects that it has found. * If `Sizes` finds a new object that is bigger than the biggest that it has seen before, have it discard the (maybe only partly-computed) path of the old object and replace it with the new one. `PathResolver` knows how to free resources associated with the partly-computed paths, so basically the number of paths being computed at any time is at most one for each of the "largest" things that we want to track. * Output the paths (still in crude format) to the JSON output. This costs about 15% in runtime and hardly any additional memory consumption. Still to do: (1) Improve the format of the path output. (2) Emit better output for commits that are not referred to directly by references (perhaps using `git name-rev`). (3) Include the paths in non-JSON output, and probably change where they are presented in JSON format.
2017-12-08 19:14:47 +03:00
MaxExpandedBlobCountTree *Path `json:"max_expanded_blob_count_tree,omitempty"`
Keep track of the worst objects in several dimensions
2017-12-08 02:32:40 +03:00
Inline the `TreeSize` into `HistorySize`
2017-12-08 02:07:38 +03:00
// The total size of all blobs, including duplicates.
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
MaxExpandedBlobSize counts.Count64 `json:"max_expanded_blob_size"`
Inline the `TreeSize` into `HistorySize`
2017-12-08 02:07:38 +03:00
Keep track of the worst objects in several dimensions
2017-12-08 02:32:40 +03:00
// The tree with the maximum expanded blob size.
Report the paths to the worst objects It is frustrating to find out that there is, say, a gigantic blob in a repository with no easy way to find out where it is in the history. (Git's tooling for answering that question is almost nonexistent.) So... * Define a new class, `PathResolver`, that can compute the paths (i.e., the `git rev-parse` expression) that can be used to access an object with a specific OID. This class must be fed the objects and references in depth-first order, but that's OK because that's the order we need to process them to compute their expanded sizes. * Change `Sizes` to record `*Path` pointers, rather than OIDs, for the big objects that it has found. * If `Sizes` finds a new object that is bigger than the biggest that it has seen before, have it discard the (maybe only partly-computed) path of the old object and replace it with the new one. `PathResolver` knows how to free resources associated with the partly-computed paths, so basically the number of paths being computed at any time is at most one for each of the "largest" things that we want to track. * Output the paths (still in crude format) to the JSON output. This costs about 15% in runtime and hardly any additional memory consumption. Still to do: (1) Improve the format of the path output. (2) Emit better output for commits that are not referred to directly by references (perhaps using `git name-rev`). (3) Include the paths in non-JSON output, and probably change where they are presented in JSON format.
2017-12-08 19:14:47 +03:00
MaxExpandedBlobSizeTree *Path `json:"max_expanded_blob_size_tree,omitempty"`
Keep track of the worst objects in several dimensions
2017-12-08 02:32:40 +03:00
Inline the `TreeSize` into `HistorySize`
2017-12-08 02:07:38 +03:00
// The total number of symbolic links, including duplicates.
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
MaxExpandedLinkCount counts.Count32 `json:"max_expanded_link_count"`
Inline the `TreeSize` into `HistorySize`
2017-12-08 02:07:38 +03:00
Keep track of the worst objects in several dimensions
2017-12-08 02:32:40 +03:00
// The tree with the maximum expanded link count.
Report the paths to the worst objects It is frustrating to find out that there is, say, a gigantic blob in a repository with no easy way to find out where it is in the history. (Git's tooling for answering that question is almost nonexistent.) So... * Define a new class, `PathResolver`, that can compute the paths (i.e., the `git rev-parse` expression) that can be used to access an object with a specific OID. This class must be fed the objects and references in depth-first order, but that's OK because that's the order we need to process them to compute their expanded sizes. * Change `Sizes` to record `*Path` pointers, rather than OIDs, for the big objects that it has found. * If `Sizes` finds a new object that is bigger than the biggest that it has seen before, have it discard the (maybe only partly-computed) path of the old object and replace it with the new one. `PathResolver` knows how to free resources associated with the partly-computed paths, so basically the number of paths being computed at any time is at most one for each of the "largest" things that we want to track. * Output the paths (still in crude format) to the JSON output. This costs about 15% in runtime and hardly any additional memory consumption. Still to do: (1) Improve the format of the path output. (2) Emit better output for commits that are not referred to directly by references (perhaps using `git name-rev`). (3) Include the paths in non-JSON output, and probably change where they are presented in JSON format.
2017-12-08 19:14:47 +03:00
MaxExpandedLinkCountTree *Path `json:"max_expanded_link_count_tree,omitempty"`
Keep track of the worst objects in several dimensions
2017-12-08 02:32:40 +03:00
Inline the `TreeSize` into `HistorySize`
2017-12-08 02:07:38 +03:00
// The total number of submodules referenced, including duplicates.
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
MaxExpandedSubmoduleCount counts.Count32 `json:"max_expanded_submodule_count"`
Keep track of the worst objects in several dimensions
2017-12-08 02:32:40 +03:00
// The tree with the maximum expanded submodule count.
Report the paths to the worst objects It is frustrating to find out that there is, say, a gigantic blob in a repository with no easy way to find out where it is in the history. (Git's tooling for answering that question is almost nonexistent.) So... * Define a new class, `PathResolver`, that can compute the paths (i.e., the `git rev-parse` expression) that can be used to access an object with a specific OID. This class must be fed the objects and references in depth-first order, but that's OK because that's the order we need to process them to compute their expanded sizes. * Change `Sizes` to record `*Path` pointers, rather than OIDs, for the big objects that it has found. * If `Sizes` finds a new object that is bigger than the biggest that it has seen before, have it discard the (maybe only partly-computed) path of the old object and replace it with the new one. `PathResolver` knows how to free resources associated with the partly-computed paths, so basically the number of paths being computed at any time is at most one for each of the "largest" things that we want to track. * Output the paths (still in crude format) to the JSON output. This costs about 15% in runtime and hardly any additional memory consumption. Still to do: (1) Improve the format of the path output. (2) Emit better output for commits that are not referred to directly by references (perhaps using `git name-rev`). (3) Include the paths in non-JSON output, and probably change where they are presented in JSON format.
2017-12-08 19:14:47 +03:00
MaxExpandedSubmoduleCountTree *Path `json:"max_expanded_submodule_count_tree,omitempty"`
}
// Convenience function: forget `*path` if it is non-nil and overwrite
// it with a `*Path` for the object corresponding to `(oid,
// objectType)`. This function can be used if a new largest item was
// found.
func setPath(
If paths are not being output, don't even compute them This saves a bit of time.
2017-12-19 14:40:13 +03:00
pr PathResolver,
Report the paths to the worst objects It is frustrating to find out that there is, say, a gigantic blob in a repository with no easy way to find out where it is in the history. (Git's tooling for answering that question is almost nonexistent.) So... * Define a new class, `PathResolver`, that can compute the paths (i.e., the `git rev-parse` expression) that can be used to access an object with a specific OID. This class must be fed the objects and references in depth-first order, but that's OK because that's the order we need to process them to compute their expanded sizes. * Change `Sizes` to record `*Path` pointers, rather than OIDs, for the big objects that it has found. * If `Sizes` finds a new object that is bigger than the biggest that it has seen before, have it discard the (maybe only partly-computed) path of the old object and replace it with the new one. `PathResolver` knows how to free resources associated with the partly-computed paths, so basically the number of paths being computed at any time is at most one for each of the "largest" things that we want to track. * Output the paths (still in crude format) to the JSON output. This costs about 15% in runtime and hardly any additional memory consumption. Still to do: (1) Improve the format of the path output. (2) Emit better output for commits that are not referred to directly by references (perhaps using `git name-rev`). (3) Include the paths in non-JSON output, and probably change where they are presented in JSON format.
2017-12-08 19:14:47 +03:00
path **Path,
Rename `Oid` → `OID`
2018-02-26 08:39:53 +03:00
oid git.OID, objectType string) {
Report the paths to the worst objects It is frustrating to find out that there is, say, a gigantic blob in a repository with no easy way to find out where it is in the history. (Git's tooling for answering that question is almost nonexistent.) So... * Define a new class, `PathResolver`, that can compute the paths (i.e., the `git rev-parse` expression) that can be used to access an object with a specific OID. This class must be fed the objects and references in depth-first order, but that's OK because that's the order we need to process them to compute their expanded sizes. * Change `Sizes` to record `*Path` pointers, rather than OIDs, for the big objects that it has found. * If `Sizes` finds a new object that is bigger than the biggest that it has seen before, have it discard the (maybe only partly-computed) path of the old object and replace it with the new one. `PathResolver` knows how to free resources associated with the partly-computed paths, so basically the number of paths being computed at any time is at most one for each of the "largest" things that we want to track. * Output the paths (still in crude format) to the JSON output. This costs about 15% in runtime and hardly any additional memory consumption. Still to do: (1) Improve the format of the path output. (2) Emit better output for commits that are not referred to directly by references (perhaps using `git name-rev`). (3) Include the paths in non-JSON output, and probably change where they are presented in JSON format.
2017-12-08 19:14:47 +03:00
if *path != nil {
pr.ForgetPath(*path)
}
*path = pr.RequestPath(oid, objectType)
sizes.go: move some definitions around
2017-10-30 05:44:06 +03:00
}
Rename `Oid` → `OID`
2018-02-26 08:39:53 +03:00
func (s *HistorySize) recordBlob(g *Graph, oid git.OID, blobSize BlobSize) {
sizes.go: move some definitions around
2017-10-30 05:44:06 +03:00
s.UniqueBlobCount.Increment(1)
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
s.UniqueBlobSize.Increment(counts.Count64(blobSize.Size))
Systematically prefer to cite more recent objects Given two objects that are the same size, or multiple trees/commits that refer to the same large object, which one should we prefer to cite in the footnotes? I think that new objects are more interesting than old objects [1], because * It is interesting to know whether the "bad" objects are still in use. * Once you have a commit containing a "bad" object, it is easier to trace it backwards in time than forwards. * New objects are more likely to be at the tips of branches and thus more likely to have a reference pointing at them. The old code was somewhat inconsistent because blobs and trees are processed in roughly reverse-chronological order whereas commits are processed in roughly chronological order. Improve the situation by implementing two pairs of `Count.AdjustMax()` methods, `AdjustMaxIfNecessary()` and `AdjustMaxIfPossible()`, differing in the obvious way if the two values that they are comparing are equal. By choosing which variant to use in which situation, we can arrange to preferentially pick the new of equal-sized objects. Note that this still doesn't cause newer commits to be preferred in the paths of trees and blobs. [1] Note that there are also arguments that the old objects are more interesting, because they are more likely to indicate the commits where a large object was first introduced.
2017-12-11 19:05:52 +03:00
if s.MaxBlobSize.AdjustMaxIfNecessary(blobSize.Size) {
Report the paths to the worst objects It is frustrating to find out that there is, say, a gigantic blob in a repository with no easy way to find out where it is in the history. (Git's tooling for answering that question is almost nonexistent.) So... * Define a new class, `PathResolver`, that can compute the paths (i.e., the `git rev-parse` expression) that can be used to access an object with a specific OID. This class must be fed the objects and references in depth-first order, but that's OK because that's the order we need to process them to compute their expanded sizes. * Change `Sizes` to record `*Path` pointers, rather than OIDs, for the big objects that it has found. * If `Sizes` finds a new object that is bigger than the biggest that it has seen before, have it discard the (maybe only partly-computed) path of the old object and replace it with the new one. `PathResolver` knows how to free resources associated with the partly-computed paths, so basically the number of paths being computed at any time is at most one for each of the "largest" things that we want to track. * Output the paths (still in crude format) to the JSON output. This costs about 15% in runtime and hardly any additional memory consumption. Still to do: (1) Improve the format of the path output. (2) Emit better output for commits that are not referred to directly by references (perhaps using `git name-rev`). (3) Include the paths in non-JSON output, and probably change where they are presented in JSON format.
2017-12-08 19:14:47 +03:00
setPath(g.pathResolver, &s.MaxBlobSizeBlob, oid, "blob")
Keep track of the worst objects in several dimensions
2017-12-08 02:32:40 +03:00
}
sizes.go: move some definitions around
2017-10-30 05:44:06 +03:00
}
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
func (s *HistorySize) recordTree(
Rename `Oid` → `OID`
2018-02-26 08:39:53 +03:00
g *Graph, oid git.OID, treeSize TreeSize, size counts.Count32, treeEntries counts.Count32,
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
) {
sizes.go: move some definitions around
2017-10-30 05:44:06 +03:00
s.UniqueTreeCount.Increment(1)
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
s.UniqueTreeSize.Increment(counts.Count64(size))
s.UniqueTreeEntries.Increment(counts.Count64(treeEntries))
Systematically prefer to cite more recent objects Given two objects that are the same size, or multiple trees/commits that refer to the same large object, which one should we prefer to cite in the footnotes? I think that new objects are more interesting than old objects [1], because * It is interesting to know whether the "bad" objects are still in use. * Once you have a commit containing a "bad" object, it is easier to trace it backwards in time than forwards. * New objects are more likely to be at the tips of branches and thus more likely to have a reference pointing at them. The old code was somewhat inconsistent because blobs and trees are processed in roughly reverse-chronological order whereas commits are processed in roughly chronological order. Improve the situation by implementing two pairs of `Count.AdjustMax()` methods, `AdjustMaxIfNecessary()` and `AdjustMaxIfPossible()`, differing in the obvious way if the two values that they are comparing are equal. By choosing which variant to use in which situation, we can arrange to preferentially pick the new of equal-sized objects. Note that this still doesn't cause newer commits to be preferred in the paths of trees and blobs. [1] Note that there are also arguments that the old objects are more interesting, because they are more likely to indicate the commits where a large object was first introduced.
2017-12-11 19:05:52 +03:00
if s.MaxTreeEntries.AdjustMaxIfNecessary(treeEntries) {
Report the paths to the worst objects It is frustrating to find out that there is, say, a gigantic blob in a repository with no easy way to find out where it is in the history. (Git's tooling for answering that question is almost nonexistent.) So... * Define a new class, `PathResolver`, that can compute the paths (i.e., the `git rev-parse` expression) that can be used to access an object with a specific OID. This class must be fed the objects and references in depth-first order, but that's OK because that's the order we need to process them to compute their expanded sizes. * Change `Sizes` to record `*Path` pointers, rather than OIDs, for the big objects that it has found. * If `Sizes` finds a new object that is bigger than the biggest that it has seen before, have it discard the (maybe only partly-computed) path of the old object and replace it with the new one. `PathResolver` knows how to free resources associated with the partly-computed paths, so basically the number of paths being computed at any time is at most one for each of the "largest" things that we want to track. * Output the paths (still in crude format) to the JSON output. This costs about 15% in runtime and hardly any additional memory consumption. Still to do: (1) Improve the format of the path output. (2) Emit better output for commits that are not referred to directly by references (perhaps using `git name-rev`). (3) Include the paths in non-JSON output, and probably change where they are presented in JSON format.
2017-12-08 19:14:47 +03:00
setPath(g.pathResolver, &s.MaxTreeEntriesTree, oid, "tree")
Keep track of the worst objects in several dimensions
2017-12-08 02:32:40 +03:00
}
Inline the `TreeSize` into `HistorySize`
2017-12-08 02:07:38 +03:00
Systematically prefer to cite more recent objects Given two objects that are the same size, or multiple trees/commits that refer to the same large object, which one should we prefer to cite in the footnotes? I think that new objects are more interesting than old objects [1], because * It is interesting to know whether the "bad" objects are still in use. * Once you have a commit containing a "bad" object, it is easier to trace it backwards in time than forwards. * New objects are more likely to be at the tips of branches and thus more likely to have a reference pointing at them. The old code was somewhat inconsistent because blobs and trees are processed in roughly reverse-chronological order whereas commits are processed in roughly chronological order. Improve the situation by implementing two pairs of `Count.AdjustMax()` methods, `AdjustMaxIfNecessary()` and `AdjustMaxIfPossible()`, differing in the obvious way if the two values that they are comparing are equal. By choosing which variant to use in which situation, we can arrange to preferentially pick the new of equal-sized objects. Note that this still doesn't cause newer commits to be preferred in the paths of trees and blobs. [1] Note that there are also arguments that the old objects are more interesting, because they are more likely to indicate the commits where a large object was first introduced.
2017-12-11 19:05:52 +03:00
if s.MaxPathDepth.AdjustMaxIfNecessary(treeSize.MaxPathDepth) {
Report the paths to the worst objects It is frustrating to find out that there is, say, a gigantic blob in a repository with no easy way to find out where it is in the history. (Git's tooling for answering that question is almost nonexistent.) So... * Define a new class, `PathResolver`, that can compute the paths (i.e., the `git rev-parse` expression) that can be used to access an object with a specific OID. This class must be fed the objects and references in depth-first order, but that's OK because that's the order we need to process them to compute their expanded sizes. * Change `Sizes` to record `*Path` pointers, rather than OIDs, for the big objects that it has found. * If `Sizes` finds a new object that is bigger than the biggest that it has seen before, have it discard the (maybe only partly-computed) path of the old object and replace it with the new one. `PathResolver` knows how to free resources associated with the partly-computed paths, so basically the number of paths being computed at any time is at most one for each of the "largest" things that we want to track. * Output the paths (still in crude format) to the JSON output. This costs about 15% in runtime and hardly any additional memory consumption. Still to do: (1) Improve the format of the path output. (2) Emit better output for commits that are not referred to directly by references (perhaps using `git name-rev`). (3) Include the paths in non-JSON output, and probably change where they are presented in JSON format.
2017-12-08 19:14:47 +03:00
setPath(g.pathResolver, &s.MaxPathDepthTree, oid, "tree")
Keep track of the worst objects in several dimensions
2017-12-08 02:32:40 +03:00
}
Systematically prefer to cite more recent objects Given two objects that are the same size, or multiple trees/commits that refer to the same large object, which one should we prefer to cite in the footnotes? I think that new objects are more interesting than old objects [1], because * It is interesting to know whether the "bad" objects are still in use. * Once you have a commit containing a "bad" object, it is easier to trace it backwards in time than forwards. * New objects are more likely to be at the tips of branches and thus more likely to have a reference pointing at them. The old code was somewhat inconsistent because blobs and trees are processed in roughly reverse-chronological order whereas commits are processed in roughly chronological order. Improve the situation by implementing two pairs of `Count.AdjustMax()` methods, `AdjustMaxIfNecessary()` and `AdjustMaxIfPossible()`, differing in the obvious way if the two values that they are comparing are equal. By choosing which variant to use in which situation, we can arrange to preferentially pick the new of equal-sized objects. Note that this still doesn't cause newer commits to be preferred in the paths of trees and blobs. [1] Note that there are also arguments that the old objects are more interesting, because they are more likely to indicate the commits where a large object was first introduced.
2017-12-11 19:05:52 +03:00
if s.MaxPathLength.AdjustMaxIfNecessary(treeSize.MaxPathLength) {
Report the paths to the worst objects It is frustrating to find out that there is, say, a gigantic blob in a repository with no easy way to find out where it is in the history. (Git's tooling for answering that question is almost nonexistent.) So... * Define a new class, `PathResolver`, that can compute the paths (i.e., the `git rev-parse` expression) that can be used to access an object with a specific OID. This class must be fed the objects and references in depth-first order, but that's OK because that's the order we need to process them to compute their expanded sizes. * Change `Sizes` to record `*Path` pointers, rather than OIDs, for the big objects that it has found. * If `Sizes` finds a new object that is bigger than the biggest that it has seen before, have it discard the (maybe only partly-computed) path of the old object and replace it with the new one. `PathResolver` knows how to free resources associated with the partly-computed paths, so basically the number of paths being computed at any time is at most one for each of the "largest" things that we want to track. * Output the paths (still in crude format) to the JSON output. This costs about 15% in runtime and hardly any additional memory consumption. Still to do: (1) Improve the format of the path output. (2) Emit better output for commits that are not referred to directly by references (perhaps using `git name-rev`). (3) Include the paths in non-JSON output, and probably change where they are presented in JSON format.
2017-12-08 19:14:47 +03:00
setPath(g.pathResolver, &s.MaxPathLengthTree, oid, "tree")
Keep track of the worst objects in several dimensions
2017-12-08 02:32:40 +03:00
}
Systematically prefer to cite more recent objects Given two objects that are the same size, or multiple trees/commits that refer to the same large object, which one should we prefer to cite in the footnotes? I think that new objects are more interesting than old objects [1], because * It is interesting to know whether the "bad" objects are still in use. * Once you have a commit containing a "bad" object, it is easier to trace it backwards in time than forwards. * New objects are more likely to be at the tips of branches and thus more likely to have a reference pointing at them. The old code was somewhat inconsistent because blobs and trees are processed in roughly reverse-chronological order whereas commits are processed in roughly chronological order. Improve the situation by implementing two pairs of `Count.AdjustMax()` methods, `AdjustMaxIfNecessary()` and `AdjustMaxIfPossible()`, differing in the obvious way if the two values that they are comparing are equal. By choosing which variant to use in which situation, we can arrange to preferentially pick the new of equal-sized objects. Note that this still doesn't cause newer commits to be preferred in the paths of trees and blobs. [1] Note that there are also arguments that the old objects are more interesting, because they are more likely to indicate the commits where a large object was first introduced.
2017-12-11 19:05:52 +03:00
if s.MaxExpandedTreeCount.AdjustMaxIfNecessary(treeSize.ExpandedTreeCount) {
Report the paths to the worst objects It is frustrating to find out that there is, say, a gigantic blob in a repository with no easy way to find out where it is in the history. (Git's tooling for answering that question is almost nonexistent.) So... * Define a new class, `PathResolver`, that can compute the paths (i.e., the `git rev-parse` expression) that can be used to access an object with a specific OID. This class must be fed the objects and references in depth-first order, but that's OK because that's the order we need to process them to compute their expanded sizes. * Change `Sizes` to record `*Path` pointers, rather than OIDs, for the big objects that it has found. * If `Sizes` finds a new object that is bigger than the biggest that it has seen before, have it discard the (maybe only partly-computed) path of the old object and replace it with the new one. `PathResolver` knows how to free resources associated with the partly-computed paths, so basically the number of paths being computed at any time is at most one for each of the "largest" things that we want to track. * Output the paths (still in crude format) to the JSON output. This costs about 15% in runtime and hardly any additional memory consumption. Still to do: (1) Improve the format of the path output. (2) Emit better output for commits that are not referred to directly by references (perhaps using `git name-rev`). (3) Include the paths in non-JSON output, and probably change where they are presented in JSON format.
2017-12-08 19:14:47 +03:00
setPath(g.pathResolver, &s.MaxExpandedTreeCountTree, oid, "tree")
Keep track of the worst objects in several dimensions
2017-12-08 02:32:40 +03:00
}
Systematically prefer to cite more recent objects Given two objects that are the same size, or multiple trees/commits that refer to the same large object, which one should we prefer to cite in the footnotes? I think that new objects are more interesting than old objects [1], because * It is interesting to know whether the "bad" objects are still in use. * Once you have a commit containing a "bad" object, it is easier to trace it backwards in time than forwards. * New objects are more likely to be at the tips of branches and thus more likely to have a reference pointing at them. The old code was somewhat inconsistent because blobs and trees are processed in roughly reverse-chronological order whereas commits are processed in roughly chronological order. Improve the situation by implementing two pairs of `Count.AdjustMax()` methods, `AdjustMaxIfNecessary()` and `AdjustMaxIfPossible()`, differing in the obvious way if the two values that they are comparing are equal. By choosing which variant to use in which situation, we can arrange to preferentially pick the new of equal-sized objects. Note that this still doesn't cause newer commits to be preferred in the paths of trees and blobs. [1] Note that there are also arguments that the old objects are more interesting, because they are more likely to indicate the commits where a large object was first introduced.
2017-12-11 19:05:52 +03:00
if s.MaxExpandedBlobCount.AdjustMaxIfNecessary(treeSize.ExpandedBlobCount) {
Report the paths to the worst objects It is frustrating to find out that there is, say, a gigantic blob in a repository with no easy way to find out where it is in the history. (Git's tooling for answering that question is almost nonexistent.) So... * Define a new class, `PathResolver`, that can compute the paths (i.e., the `git rev-parse` expression) that can be used to access an object with a specific OID. This class must be fed the objects and references in depth-first order, but that's OK because that's the order we need to process them to compute their expanded sizes. * Change `Sizes` to record `*Path` pointers, rather than OIDs, for the big objects that it has found. * If `Sizes` finds a new object that is bigger than the biggest that it has seen before, have it discard the (maybe only partly-computed) path of the old object and replace it with the new one. `PathResolver` knows how to free resources associated with the partly-computed paths, so basically the number of paths being computed at any time is at most one for each of the "largest" things that we want to track. * Output the paths (still in crude format) to the JSON output. This costs about 15% in runtime and hardly any additional memory consumption. Still to do: (1) Improve the format of the path output. (2) Emit better output for commits that are not referred to directly by references (perhaps using `git name-rev`). (3) Include the paths in non-JSON output, and probably change where they are presented in JSON format.
2017-12-08 19:14:47 +03:00
setPath(g.pathResolver, &s.MaxExpandedBlobCountTree, oid, "tree")
Keep track of the worst objects in several dimensions
2017-12-08 02:32:40 +03:00
}
Systematically prefer to cite more recent objects Given two objects that are the same size, or multiple trees/commits that refer to the same large object, which one should we prefer to cite in the footnotes? I think that new objects are more interesting than old objects [1], because * It is interesting to know whether the "bad" objects are still in use. * Once you have a commit containing a "bad" object, it is easier to trace it backwards in time than forwards. * New objects are more likely to be at the tips of branches and thus more likely to have a reference pointing at them. The old code was somewhat inconsistent because blobs and trees are processed in roughly reverse-chronological order whereas commits are processed in roughly chronological order. Improve the situation by implementing two pairs of `Count.AdjustMax()` methods, `AdjustMaxIfNecessary()` and `AdjustMaxIfPossible()`, differing in the obvious way if the two values that they are comparing are equal. By choosing which variant to use in which situation, we can arrange to preferentially pick the new of equal-sized objects. Note that this still doesn't cause newer commits to be preferred in the paths of trees and blobs. [1] Note that there are also arguments that the old objects are more interesting, because they are more likely to indicate the commits where a large object was first introduced.
2017-12-11 19:05:52 +03:00
if s.MaxExpandedBlobSize.AdjustMaxIfNecessary(treeSize.ExpandedBlobSize) {
Report the paths to the worst objects It is frustrating to find out that there is, say, a gigantic blob in a repository with no easy way to find out where it is in the history. (Git's tooling for answering that question is almost nonexistent.) So... * Define a new class, `PathResolver`, that can compute the paths (i.e., the `git rev-parse` expression) that can be used to access an object with a specific OID. This class must be fed the objects and references in depth-first order, but that's OK because that's the order we need to process them to compute their expanded sizes. * Change `Sizes` to record `*Path` pointers, rather than OIDs, for the big objects that it has found. * If `Sizes` finds a new object that is bigger than the biggest that it has seen before, have it discard the (maybe only partly-computed) path of the old object and replace it with the new one. `PathResolver` knows how to free resources associated with the partly-computed paths, so basically the number of paths being computed at any time is at most one for each of the "largest" things that we want to track. * Output the paths (still in crude format) to the JSON output. This costs about 15% in runtime and hardly any additional memory consumption. Still to do: (1) Improve the format of the path output. (2) Emit better output for commits that are not referred to directly by references (perhaps using `git name-rev`). (3) Include the paths in non-JSON output, and probably change where they are presented in JSON format.
2017-12-08 19:14:47 +03:00
setPath(g.pathResolver, &s.MaxExpandedBlobSizeTree, oid, "tree")
Keep track of the worst objects in several dimensions
2017-12-08 02:32:40 +03:00
}
Systematically prefer to cite more recent objects Given two objects that are the same size, or multiple trees/commits that refer to the same large object, which one should we prefer to cite in the footnotes? I think that new objects are more interesting than old objects [1], because * It is interesting to know whether the "bad" objects are still in use. * Once you have a commit containing a "bad" object, it is easier to trace it backwards in time than forwards. * New objects are more likely to be at the tips of branches and thus more likely to have a reference pointing at them. The old code was somewhat inconsistent because blobs and trees are processed in roughly reverse-chronological order whereas commits are processed in roughly chronological order. Improve the situation by implementing two pairs of `Count.AdjustMax()` methods, `AdjustMaxIfNecessary()` and `AdjustMaxIfPossible()`, differing in the obvious way if the two values that they are comparing are equal. By choosing which variant to use in which situation, we can arrange to preferentially pick the new of equal-sized objects. Note that this still doesn't cause newer commits to be preferred in the paths of trees and blobs. [1] Note that there are also arguments that the old objects are more interesting, because they are more likely to indicate the commits where a large object was first introduced.
2017-12-11 19:05:52 +03:00
if s.MaxExpandedLinkCount.AdjustMaxIfNecessary(treeSize.ExpandedLinkCount) {
Report the paths to the worst objects It is frustrating to find out that there is, say, a gigantic blob in a repository with no easy way to find out where it is in the history. (Git's tooling for answering that question is almost nonexistent.) So... * Define a new class, `PathResolver`, that can compute the paths (i.e., the `git rev-parse` expression) that can be used to access an object with a specific OID. This class must be fed the objects and references in depth-first order, but that's OK because that's the order we need to process them to compute their expanded sizes. * Change `Sizes` to record `*Path` pointers, rather than OIDs, for the big objects that it has found. * If `Sizes` finds a new object that is bigger than the biggest that it has seen before, have it discard the (maybe only partly-computed) path of the old object and replace it with the new one. `PathResolver` knows how to free resources associated with the partly-computed paths, so basically the number of paths being computed at any time is at most one for each of the "largest" things that we want to track. * Output the paths (still in crude format) to the JSON output. This costs about 15% in runtime and hardly any additional memory consumption. Still to do: (1) Improve the format of the path output. (2) Emit better output for commits that are not referred to directly by references (perhaps using `git name-rev`). (3) Include the paths in non-JSON output, and probably change where they are presented in JSON format.
2017-12-08 19:14:47 +03:00
setPath(g.pathResolver, &s.MaxExpandedLinkCountTree, oid, "tree")
Keep track of the worst objects in several dimensions
2017-12-08 02:32:40 +03:00
}
Systematically prefer to cite more recent objects Given two objects that are the same size, or multiple trees/commits that refer to the same large object, which one should we prefer to cite in the footnotes? I think that new objects are more interesting than old objects [1], because * It is interesting to know whether the "bad" objects are still in use. * Once you have a commit containing a "bad" object, it is easier to trace it backwards in time than forwards. * New objects are more likely to be at the tips of branches and thus more likely to have a reference pointing at them. The old code was somewhat inconsistent because blobs and trees are processed in roughly reverse-chronological order whereas commits are processed in roughly chronological order. Improve the situation by implementing two pairs of `Count.AdjustMax()` methods, `AdjustMaxIfNecessary()` and `AdjustMaxIfPossible()`, differing in the obvious way if the two values that they are comparing are equal. By choosing which variant to use in which situation, we can arrange to preferentially pick the new of equal-sized objects. Note that this still doesn't cause newer commits to be preferred in the paths of trees and blobs. [1] Note that there are also arguments that the old objects are more interesting, because they are more likely to indicate the commits where a large object was first introduced.
2017-12-11 19:05:52 +03:00
if s.MaxExpandedSubmoduleCount.AdjustMaxIfNecessary(treeSize.ExpandedSubmoduleCount) {
Report the paths to the worst objects It is frustrating to find out that there is, say, a gigantic blob in a repository with no easy way to find out where it is in the history. (Git's tooling for answering that question is almost nonexistent.) So... * Define a new class, `PathResolver`, that can compute the paths (i.e., the `git rev-parse` expression) that can be used to access an object with a specific OID. This class must be fed the objects and references in depth-first order, but that's OK because that's the order we need to process them to compute their expanded sizes. * Change `Sizes` to record `*Path` pointers, rather than OIDs, for the big objects that it has found. * If `Sizes` finds a new object that is bigger than the biggest that it has seen before, have it discard the (maybe only partly-computed) path of the old object and replace it with the new one. `PathResolver` knows how to free resources associated with the partly-computed paths, so basically the number of paths being computed at any time is at most one for each of the "largest" things that we want to track. * Output the paths (still in crude format) to the JSON output. This costs about 15% in runtime and hardly any additional memory consumption. Still to do: (1) Improve the format of the path output. (2) Emit better output for commits that are not referred to directly by references (perhaps using `git name-rev`). (3) Include the paths in non-JSON output, and probably change where they are presented in JSON format.
2017-12-08 19:14:47 +03:00
setPath(g.pathResolver, &s.MaxExpandedSubmoduleCountTree, oid, "tree")
Keep track of the worst objects in several dimensions
2017-12-08 02:32:40 +03:00
}
sizes.go: move some definitions around
2017-10-30 05:44:06 +03:00
}
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
func (s *HistorySize) recordCommit(
Rename `Oid` → `OID`
2018-02-26 08:39:53 +03:00
g *Graph, oid git.OID, commitSize CommitSize,
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
size counts.Count32, parentCount counts.Count32,
) {
sizes.go: move some definitions around
2017-10-30 05:44:06 +03:00
s.UniqueCommitCount.Increment(1)
Separate out a `counts` package
2018-02-26 08:12:49 +03:00
s.UniqueCommitSize.Increment(counts.Count64(size))
Systematically prefer to cite more recent objects Given two objects that are the same size, or multiple trees/commits that refer to the same large object, which one should we prefer to cite in the footnotes? I think that new objects are more interesting than old objects [1], because * It is interesting to know whether the "bad" objects are still in use. * Once you have a commit containing a "bad" object, it is easier to trace it backwards in time than forwards. * New objects are more likely to be at the tips of branches and thus more likely to have a reference pointing at them. The old code was somewhat inconsistent because blobs and trees are processed in roughly reverse-chronological order whereas commits are processed in roughly chronological order. Improve the situation by implementing two pairs of `Count.AdjustMax()` methods, `AdjustMaxIfNecessary()` and `AdjustMaxIfPossible()`, differing in the obvious way if the two values that they are comparing are equal. By choosing which variant to use in which situation, we can arrange to preferentially pick the new of equal-sized objects. Note that this still doesn't cause newer commits to be preferred in the paths of trees and blobs. [1] Note that there are also arguments that the old objects are more interesting, because they are more likely to indicate the commits where a large object was first introduced.
2017-12-11 19:05:52 +03:00
if s.MaxCommitSize.AdjustMaxIfPossible(size) {
Report the paths to the worst objects It is frustrating to find out that there is, say, a gigantic blob in a repository with no easy way to find out where it is in the history. (Git's tooling for answering that question is almost nonexistent.) So... * Define a new class, `PathResolver`, that can compute the paths (i.e., the `git rev-parse` expression) that can be used to access an object with a specific OID. This class must be fed the objects and references in depth-first order, but that's OK because that's the order we need to process them to compute their expanded sizes. * Change `Sizes` to record `*Path` pointers, rather than OIDs, for the big objects that it has found. * If `Sizes` finds a new object that is bigger than the biggest that it has seen before, have it discard the (maybe only partly-computed) path of the old object and replace it with the new one. `PathResolver` knows how to free resources associated with the partly-computed paths, so basically the number of paths being computed at any time is at most one for each of the "largest" things that we want to track. * Output the paths (still in crude format) to the JSON output. This costs about 15% in runtime and hardly any additional memory consumption. Still to do: (1) Improve the format of the path output. (2) Emit better output for commits that are not referred to directly by references (perhaps using `git name-rev`). (3) Include the paths in non-JSON output, and probably change where they are presented in JSON format.
2017-12-08 19:14:47 +03:00
setPath(g.pathResolver, &s.MaxCommitSizeCommit, oid, "commit")
Keep track of the worst objects in several dimensions
2017-12-08 02:32:40 +03:00
}
Systematically prefer to cite more recent objects Given two objects that are the same size, or multiple trees/commits that refer to the same large object, which one should we prefer to cite in the footnotes? I think that new objects are more interesting than old objects [1], because * It is interesting to know whether the "bad" objects are still in use. * Once you have a commit containing a "bad" object, it is easier to trace it backwards in time than forwards. * New objects are more likely to be at the tips of branches and thus more likely to have a reference pointing at them. The old code was somewhat inconsistent because blobs and trees are processed in roughly reverse-chronological order whereas commits are processed in roughly chronological order. Improve the situation by implementing two pairs of `Count.AdjustMax()` methods, `AdjustMaxIfNecessary()` and `AdjustMaxIfPossible()`, differing in the obvious way if the two values that they are comparing are equal. By choosing which variant to use in which situation, we can arrange to preferentially pick the new of equal-sized objects. Note that this still doesn't cause newer commits to be preferred in the paths of trees and blobs. [1] Note that there are also arguments that the old objects are more interesting, because they are more likely to indicate the commits where a large object was first introduced.
2017-12-11 19:05:52 +03:00
s.MaxHistoryDepth.AdjustMaxIfPossible(commitSize.MaxAncestorDepth)
if s.MaxParentCount.AdjustMaxIfPossible(parentCount) {
Add footnotes containing the paths to the tabular output For the non-JSON output, add footnote after the table listing the "biggest" objects referred to by the table, with bracketed citation numbers in the table indicating the appropriate footnote. If an object is "biggest" in more than one way, make the lines in the table refer to the same footnote.
2017-12-09 01:23:22 +03:00
setPath(g.pathResolver, &s.MaxParentCountCommit, oid, "commit")
}
sizes.go: move some definitions around
2017-10-30 05:44:06 +03:00
}
Rename `Oid` → `OID`
2018-02-26 08:39:53 +03:00
func (s *HistorySize) recordTag(g *Graph, oid git.OID, tagSize TagSize, size counts.Count32) {
Handle tag objects
2017-10-30 09:09:40 +03:00
s.UniqueTagCount.Increment(1)
Systematically prefer to cite more recent objects Given two objects that are the same size, or multiple trees/commits that refer to the same large object, which one should we prefer to cite in the footnotes? I think that new objects are more interesting than old objects [1], because * It is interesting to know whether the "bad" objects are still in use. * Once you have a commit containing a "bad" object, it is easier to trace it backwards in time than forwards. * New objects are more likely to be at the tips of branches and thus more likely to have a reference pointing at them. The old code was somewhat inconsistent because blobs and trees are processed in roughly reverse-chronological order whereas commits are processed in roughly chronological order. Improve the situation by implementing two pairs of `Count.AdjustMax()` methods, `AdjustMaxIfNecessary()` and `AdjustMaxIfPossible()`, differing in the obvious way if the two values that they are comparing are equal. By choosing which variant to use in which situation, we can arrange to preferentially pick the new of equal-sized objects. Note that this still doesn't cause newer commits to be preferred in the paths of trees and blobs. [1] Note that there are also arguments that the old objects are more interesting, because they are more likely to indicate the commits where a large object was first introduced.
2017-12-11 19:05:52 +03:00
if s.MaxTagDepth.AdjustMaxIfNecessary(tagSize.TagDepth) {
Report the paths to the worst objects It is frustrating to find out that there is, say, a gigantic blob in a repository with no easy way to find out where it is in the history. (Git's tooling for answering that question is almost nonexistent.) So... * Define a new class, `PathResolver`, that can compute the paths (i.e., the `git rev-parse` expression) that can be used to access an object with a specific OID. This class must be fed the objects and references in depth-first order, but that's OK because that's the order we need to process them to compute their expanded sizes. * Change `Sizes` to record `*Path` pointers, rather than OIDs, for the big objects that it has found. * If `Sizes` finds a new object that is bigger than the biggest that it has seen before, have it discard the (maybe only partly-computed) path of the old object and replace it with the new one. `PathResolver` knows how to free resources associated with the partly-computed paths, so basically the number of paths being computed at any time is at most one for each of the "largest" things that we want to track. * Output the paths (still in crude format) to the JSON output. This costs about 15% in runtime and hardly any additional memory consumption. Still to do: (1) Improve the format of the path output. (2) Emit better output for commits that are not referred to directly by references (perhaps using `git name-rev`). (3) Include the paths in non-JSON output, and probably change where they are presented in JSON format.
2017-12-08 19:14:47 +03:00
setPath(g.pathResolver, &s.MaxTagDepthTag, oid, "tag")
Keep track of the worst objects in several dimensions
2017-12-08 02:32:40 +03:00
}
Handle tag objects
2017-10-30 09:09:40 +03:00
}
Separate out a `git` package
2018-02-26 08:30:45 +03:00
func (s *HistorySize) recordReference(g *Graph, ref git.Reference) {
Keep a count of the number of references analyzed
2017-10-30 09:11:38 +03:00
s.ReferenceCount.Increment(1)
}
Graph: pass the reference group symbols along to `HistorySize`
2021-10-13 15:17:01 +03:00
func (s *HistorySize) recordReferenceGroup(g *Graph, group RefGroupSymbol) {
HistorySize: keep track of counts for each reference group
2021-10-13 15:29:39 +03:00
c, ok := s.ReferenceGroups[group]
if ok {
c.Increment(1)
} else {
n := counts.Count32(1)
s.ReferenceGroups[group] = &n
}
Graph: pass the reference group symbols along to `HistorySize`
2021-10-13 15:17:01 +03:00
}