You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: _posts/2023-09-07-add-files-to-git-lfs.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -18,7 +18,7 @@ image:
18
18
Git LFS can be very helpful to store large files along with your code in Git repositories. Git LFS artifacts are treated like any other file in Git, but the contents of the file are stored in a separate location. This allows you to store large files in Git without bloating the size of your Git repository. Tracking new files in Git LFS is easy, but there are a few steps you need to follow when using LFS for the first time.
19
19
20
20
> See my other Git LFS posts:
21
-
> -[Migrating Git Repos with LFS Artifacts](/posts/migrate-git-lfs-artifacts/)
21
+
> -[Migrating GitHub Repositories: Tackling Files in Git LFS](/posts/migrate-git-lfs-artifacts/)
22
22
> -[Migrating Large Files in Git to Git LFS](/posts/migrate-to-git-lfs/)
Git Large File Storage (LFS) replaces large files such as audio samples, videos, datasets, and graphics with text pointers inside Git, while storing the file contents on a remote server like GitHub. Git LFS works seamlessly, you hardly know it's there when interacting with Git. However, there are special instructions that you need to follow if you are migrating Git repositories that contain LFS artifacts.
18
+
Git Large File Storage (LFS) replaces large files such as audio samples, videos, datasets, and graphics with text pointers inside Git, while storing the file contents on a remote server like GitHub. Git LFS works seamlessly, you hardly know it's there when interacting with Git.
19
+
20
+
You can migrate repositories to GitHub using [GitHub Enterprise Importer (GEI)](https://docs.github.com/en/migrations/using-github-enterprise-importer/understanding-github-enterprise-importer/about-github-enterprise-importer), however, since Git LFS is an extension to Git, they are [not migrated](https://docs.github.com/en/migrations/using-github-enterprise-importer/migrating-between-github-products/about-migrations-between-github-products#data-that-is-not-migrated) automatically. There are special instructions that you need to follow if you are migrating Git repositories that contain LFS objects.
21
+
22
+
This post highlights two methods to migrate Git LFS objects from one repository to another:
23
+
24
+
-[Method 1: Using `git lfs push --all` to migrate all LFS objects](#method-1-git-lfs-push---all)
25
+
-[Method 2: Methodically pushing each LFS object](#method-2-methodically-pushing-each-object) (slower but more reliable for large repos and instances with higher network latency)
19
26
20
27
> See my other Git LFS posts:
21
28
> -[Migrating Large Files in Git to Git LFS](/posts/migrate-to-git-lfs/)
@@ -29,50 +36,87 @@ Git Large File Storage (LFS) replaces large files such as audio samples, videos,
29
36
- On Windows, `git lfs` should be installed with [Git for Windows](https://gitforwindows.org/), but you can install an updated version via `choco install git-lfs`
30
37
2. Once installed, you need to run `git lfs install` to configure Git hooks to use LFS.
31
38
32
-
## Migrating a Repository with LFS Artifacts
39
+
## Migrating a Repository with LFS Objects
40
+
41
+
No matter which method you choose, if you are using GEI to migrate the repository, you should run the migration first. This will migrate the Git repository, including the text pointers for the LFS objects, as well metadata such as pull requests and issues. After the migration is complete, you can then run one of the methods below to migrate the LFS objects.
42
+
43
+
> If you use GEI, I recommend waiting for the migration to fully complete before pushing LFS objects. However, if time is critical, you can proceed once the repository and code are visible in GitHub.
44
+
{: .prompt-tip }
33
45
34
-
If you are migrating a repository that contains LFS artifacts, you need to make sure that you migrate the LFS artifacts as well. If you don't, you will end up with a repository that contains text pointers to files that don't exist. This is because the LFS artifacts are stored in a separate location from the repository itself.
46
+
### Method 1: git lfs push --all
47
+
48
+
If you are migrating a repository that contains LFS objects, you need to make sure that you migrate the LFS objects as well. If you don't, you will end up with a repository that contains text pointers to files that don't exist. This is because the LFS objects are stored in a separate location from the repository itself.
35
49
36
50
Instructions:
37
51
38
52
```bash
39
53
git clone --mirror <source-url> temp
40
54
cd temp
41
-
git push --mirror <target-url>
55
+
#git push --mirror <target-url> # or migrate with GEI
42
56
git lfs fetch --all
43
57
git lfs push <target-url> --all
44
58
```
59
+
{: .nolineno}
45
60
46
-
For Git repos with LFS, the two `git lfs` commands are key. This `git lfs fetch --all` command will download all the LFS artifacts from the source repository and store them in the target repository. The `git lfs push <target-url> --all` command will push all of the LFS artifacts to the target repository. After the push, the text pointers will be updated - including the same commit hash!
61
+
For Git repos with LFS, the two `git lfs` commands are key. This `git lfs fetch --all` command will download all the LFS objects from the source repository and store them in the target repository. The `git lfs push <target-url> --all` command will push all of the LFS objects to the target repository. After the push, the text pointers will be updated - including the same commit hash!
47
62
48
-
Here is an example of migrating a repository that contains LFS artifacts:
63
+
Here is an example of migrating a repository that contains LFS objects:
The method above worked well for me in my tests, even in tests with 1,000 LFS artifacts each 1mb in size. However, I have seen issues running these commands for some customers with larger repositories. Sometimes the `git lfs fetch` or `git lfs push` will hang or not appear to migrate all of the LFS artifacts. If you run into issues, you can run this script to more thoroughly (but slower) way to migrate the LFS artifacts. It works by looping through each LFS object and pushing it individually pushing it.
74
+
The method above worked well for me in my tests, even in tests with 1,000 LFS objects each 1mb in size. However, I have seen issues running these commands for some customers with larger repositories. Sometimes the `git lfs fetch` or `git lfs push` will hang or not appear to migrate all of the LFS objects. If you run into issues, you can run this script a more thorough (but slower) way to migrate the LFS objects. It works by looping through each LFS object and individually pushing it.
60
75
61
-
The alternative script to migrate the LFS artifacts to another repository:
76
+
The alternative script to migrate the LFS objects to another repository:
> Parallelization is much harder to debug! I would recommend starting with the non-parallel version first and only use this sample if you had an extremely large number of LFS objects and are comfortable debugging any issues that might arise.
102
+
{: .prompt-warning }
103
+
104
+
## Other Tips
105
+
106
+
To speed up the migration process, you can prioritize the main branch since developers typically work with LFS files on the default branch most often. Instead of migrating all LFS objects at once, migrate the main branch first, then handle other branches as needed. You can do this by omitting the `--all` flag to only migrate LFS objects reachable from the current branch:
107
+
108
+
```bash
109
+
git clone --mirror <source-url> temp
110
+
cd temp
111
+
# git push --mirror <target-url> # or migrate with GEI
112
+
git lfs fetch
113
+
git lfs push <target-url>
114
+
# now use the `--all` option or for loop to migrate all other reachable LFS objects
115
+
```
116
+
{: .nolineno}
117
+
74
118
## Summary
75
119
76
-
Migrating Git repositories is simple. However, if the repository contains LFS artifacts, you need to make sure to run the `git lfs fetch` and `git lfs push` commands to migrate them along with the repository. If you don't, your Git LFS files will be missing.
120
+
Migrating Git repositories is simple. However, if the repository contains LFS objects, you need to make sure to run the `git lfs fetch` and `git lfs push` commands to migrate them along with the repository. If you don't, your Git LFS files will be missing.
77
121
78
-
Now that your Git repo, including LFS artifacts, is migrated, you can delete the temp folder, update remotes on your local repo, and continue working as normal! 🚀
122
+
Now that your Git repo, including LFS objects, is migrated, you can delete the temp folder (`cd .. && rm -rf temp`), update remotes on your local repo (`git remote remove origin` / `git remote add origin <url>`), and continue working as normal! 🚀
0 commit comments