Project

General

Profile

Bug #1551

git index.lock issue

Added by Roland Schulz over 5 years ago. Updated over 2 years ago.

Status:
Feedback wanted
Priority:
Normal
Assignee:
-
Category:
Jenkins
Target version:
-
Affected version - extra info:
Affected version:
Close

Description

We still have the problem that the index.lock is sometimes left behind and then subsequent build fail. This usually happens on Windows and when the previous build crashes with "socket closed". An example is: http://jenkins.gromacs.org/job/Gromacs_Gerrit_master/5970/OPTIONS=Compiler=icc%20CompilerVersion=12.1.2%20GMX_GPU=OFF%20GMX_DLOPEN=OFF%20GMX_THREAD_MPI=OFF%20GMX_FFT_LIBRARY=fftpack%20CMAKE_BUILD_TYPE=Release%20host=bs_Win2008_64,label=bs_Win2008_64/consoleFull (socket closed) and http://jenkins.gromacs.org/job/Gromacs_Gerrit_master/5972/OPTIONS=Compiler=icc%20CompilerVersion=12.1.2%20GMX_GPU=OFF%20GMX_DLOPEN=OFF%20GMX_THREAD_MPI=OFF%20GMX_FFT_LIBRARY=fftpack%20CMAKE_BUILD_TYPE=Release%20host=bs_Win2008_64,label=bs_Win2008_64/consoleFull (next build using the same workspace which failed).

The reason for the socket closed is probably a reboot of the VM. I'm not sure we still do this for backup. If so that would be something we could tune but even than the VM could still reboot for other reasons. https://issues.jenkins-ci.org/browse/JENKINS-12513 tracks this bug for Jenkins but there is no progress there. We could either implement this ourselves in the git jenkins plugin or find some workaround.

History

#2 Updated by Mark Abraham over 5 years ago

If by construction we never share a workspace between two builds, can't we just delete the lock upon entry?

#3 Updated by Roland Schulz over 5 years ago

I think so. Maybe better would be to delete the whole workspace (in which case we automatically do a full clone). I had it at least once that the lock was there and the repository was corrupted. But just deleting the lock would cover at least 95% of the problems. BTW: We only have the problem because we do not always delete the workspace, because otherwise we put a lot of stress on gerrit (and make it slower) if we do a full clone everytime.

#4 Updated by Roland Schulz about 5 years ago

We today had a similar issue. The .git/config file was corrupted causing builds to fail. I think the only way to avoid any of those kind of errors is to use "Wipe out workspace before build" and then redownload git every time. This could be made faster by using "Use shallow clone". Also Gerrit Trigger 2.12.0-beta-3 allows to wait until replication is done. Thus we don't need to download from gerrit but could download from git.gromacs.org to avoid overloading gerrit. We could even replicate to all jenkins nodes and then download from a local folder. But that last option might have the same problem as the current solution. I assume the corrupted file today was caused by Windows restarting/crashing. If that happens during the replication that repository might get corrupted too and we might have to manually fix that too.

#5 Updated by Szilárd Páll over 2 years ago

  • Status changed from New to Feedback wanted

I have not seen this issue in a long time, I assume we can close this.

Also available in: Atom PDF