The very next commit introduces a new feature to show git hunks, which
seems to have a negative impact on performance when moving the cursor
around, or adding/removing text within lines which are highlighted by
diff-hl while diff-hl-flydiff-mode is enabled.
So for now I'm pinning diff-hl to the commit right before this was
introduced.
Packages that need to be loaded on emacs startup, should just be loaded
through use-package in a non-deferred manner.
It makes no real difference to startup, loading the packages either
slows down emacs before "startup" is complete, or right after it
completes. End result is that Emacs is unresponsive for basically the
same amount of time regardless.