chore(release): 0.4.14

feat(cli): add "emacs" CLI launcher script to Emacs.app/Conents/MacOS/bin

CHANGELOG.md

@@ -2,6 +2,42 @@
 
 All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.
 
+### [0.4.14](https://github.com/jimeh/build-emacs-for-macos/compare/0.4.13...0.4.14) (2021-05-22)
+
+
+### Features
+
+* **cli:** add "emacs" CLI launcher script to Emacs.app/Conents/MacOS/bin ([8237aa9](https://github.com/jimeh/build-emacs-for-macos/commit/8237aa9272ce1d13a412b2495cbaa90df38d928b)), closes [#41](https://github.com/jimeh/build-emacs-for-macos/issues/41)
+
+### [0.4.13](https://github.com/jimeh/build-emacs-for-macos/compare/0.4.12...0.4.13) (2021-05-22)
+
+
+### Bug Fixes
+
+* **codesign:** prevent "bundle format unrecognized" error from codesign ([7259111](https://github.com/jimeh/build-emacs-for-macos/commit/7259111478ecb838dea9c8f50ea39eafdf47ed5a))
+* **embed:** avoid potential error caused by trying to set duplicate rpath ([bb45cda](https://github.com/jimeh/build-emacs-for-macos/commit/bb45cda0231e99618571dc835348cf5c3345e277))
+
+### [0.4.12](https://github.com/jimeh/build-emacs-for-macos/compare/0.4.11...0.4.12) (2021-05-17)
+
+
+### Bug Fixes
+
+* **shared-libraries:** stop aggressive dylib re-linking ([0a22d83](https://github.com/jimeh/build-emacs-for-macos/commit/0a22d8393c53305354c4c6d8e784e7d59caa039a)), closes [#12](https://github.com/jimeh/build-emacs-for-macos/issues/12)
+* **svg:** enable SVG by default via librsvg ([bf7c4d5](https://github.com/jimeh/build-emacs-for-macos/commit/bf7c4d5debf32980dbbabc1ea99b58b266390011))
+
+### [0.4.11](https://github.com/jimeh/build-emacs-for-macos/compare/0.4.10...0.4.11) (2021-05-08)
+
+
+### Features
+
+* **builds:** update build script for new plan.yml format ([1df39fa](https://github.com/jimeh/build-emacs-for-macos/commit/1df39fafe62ada385aa1d92e6b7f591c16c0a80c))
+* **release:** initial attempt at providing automatic builds ([6328921](https://github.com/jimeh/build-emacs-for-macos/commit/63289216d70e496d664a7e3078dea5a82eb8f65d))
+
+
+### Bug Fixes
+
+* **release:** attempt to fix issue with talking to GitHub API ([272a300](https://github.com/jimeh/build-emacs-for-macos/commit/272a3000a1f96d8f131e684736127b923513a205))
+
 ### [0.4.10](https://github.com/jimeh/build-emacs-for-macos/compare/0.4.9...0.4.10) (2021-04-25)
 
 

README.md

@@ -72,14 +72,14 @@ available here: https://github.com/emacs-mirror/emacs
 Options:
     -j, --parallel COUNT             Compile using COUNT parallel processes (detected: 8)
         --git-sha SHA                Override detected git SHA of specified branch allowing builds of old commits
-        --[no-]xwidgets              Enable/disable XWidgets (default: enabled if supported)
+        --[no-]xwidgets              Enable/disable XWidgets if supported (default: enabled)
         --[no-]native-comp           Enable/disable native-comp (default: enabled if supported)
         --[no-]native-full-aot       Enable/disable NATIVE_FULL_AOT / Ahead of Time compilation (default: disabled)
-        --rsvg                       Enable SVG image support via librsvg, can yield a unstable build (default: disabled)
+        --[no-]rsvg                  Enable/disable SVG image support via librsvg (default: enabled)
         --no-titlebar                Apply no-titlebar patch (default: disabled)
         --no-frame-refocus           Apply no-frame-refocus patch (default: disabled)
-        --[no-]native-fast-boot      DEPRECATED: use --[no-]native-full-aot instead
-        --[no-]launcher              DEPRECATED: Launcher script is no longer used.
+        --work-dir DIR               Specify a working directory where tarballs, sources, and builds will be stored and worked with
+        --plan FILE                  Follow given plan file, instead of using given git ref/sha
 ```
 
 Resulting applications are saved to the `builds` directory in a bzip2 compressed
@@ -111,29 +111,25 @@ All sources as downloaded as tarballs from the
 to get a list of tags/branches available to install, simply check said
 repository.
 
-## Use Self-Contained Emacs.app as `emacs` CLI Tool
+## Use Emacs.app as `emacs` CLI Tool
 
-As the application bundle is self-contained, the main executable needs to be run
-from within the application bundle. This means a simple symlink to
-`Emacs.app/Contents/MacOS/Emacs` will not work. Instead the best approach is to
-create a shell alias called `emacs` pointing to the right place.
+Builds come with a custom `emacs` shell script launcher for use from the command
+line. It makes sure to use the main `Emacs.app/Contents/MacOS/Emacs` executable
+from the correct path, ensuring it finds all the relevant dependencies within
+the Emacs.app bundle.
 
-Personally I use something similar to this:
+To use it, simply add `Emacs.app/Contents/MacOS/bin` to your `PATH`. For
+example, if you place Emacs.app in `/Applications`:
 
 ```bash
-if [ -f "/Applications/Emacs.app/Contents/MacOS/Emacs" ]; then
-  export EMACS="/Applications/Emacs.app/Contents/MacOS/Emacs"
-  alias emacs="$EMACS -nw"
-fi
-
-if [ -f "/Applications/Emacs.app/Contents/MacOS/bin/emacsclient" ]; then
-  alias emacsclient="/Applications/Emacs.app/Contents/MacOS/bin/emacsclient"
+if [ -d "/Applications/Emacs.app/Contents/MacOS/bin" ]; then
+  export PATH="/Applications/Emacs.app/Contents/MacOS/bin:$PATH"
+  alias emacs="emacs -nw" # Always launch "emacs" in terminal mode.
 fi
 ```
 
-Setting the `EMACS` variable to the binary path seems to be a good idea, as some
-tools seems to use it to figure out the path to Emacs' executable, including
-[doom-emacs](https://github.com/hlissner/doom-emacs)' `doom` CLI tool.
+If you want `emacs` in your terminal to launch a GUI instance of Emacs, don't
+use the alias from the above example.
 
 ## Native-Comp
 
@@ -142,9 +138,9 @@ _Note: On 2021-04-25 the `feature/native-comp` branch was
 into `master`._
 
 The build script will automatically detect if the source tree being built
-supports native-compilation, and enable it if available. You can override this
-to force it on/off by passing `--native-comp` or `--no-native-comp`
-respectfully.
+supports native-compilation, and enable it if available. You can override the
+auto-detection logic to force enable or force disable native-compilation by
+passing `--native-comp` or `--no-native-comp` respectfully.
 
 By default `NATIVE_FULL_AOT` is disabled which ensures a fast build by native
 compiling as few elisp source files as possible to build Emacs itself. Any
@@ -152,32 +148,39 @@ remaining elisp files will be dynamically compiled in the background the first
 time they are used.
 
 To enable native full Ahead-of-Time compilation, pass in the `--native-full-aot`
-option, which will native-compile all of Emacs' elisp as built-time. On my
+option, which will native-compile all of Emacs' elisp at built-time. On my
 machine it takes around 10 minutes to build Emacs.app with `NATIVE_FULL_AOT`
 disabled, and around 20-25 minutes with it enabled.
 
 ### Configuration
 
-Add the following near the top of your `early-init.el` or `init.el`:
-
-```elisp
-(setq comp-speed 2)
-```
+#### Native-Lisp Cache Directory
 
 By default natively compiled `*.eln` files will be cached in
 `~/.emacs.d/eln-cache/`. If you want to customize that, simply set a new path as
-the first element of the `comp-eln-load-path` variable. The path string must end
-with a `/`.
+the first element of the `native-comp-eln-load-path` variable. The path string
+must end with a `/`.
 
 Below is an example which stores all compiled `*.eln` files in `cache/eln-cache`
 within your Emacs configuration directory:
 
 ```elisp
-(when (boundp 'comp-eln-load-path)
-  (setcar comp-eln-load-path
+(when (boundp 'native-comp-eln-load-path)
+  (setcar native-comp-eln-load-path
           (expand-file-name "cache/eln-cache/" user-emacs-directory)))
 ```
 
+#### Compilation Warnings
+
+By default any warnings encountered during async native compilation will pop up
+a warnings buffer. As this tends to happen rather frequently with a lot of
+packages, it can get annoying. You can disable showing these warnings by setting
+`native-comp-async-report-warnings-errors` to `nil`:
+
+```elisp
+(setq native-comp-async-report-warnings-errors nil)
+```
+
 ### Issues
 
 Please see all issues with the
@@ -188,7 +191,7 @@ types of issues and or behavior you can expect.
 ### Known Good Commits/Builds
 
 A list of known "good" commits which produce working builds is tracked in:
-[#6 Known good commits of feature/native-comp branch](https://github.com/jimeh/build-emacs-for-macos/issues/6)
+[#6 Known good commits for native-comp](https://github.com/jimeh/build-emacs-for-macos/issues/6)
 
 ## Credits
 

build-emacs-for-macos

@@ -10,7 +10,9 @@ require 'json'
 require 'net/http'
 require 'optparse'
 require 'pathname'
+require 'time'
 require 'uri'
+require 'yaml'
 
 class Error < StandardError; end
 
@@ -21,9 +23,9 @@ module Output
 
   def out(msg, newline: true)
     if newline
-      puts "==> #{msg}"
+      warn "==> #{msg}"
     else
-      print "==> #{msg}"
+      STDERR.print "==> #{msg}"
     end
   end
 
@@ -87,8 +89,10 @@ class Build
   end
 
   def build
+    load_plan(options[:plan]) if options[:plan]
+
     unless meta[:sha] && meta[:date]
-      err 'Failed to get commit info from GitHub API.'
+      err 'Failed to get commit info from GitHub.'
     end
 
     tarball = download_tarball(meta[:sha])
@@ -99,6 +103,7 @@ class Build
 
     app = compile_source(@source_dir)
     symlink_internals(app)
+    add_cli_helper(app)
 
     LibEmbedder.new(app, brew_dir, extra_libs).embed
     GccLibEmbedder.new(app, gcc_info).embed if options[:native_comp]
@@ -108,6 +113,17 @@ class Build
 
   private
 
+  def load_plan(filename)
+    plan = YAML.safe_load(File.read(filename), [:Time])
+
+    @meta = {
+      sha: plan.dig('commit', 'sha'),
+      ref: plan.dig('commit', 'ref'),
+      date: plan.dig('commit', 'date')
+    }
+    @archive_filename = plan['archive']
+  end
+
   def tarballs_dir
     @tarballs_dir ||= File.join(root_dir, 'tarballs')
   end
@@ -299,7 +315,7 @@ class Build
         configure_flags << '--with-xwidgets'
       end
       configure_flags << native_comp_configure_flag if options[:native_comp]
-      configure_flags << '--without-rsvg' unless options[:rsvg]
+      configure_flags << '--without-rsvg' if options[:rsvg] == false
 
       run_cmd './configure', *configure_flags
 
@@ -358,29 +374,51 @@ class Build
     end
   end
 
-  def archive_app(app)
-    FileUtils.mkdir_p(builds_dir)
+  def add_cli_helper(app)
+    source = File.join(__dir__, 'helper', 'emacs-cli.bash')
+    target = File.join(app, 'Contents', 'MacOS', 'bin', 'emacs')
+    dir = File.dirname(target)
+
+    info "Adding \"emacs\" CLI helper to #{dir}"
+
+    FileUtils.mkdir_p(dir)
+    FileUtils.cp(source, target)
+    FileUtils.chmod('+w', target)
+  end
+
+  def archive_filename
+    return @archive_filename if @archive_filename
 
     metadata = [
       meta[:ref]&.gsub(/\W/, '-'),
-      meta[:date],
+      meta[:date]&.strftime('%Y-%m-%d'),
       meta[:sha][0..6],
       "macOS-#{OS.version}",
       OS.arch
     ].compact
 
     filename = "Emacs.app-[#{metadata.join('][')}].tbz"
-    target = "#{builds_dir}/#{filename}"
+    @archive_filename = File.join(builds_dir, filename)
+  end
+
+  def archive_app(app)
+    filename = File.basename(archive_filename)
+    target_dir = File.dirname(archive_filename)
+    relative_target_dir = target_dir.gsub(root_dir + '/', '')
+
+    FileUtils.mkdir_p(target_dir)
 
     app_base = File.basename(app)
     app_dir = File.dirname(app)
 
-    if !File.exist?(target)
-      info "Creating #{filename} archive in \"#{builds_dir}\"..."
-      FileUtils.cd(app_dir) { system('tar', '-cjf', target, app_base) }
+    if !File.exist?(archive_filename)
+      info "Creating #{filename} archive in \"#{relative_target_dir}\"..."
+      FileUtils.cd(app_dir) do
+        system('tar', '-cjf', archive_filename, app_base)
+      end
     else
       info "#{filename} archive exists in " \
-           "#{builds_dir.gsub(root_dir + '/', '')}, skipping archving."
+           "#{relative_target_dir}, skipping archving."
     end
   end
 
@@ -411,7 +449,7 @@ class Build
     commit = JSON.parse(commit_json)
     meta = {
       sha: commit['sha'],
-      date: Date.parse(commit['commit']['committer']['date'])
+      date: Time.parse(commit['commit']['committer']['date'])
     }
     meta[:ref] = ref if ref && ref[0..6] != meta[:sha][0..6]
 
@@ -560,7 +598,7 @@ class AbstractEmbedder
   private
 
   def invocation_dir
-    File.join(app, 'Contents/MacOS')
+    File.join(app, 'Contents', 'MacOS')
   end
 
   def bin
@@ -592,7 +630,6 @@ class LibEmbedder < AbstractEmbedder
     FileUtils.cd(File.dirname(app)) do
       copy_libs(binary)
       copy_extra_libs(extra_libs, binary) if extra_libs.any?
-      self_ref_libs(binary)
     end
   end
 
@@ -604,6 +641,16 @@ class LibEmbedder < AbstractEmbedder
       Pathname.new(File.dirname(exe))
     ).to_s
 
+    rpath = File.join('@executable_path', rel_path)
+    rpaths = `otool -l "#{exe}" | grep -A 2 'cmd LC_RPATH' | grep 'path'`
+
+    unless rpaths.include?(rpath)
+      while_writable(exe) do
+        system('install_name_tool', '-add_rpath',
+               File.join('@executable_path', rel_path), exe)
+      end
+    end
+
     `otool -L "#{exe}"`.split("\n")[1..-1].each do |line|
       match = line.match(%r{^\s+(.+/(lib[^/ ]+))\s})
       next unless match && match[1].start_with?(lib_source)
@@ -648,31 +695,6 @@ class LibEmbedder < AbstractEmbedder
     end
   end
 
-  def self_ref_libs(exe)
-    rel_path = Pathname.new(lib_dir).relative_path_from(
-      Pathname.new(File.dirname(exe))
-    ).to_s
-    lib_paths ||= Dir.glob("#{lib_dir}/*").select { |f| File.file?(f) }
-    libs = lib_paths.map { |f| File.basename(f) }
-
-    ([exe] + lib_paths).each do |bin_path|
-      `otool -L "#{bin_path}"`.split("\n")[1..-1].each do |line|
-        match = line.match(%r{^\s+(.+/(lib[^/ ]+))\s})
-        next unless match
-        next if match[1].start_with?('@executable_path/')
-        next unless libs.include?(match[2])
-
-        while_writable(bin_path) do
-          system(
-            'install_name_tool', '-change', match[1],
-            File.join('@executable_path', rel_path, match[2].to_s),
-            bin_path
-          )
-        end
-      end
-    end
-  end
-
   def while_writable(file)
     mode = File.stat(file).mode
     File.chmod(0o775, file)
@@ -704,7 +726,9 @@ class GccLibEmbedder < AbstractEmbedder
 
     FileUtils.mkdir_p(File.dirname(target_dir))
     FileUtils.cp_r(source_dir, target_dir)
-    FileUtils.rm(Dir[File.join(target_dir, '**/.DS_Store')], force: true)
+    FileUtils.rm(Dir[File.join(target_dir, '**', '.DS_Store')], force: true)
+    FileUtils.chmod_R('u+w', target_dir)
+    FileUtils.mv(source_darwin_dir, target_darwin_dir)
   end
 
   private
@@ -717,9 +741,21 @@ class GccLibEmbedder < AbstractEmbedder
     File.join(invocation_dir, gcc_info.relative_lib_dir)
   end
 
+  def source_darwin_dir
+    File.join(invocation_dir, gcc_info.relative_darwin_lib_dir)
+  end
+
+  def target_darwin_dir
+    File.join(invocation_dir, gcc_info.sanitized_relative_darwin_lib_dir)
+  end
+
   def source_dir
     gcc_info.lib_dir
   end
+
+  def relative_dir(path, root)
+    Pathname.new(path).relative_path_from(Pathname.new(root)).to_s
+  end
 end
 
 class GccInfo
@@ -759,6 +795,17 @@ class GccInfo
     @relative_darwin_lib_dir ||= relative_dir(darwin_lib_dir, root_dir)
   end
 
+  # Sanitize folder name with full "MAJOR.MINOR.PATCH" version number to just
+  # the MAJOR version. Apple's codesign CLI tool throws a "bundle format
+  # unrecognized" error if there are any folders with two dots in their name
+  # within the Emacs.app application bundle.
+  def sanitized_relative_darwin_lib_dir
+    @sanitized_relative_darwin_lib_dir ||= File.join(
+      File.dirname(relative_darwin_lib_dir),
+      File.basename(relative_darwin_lib_dir).split('.').first
+    )
+  end
+
   def libgccjit_root_dir
     @libgccjit_root_dir ||= `brew --prefix libgccjit`.chomp
   end
@@ -803,18 +850,16 @@ class GccInfo
   private
 
   def relative_dir(path, root)
-    root += '/' unless root[-1] == '/'
-    return if path[0..root.size - 1] != root
-
-    path[root.size..-1]
+    Pathname.new(path).relative_path_from(Pathname.new(root)).to_s
   end
 end
 
 if __FILE__ == $PROGRAM_NAME
   cli_options = {
+    work_dir: File.expand_path(__dir__),
     native_full_aot: false,
     parallel: Etc.nprocessors,
-    rsvg: false,
+    rsvg: true,
     xwidgets: true
   }
 
@@ -835,14 +880,15 @@ if __FILE__ == $PROGRAM_NAME
         cli_options[:parallel] = v
       end
 
-      opts.on('--git-sha SHA', 'Override detected git SHA of specified ' \
-                               'branch allowing builds of old commits') do |v|
+      opts.on('--git-sha SHA',
+              'Override detected git SHA of specified ' \
+              'branch allowing builds of old commits') do |v|
         cli_options[:git_sha] = v
       end
 
       opts.on('--[no-]xwidgets',
-              'Enable/disable XWidgets ' \
-              '(default: enabled if supported)') do |v|
+              'Enable/disable XWidgets if supported ' \
+              '(default: enabled)') do |v|
         cli_options[:xwidgets] = v
       end
 
@@ -858,9 +904,10 @@ if __FILE__ == $PROGRAM_NAME
         cli_options[:native_full_aot] = v
       end
 
-      opts.on('--rsvg', 'Enable SVG image support via librsvg, ' \
-                        'can yield a unstable build (default: disabled)') do
-        cli_options[:rsvg] = true
+      opts.on('--[no-]rsvg',
+              'Enable/disable SVG image support via librsvg ' \
+              '(default: enabled)') do |v|
+        cli_options[:rsvg] = v
       end
 
       opts.on('--no-titlebar', 'Apply no-titlebar patch (default: disabled)') do
@@ -872,25 +919,22 @@ if __FILE__ == $PROGRAM_NAME
         cli_options[:no_frame_refocus] = true
       end
 
-      opts.on('--[no-]native-fast-boot',
-              'DEPRECATED: use --[no-]native-full-aot instead') do |v|
-        if v
-          raise Error, '--native-fast-boot option is deprecated, ' \
-                       'use --no-native-full-aot instead'
-        else
-          raise Error, '--no-native-fast-boot option is deprecated, ' \
-                       'use --native-full-aot instead'
-        end
+      opts.on('--work-dir DIR',
+              'Specify a working directory where tarballs, sources, and ' \
+              'builds will be stored and worked with') do |v|
+        cli_options[:work_dir] = v
       end
 
-      opts.on('--[no-]launcher',
-              'DEPRECATED: Launcher script is no longer used.') do |_|
-        raise Error, '--[no-]launcher option is deprecated, launcher ' \
-                     'script is no longer used.'
+      opts.on(
+        '--plan FILE',
+        'Follow given plan file, instead of using given git ref/sha'
+      ) do |v|
+        cli_options[:plan] = v
       end
     end.parse!
 
-    Build.new(File.expand_path(__dir__), ARGV.shift, cli_options).build
+    work_dir = cli_options.delete(:work_dir)
+    Build.new(work_dir, ARGV.shift, cli_options).build
   rescue Error => e
     warn "ERROR: #{e.message}"
     exit 1

helper/emacs-cli.bash

@@ -0,0 +1,23 @@
+#!/usr/bin/env bash
+
+resolve_link() {
+  "$(type -p greadlink readlink | head -1)" "$1"
+}
+
+abs_dirname() {
+  local path="$1"
+  local name
+  local cwd
+  cwd="$(pwd)"
+
+  while [ -n "$path" ]; do
+    cd "${path%/*}" || exit 1
+    name="${path##*/}"
+    path="$(resolve_link "$name" || true)"
+  done
+
+  pwd
+  cd "$cwd" || exit 1
+}
+
+exec "$(dirname "$(abs_dirname "$0")")/Emacs" "$@"

patches/native-comp-env-setup.diff.erb

@@ -1,10 +1,10 @@
 diff --git a/lisp/emacs-lisp/comp.el b/lisp/emacs-lisp/comp.el
-index 4036080976..2ff8dbd74c 100644
+index 8c638312b0..87af889ef4 100644
 --- a/lisp/emacs-lisp/comp.el
 +++ b/lisp/emacs-lisp/comp.el
-@@ -4079,6 +4079,52 @@ of (commands) to run simultaneously."
+@@ -4215,6 +4215,52 @@ native-compile-async
    (let ((load (not (not load))))
-     (native--compile-async paths recursively load selector)))
+     (native--compile-async files recursively load selector)))
  
 +;;;###autoload
 +(defun native-compile-setup-environment-variables (&rest _args)
@@ -19,7 +19,7 @@ index 4036080976..2ff8dbd74c 100644
 +                     "<%= relative_lib_dir %>"
 +                     invocation-directory))
 +           (darwin-dir (expand-file-name
-+                        "<%= relative_darwin_lib_dir %>"
++                        "<%= sanitized_relative_darwin_lib_dir %>"
 +                        invocation-directory))
 +           (lib-paths (list)))
 +
@@ -54,4 +54,4 @@ index 4036080976..2ff8dbd74c 100644
 +
  (provide 'comp)
  
- ;;; comp.el ends here
+ ;; LocalWords: limplified limplified limplification limplify Limple LIMPLE libgccjit elc eln