chore(release): 0.4.11

feat(builds): prepare for automated builds in jimeh/emacs-builds repo

CHANGELOG.md

@@ -2,6 +2,41 @@
 
 All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.
 
+### [0.4.11](https://github.com/jimeh/build-emacs-for-macos/compare/0.4.10...0.4.11) (2021-05-08)
+
+
+### Features
+
+* **builds:** update build script for new plan.yml format ([1df39fa](https://github.com/jimeh/build-emacs-for-macos/commit/1df39fafe62ada385aa1d92e6b7f591c16c0a80c))
+* **release:** initial attempt at providing automatic builds ([6328921](https://github.com/jimeh/build-emacs-for-macos/commit/63289216d70e496d664a7e3078dea5a82eb8f65d))
+
+
+### Bug Fixes
+
+* **release:** attempt to fix issue with talking to GitHub API ([272a300](https://github.com/jimeh/build-emacs-for-macos/commit/272a3000a1f96d8f131e684736127b923513a205))
+
+### [0.4.10](https://github.com/jimeh/build-emacs-for-macos/compare/0.4.9...0.4.10) (2021-04-25)
+
+
+### Bug Fixes
+
+* **cli:** correctly default to master branch if no git ref is given ([844df73](https://github.com/jimeh/build-emacs-for-macos/commit/844df73c8fa8440e657f7900ec89cdedb7c4c312))
+
+### [0.4.9](https://github.com/jimeh/build-emacs-for-macos/compare/0.4.8...0.4.9) (2021-04-08)
+
+
+### Bug Fixes
+
+* **cli:** default to "master" if no git ref is given ([e19a6a7](https://github.com/jimeh/build-emacs-for-macos/commit/e19a6a7bc24379292ee06ae4c805b8c5365f2d97)), closes [#35](https://github.com/jimeh/build-emacs-for-macos/issues/35)
+* **native_comp:** skip symlink creation for recent builds which do not need symlinks ([1000999](https://github.com/jimeh/build-emacs-for-macos/commit/1000999eb2673dc207a390ff3f902b9987b99173))
+
+### [0.4.8](https://github.com/jimeh/build-emacs-for-macos/compare/0.4.7...0.4.8) (2021-02-27)
+
+
+### Bug Fixes
+
+* **native_comp:** add support for new --with-native-compilation flag ([581594d](https://github.com/jimeh/build-emacs-for-macos/commit/581594da3cfbf1dd2fa28e91710b767e21ff75d2))
+
 ### [0.4.7](https://github.com/jimeh/build-emacs-for-macos/compare/0.4.6...0.4.7) (2021-02-21)
 
 

Makefile

@@ -1,9 +1,12 @@
 .PHONY: new-version
-new-version:
-	$(if $(shell which npx),,\
-		$(error No npx found in PATH, please install NodeJS))
-	$(if $(shell which standard-version),,\
-		$(error No standard-version found in PATH, install with: \
-			npm install -g standard-version))
-
+new-version: check-npx
 	npx standard-version
+
+.PHONY: next-version
+next-version: check-npx
+	npx standard-version --dry-run
+
+.PHONY: check-npx
+check-npx:
+	$(if $(shell which npx),,\
+		$(error No npx execuable found in PATH, please install NodeJS))

README.md

@@ -1,7 +1,8 @@
 # build-emacs-for-macos
 
 My personal hacked together script for building a completely self-contained
-Emacs.app application on macOS, from any git branch, tag, or ref.
+Emacs.app application on macOS, from any git branch, tag, or ref. With support
+for native-compilation.
 
 Use this script at your own risk.
 
@@ -18,7 +19,7 @@ Use this script at your own risk.
 
 ## Status
 
-As of writing (2021-01-15) it works for me on my machine. Your luck may vary.
+As of writing (2021-04-25) it works for me on my machine. Your luck may vary.
 
 I have successfully built:
 
@@ -29,8 +30,8 @@ I have successfully built:
 For reference, my machine is:
 
 - 13-inch MacBook Pro (2020), 10th-gen 2.3 GHz Quad-Core Intel Core i7 (4c/8t)
-- macOS Big Sur 11.1 (20C69)
-- Xcode 12.3 (12C33)
+- macOS Big Sur 11.2.3 (20D91)
+- Xcode 12.4 (12D4e)
 
 ## Limitations
 
@@ -92,8 +93,8 @@ trash the corresponding directory from the `sources` directory.
 
 ### Examples
 
-To download a tarball of the `master` branch (Emacs 28.x as of writing) and
-build Emacs.app from it:
+To download a tarball of the `master` branch (Emacs 28.x with native-compilation
+as of writing) and build Emacs.app from it:
 
 ```
 ./build-emacs-for-macos
@@ -136,24 +137,24 @@ tools seems to use it to figure out the path to Emacs' executable, including
 
 ## Native-Comp
 
-Building a Emacs.app with native-comp support
-([gccemacs](https://akrl.sdf.org/gccemacs.html)) from the `feature/native-comp`
-branch is now supported without much hassle thanks to the newly released
-`libgccjit` Homebrew formula.
+_Note: On 2021-04-25 the `feature/native-comp` branch was
+[merged](http://git.savannah.gnu.org/cgit/emacs.git/commit/?id=289000eee729689b0cf362a21baa40ac7f9506f6)
+into `master`._
 
-To build a Emacs.app with native compilation enabled, simply run:
-
-```
-./build-emacs-for-macos feature/native-comp
-```
+The build script will automatically detect if the source tree being built
+supports native-compilation, and enable it if available. You can override this
+to force it on/off by passing `--native-comp` or `--no-native-comp`
+respectfully.
 
 By default `NATIVE_FULL_AOT` is disabled which ensures a fast build by native
-compiling as few lisp source files as possible to build the app. Any remaining
-lisp files will be dynamically compiled in the background the first time you use
-them. To enable native full AoT, pass in the `--native-full-aot` option.
+compiling as few elisp source files as possible to build Emacs itself. Any
+remaining elisp files will be dynamically compiled in the background the first
+time they are used.
 
-On my machine it takes around 10 minutes to build Emacs.app with
-`NATIVE_FULL_AOT` disabled. With it enabled it takes around 20-25 minutes.
+To enable native full Ahead-of-Time compilation, pass in the `--native-full-aot`
+option, which will native-compile all of Emacs' elisp as built-time. On my
+machine it takes around 10 minutes to build Emacs.app with `NATIVE_FULL_AOT`
+disabled, and around 20-25 minutes with it enabled.
 
 ### Configuration
 

build-emacs-for-macos

@@ -10,7 +10,9 @@ require 'json'
 require 'net/http'
 require 'optparse'
 require 'pathname'
+require 'time'
 require 'uri'
+require 'yaml'
 
 class Error < StandardError; end
 
@@ -21,9 +23,9 @@ module Output
 
   def out(msg, newline: true)
     if newline
-      puts "==> #{msg}"
+      warn "==> #{msg}"
     else
-      print "==> #{msg}"
+      STDERR.print "==> #{msg}"
     end
   end
 
@@ -81,14 +83,16 @@ class Build
 
   def initialize(root_dir, ref = nil, options = {})
     @root_dir = root_dir
-    @ref = ref
+    @ref = ref || 'master'
     @options = options
     @gcc_info = GccInfo.new
   end
 
   def build
+    load_plan(options[:plan]) if options[:plan]
+
     unless meta[:sha] && meta[:date]
-      err 'Failed to get commit info from GitHub API.'
+      err 'Failed to get commit info from GitHub.'
     end
 
     tarball = download_tarball(meta[:sha])
@@ -108,6 +112,17 @@ class Build
 
   private
 
+  def load_plan(filename)
+    plan = YAML.safe_load(File.read(filename), [:Time])
+
+    @meta = {
+      sha: plan.dig('commit', 'sha'),
+      ref: plan.dig('commit', 'ref'),
+      date: plan.dig('commit', 'date')
+    }
+    @archive_filename = plan['archive']
+  end
+
   def tarballs_dir
     @tarballs_dir ||= File.join(root_dir, 'tarballs')
   end
@@ -180,14 +195,20 @@ class Build
     @configure_help
   end
 
-  def supports_native_comp?
-    @supports_native_comp ||= !!configure_help.match(/\s+--with-nativecomp\s+/)
-  end
-
   def supports_xwidgets?
     @supports_xwidgets ||= !!configure_help.match(/\s+--with-xwidgets\s+/)
   end
 
+  def supports_native_comp?
+    @supports_native_comp ||= !native_comp_configure_flag.nil?
+  end
+
+  def native_comp_configure_flag
+    @native_comp_configure_flag ||= configure_help.match(
+      /\s+(--with-native(?:comp|-compilation))\s+/
+    )&.[](1)
+  end
+
   def detect_native_comp
     info 'Detecting native-comp support: ', newline: false
     options[:native_comp] = supports_native_comp?
@@ -292,7 +313,7 @@ class Build
       if options[:xwidgets] && supports_xwidgets?
         configure_flags << '--with-xwidgets'
       end
-      configure_flags << '--with-nativecomp' if options[:native_comp]
+      configure_flags << native_comp_configure_flag if options[:native_comp]
       configure_flags << '--without-rsvg' unless options[:rsvg]
 
       run_cmd './configure', *configure_flags
@@ -308,17 +329,17 @@ class Build
       make_flags = []
       make_flags += ['-j', options[:parallel].to_s] if options[:parallel]
 
-      if options[:native_full_aot]
-        info 'Using NATIVE_FULL_AOT=1'
-        make_flags << 'NATIVE_FULL_AOT=1'
-        ENV.delete('NATIVE_FAST_BOOT')
-      else
-        ENV.delete('NATIVE_FULL_AOT')
-        ENV['NATIVE_FAST_BOOT'] = '1'
-      end
-
       if options[:native_comp]
         make_flags << "BYTE_COMPILE_EXTRA_FLAGS=--eval '(setq comp-speed 2)'"
+
+        if options[:native_full_aot]
+          info 'Using NATIVE_FULL_AOT=1'
+          make_flags << 'NATIVE_FULL_AOT=1'
+          ENV.delete('NATIVE_FAST_BOOT')
+        else
+          ENV.delete('NATIVE_FULL_AOT')
+          ENV['NATIVE_FAST_BOOT'] = '1'
+        end
       end
 
       run_cmd 'make', *make_flags
@@ -333,41 +354,58 @@ class Build
   def symlink_internals(app)
     return unless options[:native_comp]
 
-    info 'Creating symlinks within Emacs.app needed for native-comp'
-
     FileUtils.cd(File.join(app, 'Contents')) do
+      # Skip creation of symlinks if *.eln files are located under
+      # Resources/native-lisp. Emacs is capable of finding lisp sources and
+      # *.eln cache files without symlinks.
+      return if Dir['Resources/native-lisp/**/*.eln'].any?
+
+      info 'Creating symlinks within Emacs.app needed for native-comp'
+
       FileUtils.ln_s('Resources/lisp', 'lisp') unless File.exist?('lisp')
 
       source = Dir['MacOS/libexec/emacs/**/eln-cache',
                    'MacOS/lib/emacs/**/native-lisp'].first
+      err 'Failed to find native-lisp cache directory for symlink creation.'
+
       target = File.basename(source)
       FileUtils.ln_s(source, target) unless File.exist?(target)
     end
   end
 
-  def archive_app(app)
-    FileUtils.mkdir_p(builds_dir)
+  def archive_filename
+    return @archive_filename if @archive_filename
 
     metadata = [
       meta[:ref]&.gsub(/\W/, '-'),
-      meta[:date],
+      meta[:date]&.strftime('%Y-%m-%d'),
       meta[:sha][0..6],
       "macOS-#{OS.version}",
       OS.arch
     ].compact
 
     filename = "Emacs.app-[#{metadata.join('][')}].tbz"
-    target = "#{builds_dir}/#{filename}"
+    @archive_filename = File.join(builds_dir, filename)
+  end
+
+  def archive_app(app)
+    filename = File.basename(archive_filename)
+    target_dir = File.dirname(archive_filename)
+    relative_target_dir = target_dir.gsub(root_dir + '/', '')
+
+    FileUtils.mkdir_p(target_dir)
 
     app_base = File.basename(app)
     app_dir = File.dirname(app)
 
-    if !File.exist?(target)
-      info "Creating #{filename} archive in \"#{builds_dir}\"..."
-      FileUtils.cd(app_dir) { system('tar', '-cjf', target, app_base) }
+    if !File.exist?(archive_filename)
+      info "Creating #{filename} archive in \"#{relative_target_dir}\"..."
+      FileUtils.cd(app_dir) do
+        system('tar', '-cjf', archive_filename, app_base)
+      end
     else
       info "#{filename} archive exists in " \
-           "#{builds_dir.gsub(root_dir + '/', '')}, skipping archving."
+           "#{relative_target_dir}, skipping archving."
     end
   end
 
@@ -390,6 +428,7 @@ class Build
     return @meta if @meta
 
     ref_sha = options[:git_sha] || ref
+    info "Fetching info for git ref: #{ref_sha}"
     url = format(LATEST_URL, ref_sha)
     commit_json = http_get(url)
     err "Failed to get commit info about: #{ref_sha}" if commit_json.nil?
@@ -397,7 +436,7 @@ class Build
     commit = JSON.parse(commit_json)
     meta = {
       sha: commit['sha'],
-      date: Date.parse(commit['commit']['committer']['date'])
+      date: Time.parse(commit['commit']['committer']['date'])
     }
     meta[:ref] = ref if ref && ref[0..6] != meta[:sha][0..6]
 
@@ -663,6 +702,7 @@ class LibEmbedder < AbstractEmbedder
     mode = File.stat(file).mode
     File.chmod(0o775, file)
     yield
+  ensure
     File.chmod(mode, file)
   end
 end
@@ -797,6 +837,7 @@ end
 
 if __FILE__ == $PROGRAM_NAME
   cli_options = {
+    work_dir: File.expand_path(__dir__),
     native_full_aot: false,
     parallel: Etc.nprocessors,
     rsvg: false,
@@ -857,6 +898,19 @@ if __FILE__ == $PROGRAM_NAME
         cli_options[:no_frame_refocus] = true
       end
 
+      opts.on('--work-dir DIR',
+              'Specify a working directory where tarballs, sources, and ' \
+              'builds will be stored and worked with') do |v|
+        cli_options[:work_dir] = v
+      end
+
+      opts.on(
+        '--plan FILE',
+        'Follow given plan file, instead of using given git ref/sha'
+      ) do |v|
+        cli_options[:plan] = v
+      end
+
       opts.on('--[no-]native-fast-boot',
               'DEPRECATED: use --[no-]native-full-aot instead') do |v|
         if v
@@ -875,7 +929,8 @@ if __FILE__ == $PROGRAM_NAME
       end
     end.parse!
 
-    Build.new(File.expand_path(__dir__), ARGV.shift, cli_options).build
+    work_dir = cli_options.delete(:work_dir)
+    Build.new(work_dir, ARGV.shift, cli_options).build
   rescue Error => e
     warn "ERROR: #{e.message}"
     exit 1