Part 37: Markdown It, Caching and Automatic Commit Links

More devblog
Close up photo of keyboard keys.
| 'TYPE' by SarahDeer is licensed with CC BY 2.0 |

Project Scope and ToDos

  1. Static Site Generator that can build the blog and let me host it on Github Pages
  2. I want to write posts in Markdown because I'm lazy, it's easy, and it is how I take notes now.
  3. I don't want to spend a ton of time doing design work. I'm doing complicated designs for other projects, so I want to pull a theme I like that I can rely on someone else to keep up.
  4. Once it gets going, I want template changes to be easy.
  5. It should be as easy as Jekyll, so I need to be able to build it using GitHub Actions, where I can just commit a template change or Markdown file and away it goes. If I can't figure this out than fk it, just use Jekyll.
  6. I require it to be used by a significant percent of my professional peers so I can get easy answers when something goes wrong.
  7. I want source maps. This is a dev log site which means whatever I do with it should be easy for other developers to read.
  • Can I use the template inside of dinky that already exists instead of copy/pasting it?
  • Is there a way to have permalinks to posts contain metadata without organizing them into subfolders?
  • How do I cachebreak files on the basis of new build events? Datetime? site.github.build_revision is how Jekyll accomplishes this, but is there a way to push that into the build process for Eleventy?

  • Make link text look less shitty. It looks like it is a whole, lighter, font.

  • Code blocks do not have good syntax highlighting. I want good syntax highlighting.

  • Build a Markdown-it plugin to take my typing shortcuts [prob, b/c, ...?] and expand them on build.

  • See if we can start Markdown's interpretation of H tags to start at 2, since H1 is always pulled from the page title metadata. If it isn't easy, I just have to change my pattern of writing in the MD documents.

  • Should I explore some shortcodes?

  • Order projects listing by last posted blog in that project

  • Limit the output of home page post lists to a specific number of posts

  • Show the latest post below the site intro on the homepage.

  • Tags pages with Pagination

  • Posts should be able to support a preview header image that can also be shown on post lists.

  • Create a Markdown-It plugin that reads the project's repo URL off the folder data file and renders commit messages with links to the referenced commit. (Is this even possible?) (Is there a way to do it with eleventy instead?)

  • Create Next Day/Previous Day links on each post / Next/Previous post on post templates from projects

  • Tags should be in the sidebar of articles and link to tag pages

  • Create a skiplink for the todo section (or would this be better served with the ToC plugin?) - Yes it would be!

  • Add a Things I Learned section to the project pages that are the things I learned from that specific project.

  • Add a technical reading log to the homepage

  • Hide empty sections.

  • Add byline to post pages

  • Have table of contents attach to sidebar bottom on mobile

  • Support dark mode

  • Social Icons

  • SEO/Social/JSON-LD HEAD data

Day 37

Ok,so I need to apply the rules properly.

First, let's see what rules exist:



Ruler {
__rules__: [
name: 'normalize',
enabled: true,
fn: [Function: normalize],
alt: []
{ name: 'block', enabled: true, fn: [Function: block], alt: [] },
{ name: 'inline', enabled: true, fn: [Function: inline], alt: [] },
name: 'short-phrase-fixer',
enabled: true,
fn: [Function (anonymous)],
alt: []
name: 'evernote-todo',
enabled: true,
fn: [Function (anonymous)],
alt: []
name: 'replace-link',
enabled: true,
fn: [Function (anonymous)],
alt: []
name: 'linkify',
enabled: true,
fn: [Function: linkify],
alt: []
name: 'replacements',
enabled: true,
fn: [Function: replace],
alt: []
name: 'smartquotes',
enabled: true,
fn: [Function: smartquotes],
alt: []
name: 'anchor',
enabled: true,
fn: [Function (anonymous)],
alt: []
__cache__: null

Finding the inline token

Ok, so I need to set up a token type starting with inline. Let's try and match the right token:

const testPattern = /(?<=git commit \-am [\"|\'])(.*)(?=[\"|\'])/i;

// console.dir(md.core.ruler)
md.core.ruler.after('inline', 'git_commit', state => {
const tokens = state.tokens
for (let i = 0; i < tokens.length; i++) {
if (testPattern.test(tokens[i].content)) {
console.log('tokens round 1: ', tokens[i])


 Token {
type: 'inline',
tag: '',
attrs: null,
map: [ 143, 144 ],
nesting: 0,
level: 1,
children: [
Token {
type: 'code_inline',
tag: 'code',
attrs: null,
map: null,
nesting: 0,
level: 0,
children: null,
content: 'git commit -am "Get macros in the mix."',
markup: '`',
info: '',
meta: null,
block: false,
hidden: false
content: '`git commit -am "Get macros in the mix."`',
markup: '',
info: '',
meta: null,
block: true,
hidden: false

Markdown-it State Obj

Good to know, ok, what is in this state object anyway?

{% raw %}

StateCore {
src: '',
env: {
defaults: { layout: 'default.njk', description: 'Talking about code' },
description: 'Posts tagged with Markdown-It',
layout: 'tags',
projects: [ [Object], [Object] ],
site: {
lang: 'en-US',
github: [Object],
site_url: 'http://localhost:8080',
site_name: 'Fight With Tools: A Dev Blog',
description: 'A site opening up my development process to all.',
featuredImage: 'nyc_noir.jpg',
aramPhoto: ''
pkg: {
name: 'fightwithtooldev',
version: '1.0.0',
description: "This is the repo for Aram ZS's developer notes and log, keeping track of code experiments and decisions.",
main: 'index.js',
scripts: [Object],
keywords: [],
author: '',
license: 'ISC',
devDependencies: [Object],
dependencies: [Object]
templateName: 'tag',
eleventyExcludeFromCollections: true,
pagination: {
data: 'collections.deepTagList',
size: 1,
alias: 'paged',
pages: [Array],
page: [Object],
items: [Array],
pageNumber: 37,
previousPageLink: '/tag/smo/index.html',
previous: '/tag/smo/index.html',
nextPageLink: null,
next: null,
firstPageLink: '/tag/blogroll/index.html',
lastPageLink: '/tag/markdown-it/index.html',
links: [Array],
pageLinks: [Array],
previousPageHref: '/tag/smo/',
nextPageHref: null,
firstPageHref: '/tag/blogroll/',
lastPageHref: '/tag/markdown-it/',
hrefs: [Array],
href: [Object]
permalink: 'tag/{{ paged.tagName | slug }}/{% if paged.number > 1 %}{{ paged.number }}/{% endif %}index.html',
eleventyComputed: {
title: 'Tag: {{ paged.tagName }}{% if paged.number > 1 %} | Page {{paged.number}}{% endif %}',
description: 'Posts tagged with {{ paged.tagName }}'
page: {
date: 2021-11-13T22:11:01.651Z,
inputPath: './src/',
fileSlug: 'tags-pages',
filePathStem: '/tags-pages',
url: '/tag/markdown-it/',
outputPath: 'docs/tag/markdown-it/index.html'
paged: {
tagName: 'Markdown-It',
number: 1,
posts: [Array],
first: true,
last: true
title: 'Tag: Markdown-It',
collections: {
all: [Array],
blogroll: [Array],
'Personal Blog': [Array],
links: [Array],
'Tech Critical': [Array],
Blockchain: [Array],
Cryptocurrency: [Array],
'Code Reference': [Array],
'Ad Tech': [Array],
'BAd Tech': [Array],
'Broken By Design': [Array],
posts: [Array],
projects: [Array],
Starters: [Array],
'11ty': [Array],
Node: [Array],
Sass: [Array],
WiP: [Array],
'Github Actions': [Array],
GPC: [Array],
CSS: [Array],
Aggregation: [Array],
SEO: [Array],
SMO: [Array],
'Markdown-It': [Array],
tagList: [Array],
deepTagList: [Array]
tokens: [],
inlineMode: false,
md: MarkdownIt {
inline: ParserInline { ruler: [Ruler], ruler2: [Ruler] },
block: ParserBlock { ruler: [Ruler] },
core: Core { ruler: [Ruler] },
renderer: Renderer { rules: [Object] },
linkify: LinkifyIt {
__opts__: [Object],
__index__: -1,
__last_index__: 29,
__schema__: '',
__text_cache__: 'Especially with the variable name in the ',
__schemas__: [Object],
__compiled__: [Object],
__tlds__: [Array],
__tlds_replaced__: false,
re: [Object]
validateLink: [Function: validateLink],
normalizeLink: [Function: normalizeLink],
normalizeLinkText: [Function: normalizeLinkText],
utils: {
lib: [Object],
assign: [Function: assign],
isString: [Function: isString],
has: [Function: has],
unescapeMd: [Function: unescapeMd],
unescapeAll: [Function: unescapeAll],
isValidEntityCode: [Function: isValidEntityCode],
fromCodePoint: [Function: fromCodePoint],
escapeHtml: [Function: escapeHtml],
arrayReplaceAt: [Function: arrayReplaceAt],
isSpace: [Function: isSpace],
isWhiteSpace: [Function: isWhiteSpace],
isMdAsciiPunct: [Function: isMdAsciiPunct],
isPunctChar: [Function: isPunctChar],
escapeRE: [Function: escapeRE],
normalizeReference: [Function: normalizeReference]
helpers: {
parseLinkLabel: [Function: parseLinkLabel],
parseLinkDestination: [Function: parseLinkDestination],
parseLinkTitle: [Function: parseLinkTitle]
options: {
html: true,
xhtmlOut: false,
breaks: true,
langPrefix: 'language-',
linkify: true,
typographer: false,
quotes: '“”‘’',
highlight: [Function (anonymous)],
maxNesting: 100,
replaceLink: [Function: replaceLink]

{% endraw %}

Oh, look at that. Everything I need to handle it at the rule level, instead of the rerender process!

Ok, so now I can make a plugin pretty similar to the one I did before.

Searching for git commit via API

I can find the commit message by searching through inline tokens, and pull the repo out of state.env.repo. I can then pull the commit message out of the inline token's content and use it with Octokit to search for the repo. The API query results in a data object with an items property that returns an array that looks like:

url: '',
sha: '29ae79850439397742e0b7147a0fd9b5683058a4',
node_id: 'MDY6Q29tbWl0Mzc2NzA2MzI2OjI5YWU3OTg1MDQzOTM5Nzc0MmUwYjcxNDdhMGZkOWI1NjgzMDU4YTQ=',
html_url: '',
comments_url: '',
commit: {
url: '',
author: [Object],
committer: [Object],
message: 'Set up blogroll and links and write up day 26',
tree: [Object],
comment_count: 0
author: {
login: 'AramZS',
id: 748069,
node_id: 'MDQ6VXNlcjc0ODA2OQ==',
avatar_url: '',
gravatar_id: '',
url: '',
html_url: '',
followers_url: '',
following_url: '{/other_user}',
gists_url: '{/gist_id}',
starred_url: '{/owner}{/repo}',
subscriptions_url: '',
organizations_url: '',
repos_url: '',
events_url: '{/privacy}',
received_events_url: '',
type: 'User',
site_admin: false
committer: {
login: 'AramZS',
id: 748069,
node_id: 'MDQ6VXNlcjc0ODA2OQ==',
avatar_url: '',
gravatar_id: '',
url: '',
html_url: '',
followers_url: '',
following_url: '{/other_user}',
gists_url: '{/gist_id}',
starred_url: '{/owner}{/repo}',
subscriptions_url: '',
organizations_url: '',
repos_url: '',
events_url: '{/privacy}',
received_events_url: '',
type: 'User',
site_admin: false
parents: [ [Object] ],
repository: {
id: 376706326,
node_id: 'MDEwOlJlcG9zaXRvcnkzNzY3MDYzMjY=',
name: 'devblog',
full_name: 'AramZS/devblog',
private: false,
owner: [Object],
html_url: '',
description: null,
fork: false,
url: '',
forks_url: '',
keys_url: '{/key_id}',
collaborators_url: '{/collaborator}',
teams_url: '',
hooks_url: '',
issue_events_url: '{/number}',
events_url: '',
assignees_url: '{/user}',
branches_url: '{/branch}',
tags_url: '',
blobs_url: '{/sha}',
git_tags_url: '{/sha}',
git_refs_url: '{/sha}',
trees_url: '{/sha}',
statuses_url: '{sha}',
languages_url: '',
stargazers_url: '',
contributors_url: '',
subscribers_url: '',
subscription_url: '',
commits_url: '{/sha}',
git_commits_url: '{/sha}',
comments_url: '{/number}',
issue_comment_url: '{/number}',
contents_url: '{+path}',
compare_url: '{base}...{head}',
merges_url: '',
archive_url: '{archive_format}{/ref}',
downloads_url: '',
issues_url: '{/number}',
pulls_url: '{/number}',
milestones_url: '{/number}',
notifications_url: '{?since,all,participating}',
labels_url: '{/name}',
releases_url: '{/id}',
deployments_url: ''
score: 1

So what I need is definitely in there. Now I just need to figure out how to get it out of the async request on on to my new token.

Hmm, it looks like getting the async data into there is going to be the most complex part. The right answer has to be caching, and it looks like there is an Eleventy native tool for that, but I think that might be overkill. Especially because I want something I can save as basically a static file, since this won't be changing. Also, the Eleventy plugin won't work because it is still async. This means I'm basically required to handle this as a file. Also, need to watch out as if I write to the directory I'm watching, I may end up triggering the watch in a loop.

Another thing I'll need to be careful of when caching this data is if I'm automatically creating files, I should likely be using the query as a file key, that query may contain characters not safe for file names, so I'll need to pull something in to handle sanitization.

var sanitizeFilename = require("sanitize-filename");

Ok, so let's start putting it down.

const commit_pattern = () => {
return /(?<=git commit \-am [\"|\'])(.+)(?=[\"|\'])/i;
// I can get the "repo" from the post object
// Then I need to change the commit message I captured
// Queries don't allow spaces, so replace them with "+"
const gitSearchQuery = (repo, commitMsg) => {
const searchCommitMsg = commitMsg.replace(" ", "+")
const repoName = repo.replace("", "")
return `repo:${repoName}+${searchCommitMsg}`

Now let's create a function to figure out the path to the new cache folder.

const cacheFilePath = (pageFilePath, searchKey) => {
const cacheFolder = path.join(__dirname, "../../", '/_queryCache', pageFilePath)
const cacheFile = cacheFolder+sanitizeFilename(slugify(searchKey).replace(".", ""))
// console.log('cacheFile: ', cacheFile)
return { cacheFolder, cacheFile }

I can use fs.accessSync to check if the file exists before creating a cache. After all I don't want to query GitHub every time I do a build. So we can use this in the process to find the repo commit link.

const getLinkToRepo = async (repo, commitMsg, pageFilePath) => {
const searchKey = gitSearchQuery(repo, commitMsg)
const {cacheFolder, cacheFile} = cacheFilePath(pageFilePath, searchKey)
try {
fs.accessSync(cacheFile, fs.constants.F_OK)
return true;

If it exists the function ends here and returns true.

But when we know the file doesn't exist we will have to continue using the catch. I'll make a request to GitHub using Octokit. I'm pretty much skipping over how I set up Octokit because this is basically the boilerplate.

The only thing that I have added into the mix here is to get my Github Key using the environment. Locally I'll use DotEnv require('dotenv').config(). But I'll have to figure out how to handle it on GitHub next. And I get the searchKey using my above function gitSearchQuery.

} catch (e) {
console.log('Query is not cached: ', cacheFile, e)
const MyOctokit = Octokit.plugin(retry, throttling);

const myOctokit = new MyOctokit({
auth: process.env.GITHUB_KEY,
throttle: {
onRateLimit: (retryAfter, options) => {
`Request quota exhausted for request ${options.method} ${options.url}`

if (options.request.retryCount === 0) {
// only retries once`Retrying after ${retryAfter} seconds!`);
return true;
onAbuseLimit: (retryAfter, options) => {
// does not retry, only logs a warning
`Abuse detected for request ${options.method} ${options.url}`
retry: {
doNotRetry: ["429"],
const r = await{
q: searchKey,
if (r && && &&{

Once the commit is found, I'll try to cache it by writing a file using the path to the post and then the search query as a file key.

try {
fs.mkdirSync(cacheFolder, { recursive: true })
// console.log('write data to file', cacheFile)
} catch (e) {
console.log('writing to cache failed:', e)

Now that I have a way to handle caching, I need to trigger it as part of the markdown building process.

I'll need a process to actually create the link on the commit in the markdown-it way.

I'll need to create HTML tokens for the link open and close as follows:

const createLinkTokens = (TokenConstructor,commitLink) => {
const link_open = new TokenConstructor('html_inline', '', 0)
link_open.content = '<a href="'+commitLink+'" target="_blank">'
const link_close = new TokenConstructor('html_inline', '', 0)
link_close.content = '</a>'
return {link_open, link_close}

I'll need to take the markdown-it object and create a new ruler rule. I'll use state.env to check for the repo property and test for the commit pattern in each inline token. By using inline instead of code_inline I will be able to place my new html_inline token around the commit text.

const gitCommitRule = (md) => {
md.core.ruler.after('inline', 'git_commit', state => {
const tokens = state.tokens
if (state.env.hasOwnProperty('repo')){
for (let i = 0; i < tokens.length; i++) {
if (commit_pattern().test(tokens[i].content) && tokens[i].type === 'inline') {
// console.log('tokens round 1: ', tokens[i])
const commitMessage = tokens[i].content.match(commit_pattern())[0]
const searchKey = gitSearchQuery(state.env.repo, commitMessage)
const {cacheFolder, cacheFile} = cacheFilePath(, searchKey)
getLinkToRepo(state.env.repo, commitMessage, => {

let envRepo = state.env.repo;
let linkToRepo = ''
// Let's make the default link go to the commit log, that makes more sense.
linkToRepo = envRepo
if (envRepo.slice(-1) != "/"){
// Assure the last character is a "/"
linkToRepo += "/"
linkToRepo += "commits/main"
try {
fs.accessSync(cacheFile, fs.constants.F_OK)
linkToRepo = (fs.readFileSync(cacheFile)).toString()
} catch (e) {
// No file yet
console.log('Cached link to repo not ready', e)
const { link_open, link_close } = createLinkTokens(state.Token,linkToRepo)


module.exports = (md) => {

It's looking good here, though it isn't super clear that it is a link. Maybe I can add some style to it. Let me grab an image of a link to add to the style. I'll need to size it down.

git commit -am "Adding links to commits across all new posts along with a whole new plugin for building links to commits automatically into new posts for day 37"

Then I can add the CSS.

text-decoration: underline
text-decoration-color: grey
content: ' '
background: transparent url(/img/linkicon-s.png) center right no-repeat
font-weight: normal
font-style: normal
margin: 0px 0px 0px 10px
text-decoration: none
background-size: contain
display: inline-block
width: 12px
height: 12px

git commit -am "Set the default link for repos to go to the main commit log"

After getting a very helpful response from the Markdown-It team it looks like the html_inline process I used to add the link isn't really best practice.

Push to end of core rules

So I found two changes I needed to make. First, I needed to push the rule as the last core rule. It turns out there is a function specifically for this, so I used it and now declare my rule using md.core.ruler.push('git_commit', state => {.

The other change is to use TokenConstructer to make a real token and not html_inline which doesn't have any of the tools and hooks that a normal token does. So now my function to create tokens looks like:

function setAttr(token, name, value) {
const index = token.attrIndex(name);
const attr = [name, value];

if (index < 0) {
} else {
token.attrs[index] = attr;

const createLinkTokens = (TokenConstructor, commitLink) => {
const link_open = new TokenConstructor("link_open", "a", 1);
setAttr(link_open, "target", "_blank");
setAttr(link_open, "href", commitLink);
setAttr(link_open, "class", "git-commit-link");
// This is haunting me, so I asked -
const link_close = new TokenConstructor("link_close", "a", -1);
return { link_open, link_close };

The 1/-1 here allows me to open and close the a tag and now I can use the attrPush on the token which is a much better practice.


git commit -am "Switching git-commit process to build link using link_open and link_close"

Can't start secrets with GITHUB

While prepping to merge I wanted to add the GITHUB_KEY env secret. But it turns out I can't start my keynames with GITHUB_. So I gotta rename it.