//go:fix inline and the source-level inliner
173 points by commotionfever 5 days ago | 69 comments

shoo 19 hours ago
If I follow, this isn't a compile time inline directive, it's a `go fix` time source transformation of client code calling the annotated function.

Per the post, it sounds like this is most effective in closed-ecosystem internal monorepo-like contexts where an organisation has control over every instance of client code & can `go fix` all of the call sites to completely eradicate all usage of a deprecated APIs:

> For many years now, our Google colleagues on the teams supporting Java, Kotlin, and C++ have been using source-level inliner tools like this. To date, these tools have eliminated millions of calls to deprecated functions in Google’s code base. Users simply add the directives, and wait. During the night, robots quietly prepare, test, and submit batches of code changes across a monorepo of billions of lines of code. If all goes well, by the morning the old code is no longer in use and can be safely deleted. Go’s inliner is a relative newcomer, but it has already been used to prepare more than 18,000 changelists to Google’s monorepo.

It could still have some incremental benefit for public APIs where client code is not under centralised control, but would not allow deprecated APIs to be removed without breakage.

reply
avabuildsdata 18 hours ago
yeah this is the part that got me excited honestly. we're not google-scale by any stretch but we have ~8 internal Go modules and deprecating old helper functions is always this awkward dance of "please update your imports" in slack for weeks. even if it doesn't let you delete the function immediately for external consumers, having the tooling nudge internal callers toward the replacement automatically is huge. way better than grep + manual PRs
reply
shoo 18 hours ago
it could be better than a nudge -- if you could get a mandatory `go fix` call into internal teams' CI pipelines that either fixes in place (perhaps risky) or fails the build if code isn't already identical to fixed code.
reply
RossBencina 16 hours ago
I'm not sure what all of the hazards are, but I could imagine a language (or a policy) where public APIs ship with all of the inline fix directives packaged as robust transactions (some kind of "API-version usage diffs"). When the client pulls the new API version they are required to run the update transaction against their usage as part of the validation process. The catch being that this will only work if the fix is entirely semantically equivalent, which is sometimes hard to guarantee. The benefits would be huge in terms of allowing projects to refine APIs and fix bad design decisions early rather than waiting or never fixing things "because too many people already depend on the current interface".
reply
rishabhjajoriya 4 hours ago
That was quite insightful read
reply
omoikane 19 hours ago
I wonder why they chose to add these directives as comments as opposed to adding new syntax for them. It feels like a kludge.

https://wiki.c2.com/?HotComments

reply
kjksf 19 hours ago
Go designers distinguish between Go language as defined by Go spec and implementation details.

//go:fix is something understood by a particular implementation of Go. Another implementation could implement Go without implementing support for //go:fix and it would be a fully compliant implementation of Go, the language.

If they made it part of the syntax, that would require other implementations to implement it.

reply
dwattttt 18 hours ago
If the comments impact correctness (which inlining doesn't, but I believe there are other directives that do), saying it's "an implementation detail" waves away "it's an implementation detail that everyone needs" aka part of the spec.

The reason it feels like a kludge is that "comments" are normally understood to be non-impactful. Is a source transformation that removes all comments valid? If comments have no impact per the spec, yes. But that's not the case here.

In practice comments in go are defined to be able to carry semantic meaning extensibly. Whether they're safe to ignore depends on what meaning is given to the directives, e.g. conditional compilation directives.

reply
scheme271 13 hours ago
There are directives and packages that affect correctness. E.g. the embed package allows you to initialize a variable using a directive. E.g. //go:embed foo.json followed by var jsonFile string initializes the jsonFile variable with the contents of the foo.json file. A compiler or tooling that doesn't support this results in broken code.
reply
tptacek 18 hours ago
There's nothing unique to Go about this kind of tooling. It exists in C, Java, Rust, Typescript, and probably dozens of other settings as well. It's the standard way of implementing "after-market" opt-in directives.
reply
dwattttt 17 hours ago
Are we referring to 'go fix' as after market tooling?

It's certainly done in many places. JsDoc is the biggest example I can think of. But they're all walking the line of "this doesn't have an impact, except when it does".

It being done by the language owners just makes them the ones walking the line.

reply
tptacek 17 hours ago
That's exactly how this works: it doesn't have an impact, except when you ask it to. This is an idiomatic approach to this problem.
reply
dwattttt 17 hours ago
The part I object to is overloading comments, which aren't meant to be load bearing. A tool developed outside the language has no choice but to take something that has no meaning and overload it, but language owners weren't forced to take this route; they could've made directives not comments.

In practice, the Go language developers carved syntax out of comments, so that a comment is "anything that starts with //, unless the next characters are go:"

reply
YesThatTom2 16 hours ago
So how many angels can you fit on the head of a pin?
reply
omoikane 17 hours ago
In the listed examples, the compiler will emit a diagnostic upon encountering those comments:

https://go.dev/blog/inliner#example-fixing-api-design-flaws

So these comments carry more weight than how those comment annotations might be consumed by optional tools for other languages.

For most of the listed examples, I think the corresponding C annotation would have been "[[deprecated]]", which has been added to the syntax as of C23.

reply
ternaryoperator 17 hours ago
It does not exist in Java. Comments in Java do not change code.
reply
joshuamorton 16 hours ago
This also does not change th code. It is an advertisement to a linter-loke tool to take some action on the source code. Its most similar to linter directives which usually are comments.
reply
TheDong 12 hours ago
We're talking about the "//go" comments in general I think here.

Things like "//go:embed" and "//go:build" very much do change the semantics of source code.

The comments above 'import "C"' containing C function definitions and imports change the compilation of go source code.

The "//go" comments contain a mix of ones that must be respected to compile, to being optional, to being entirely ignorable (like generate and fix).

reply
9rx 8 hours ago
It doesn't exist in Go either. https://go.dev/ref/spec

That's why you find it in the comments. That is where tools have found a place to add their own syntax without breaking the Go code.

Absolutely you can do the same in Java. It exists to the exact same degree as it does in Go. I expect it isn't done as often in the Java world because it is much harder to parse Java code so the tools don't get built.

reply
Patryk27 17 hours ago
There are no comment-based directives in Rust, are there?
reply
win311fwg 16 hours ago
It provides the feature to use. It’s possible nobody has yet.
reply
tptacek 17 hours ago
Eh, you're right, they have a structured attribute system.
reply
joshuamorton 18 hours ago
> The reason it feels like a kludge is that "comments" are normally understood to be non-impactful. Is a source transformation that removes all comments valid? If comments have no impact per the spec, yes. But that's not the case here.

This is not inlining in the compiler. It's a directive to a source transformation (refactoring) tool. So yes, this has no impact on the code. It will do things if you run `go fix` on your codebase, otherwise it won't.

reply
dwattttt 18 hours ago
And yet it still breaks "comments aren't semantic". That transformation I described is still invalid.
reply
pastel8739 16 hours ago
I don’t understand why that wouldn’t be valid. As far as I understand if you compile code with these go:fix comments, they will be ignored. But if instead of compiling the code you run ‘go fix’, the source code will be modified to inline the function call. Only after the source code has been modified in this way would compiling reflect the inlining. Do you have a different understanding?
reply
dwattttt 16 hours ago
I mean that directives other than inlining impact correctness. If you have a source file that only builds for one OS, stripping the build tag will break your build.
reply
bheadmaster 19 hours ago
That's such an elegant solution.

I keep being impressed at subtle but meaningful things that Go does right.

reply
usrnm 8 hours ago
It only seems right because there are no other implementations and the distinction is meaningless. You don't want to live in the world of multiple slightly different implementations all of which you need to support, this is, roughly, what the C++ build story looks like, and it's not fun.
reply
Groxx 17 hours ago
Because these are instructions for users for making tool-assisted changes to their source code, not a behavior that exists at runtime (or even compile time). A new syntax wouldn't make sense for it.

For other things, like `//go:noinline`, this is fair criticism. `//go:fix inline` is quite different in every way.

reply
kalterdev 11 hours ago
I suppose, to minimize its use. If annotations have the same syntactic weight as normal statements, such as “if” or “for” statements, there’s a temptation to use them liberally, which is clearly not a good fit for Go.

By making them comments, Go subtly signals that these are exceptional, making them less prominent and harder to abuse.

reply
0x696C6961 19 hours ago
The //go:xyz comments are an established pattern in the Go tooling.
reply
Mond_ 19 hours ago
This is begging the question. Yes, but why did they do that over dedicated syntax?

(My personal theory is that early go had a somewhat misguided idea of simplicity, and preferred overloading existing concepts with special cases over introducing new keywords. Capitalization for visibility is another example of that.)

reply
thwarted 18 hours ago
//go:xyz is dedicated syntax that is compatible with both the language spec and other toolchains that don't know about it.
reply
Mond_ 17 hours ago
It's an overloaded comment. I am personally quite fine with it, I don't think it's bad. but it is an overloaded comment.
reply
thwarted 17 hours ago
I'm no longer sure what you're saying. You asked why they didn't go with dedicated syntax, I listed two advantageous aspects of the chosen syntax. We know it's an overloaded comment: that's literally one of the advantages.
reply
Mond_ 13 hours ago
Well, I've been unable to follow you as well, then. Obviously if they'd used a different type of syntax (e.g. using # for annotations), those would also be compatible with the language spec, and other implementations would still be just as capable of ignoring all unknown annotations.

(Though for the record, talking about alternative implementations when discussing Go is kind of a funny joke.)

reply
overfeed 10 hours ago
Is gccgo a joke to you?
reply
debugnik 8 hours ago
Maybe? It's stuck at 1.18 without generics and no one has replaced its maintainer, Ian Lance Taylor, who seems to have moved on after leaving Google.

But to be fair to alternative toolchains, TinyGo and TamaGo are also a thing.

reply
debugnik 9 hours ago
Good luck compiling on a toolchain that doesn't know about //go:embed or /* */import "C" comments.
reply
freakynit 13 hours ago
Can't golang devs prioritize something like annotations or other attribute/metadata system instead of writing these in comments? I'm pretty sure this must have been raised a lot of times before, so just wanted to ask if there is/are any specific reason(s)?
reply
kalterdev 10 hours ago
I think the core reasoning is about minimizing its use. I have answered [1] the same question in another thread.

https://news.ycombinator.com/item?id=47395574

reply
alecthomas 12 hours ago
These are called directives [1], and are treated as metadata by the compiler.

[1] https://pkg.go.dev/go/ast#Directive

reply
freakynit 12 hours ago
Understood... but why in comments?
reply
alecthomas 12 hours ago
Someone else said this below...

> Go designers distinguish between Go language as defined by Go spec and implementation details. > //go:fix is something understood by a particular implementation of Go. Another implementation could implement Go without implementing support for //go:fix and it would be a fully compliant implementation of Go, the language. > > If they made it part of the syntax, that would require other implementations to implement it.

...I'm not sure I buy that argument TBH.

reply
OJFord 5 hours ago
That does seem a strange argument, it could simply be `%%` (or whatever) to introduce a 'metadata comment', and then a Go implementation that doesn't support metadata would simply lex both `%%` and `//` as comments and treat them identically.
reply
powerpixel 7 hours ago
I'd buy it. AFAIK the goal of Go is to have as little breaking changes as possible between versions. Although they introduced backwards compat breaking features such as generics have they not ?
reply
freakynit 12 hours ago
hmm... thanks... And yes, I don't buy it either.

"If they made it part of the syntax, that would require other implementations to implement it." ... I mean, so what? Has golang stopped ading new features to the spec? If not (which I guess so), then how is this any different? Unless you have freezed the language, this reasoning doesn't make sense to me.

reply
9rx 9 hours ago
You are right that there could be new syntax, like, say, `@tool:name args` or `#tool.name args`, but is that any different than `//tool:name args`? They all read the same to me.

The upside of that particular syntax is that only the parser used by tools needs to understand directives. All other parser implementations can be blissfully unaware, negating the need for special no-ops. The downside is...?

reply
freakynit 9 hours ago
I mean, technically you could write your entire business logic inside comments and have some tool parse it successfully. But we don't do that, because intuitively we know that's not the right place for it.

The issue isn't that this approach is incorrect, it's that it feels out of place. A comment should be a comment, nothing more. When comments start carrying executable meaning or structured directives, they stop serving their primary purpose.

It also becomes difficult to represent anything moderately complex in a clear way. Once the structure grows beyond something trivial, readability suffers quickly.

To me, it ends up feeling like command-line arguments.. technically workable, but messy and hard to reason about. Just look at something like "ffmpeg" arguments.. and then compare that to defining the same configuration through a structured format like Yaml or Json. The latter is simply clearer and easier to maintain.

It's not wrong, but, it doesn't feel right.

reply
9rx 8 hours ago
> I mean, technically you could write your entire business logic inside comments and have some tool parse it successfully.

It sounds like you are talking about cgo. I think you have a stronger case there, but it is much the same situation: It's conceptually a third-party add-on that the Go language doesn't know anything about. "Cgo is not Go"[1]

I mean, if you really did have your own business logic language that you needed to include in Go code, where else would you put it if not in the "comments"? You, a random third-party, won't be able to add syntax to Go, so you cannot reasonably consider that to be an option. What syntax could Go add that is flexible enough to allow anyone to add anything they need, but that doesn't end up being comments by another token?

> A comment should be a comment, nothing more.

It's not really a comment, though. It is a directive. A comment is structured like `// Who the hell wrote this crap?`, while this is structured like `//tool:name args`.

I think what you are saying is that you don't like overloaded syntax, which is fair, but Go overloads syntax in several places so this is not a unique case. Besides, most languages end up with overloaded syntax in practice, so it isn't even something unique to Go.

To be clear, this isn't a feature of Go. There is nothing in the spec about it. It is what outside tools have hacked on top of Go because it doesn't offer that infinitely flexible feature mentioned above. However, it remains to be seen how you add infinitely flexible syntax without it turning into just being comments again.

[1] https://go-proverbs.github.io

reply
freakynit 8 hours ago
I think your latest comment clarifies a lot of things for me here... primary being that it's something language designer did not wanted... just that tool developers went ahead with because there wasn't somethinf else (or something better) they could use.

If that is indeed the case, I believe it's fair. Im not into language/compiler design etc., but if I have to take a guess, this is where metaprogramming would have helped, right? Like the ones provided by zig or rust?

reply
pjmlp 9 hours ago
This is typical Go design, other languages do it worse, it isn't really needed, and then it gets added half way as it was supposed to be if done early on, and everyone cheers how Go is a "simple" language.

Just like some other famous languages of the authors.

reply
frou_dh 6 hours ago
It's abject hackery because such "comment" directives don't even necessarily have to be a comment to be active:

    const x = `This whole thing is
    a
    //go:generate worse-is-better
    multiline
    string literal`
A lot of people in this discussion are beating up Go for using syntactical comments for directives, but in reality the implementation is even less principled than that!
reply
ansgri 18 hours ago
Good illustration that a seemingly simple feature could require a ton of functionality under the hood. Would be nice to have this in Python.
reply
fghorow 7 hours ago
I know this is off-topic, and likely to get me down-voted, but hey! I'll live dangerously for the sake of repeating a hilarious (to me) .sig in a now ancient Usenet post.

These //go.* commands always remind me of this:

"""

//GO.SYSIN DD *

DOO DAH

DOO DAH

"""

(Why yes, that is IBM System 360 JCL from circa 1975. Why do you ask?)

reply
tapirl 19 hours ago
It looks the following code will be rewritten badly, but no ways to avoid it? If this is true, maybe the blog article should mention this.

    package main
    
    //go:fix inline
    func handle() {
        recover()
    }
    
    func foo() {
        handle()
    }
    
    func main() {
        defer foo()
        panic("bye")
    }
reply
arjvik 17 hours ago
recover()'s semantics make it so that "pointless" use like this can be inlined in a way that changes its semantics, but "correct" use remains unchanged.

Yes, maybe some code uses recover() to check if its being called as a panic handler, and perhaps `go fix` should add a check for this ("error: function to be inlined calls recover()"), but this isn't a particularly common footgun.

reply
tapirl 14 hours ago
> ... and perhaps `go fix` should add a check for this (

This is an impossible task. For a library function, you can't know whether or not the function is defer called.

Maybe this is not an important problem. But it would be better if the blog article mentions this.

reply
hrmtst93837 11 hours ago
'Not common' is comforting until you hit a codebase where recover gets abused and your 'safe' inlining breaks prod.
reply
shoo 19 hours ago
Great example, illustrating go1.26.1 go fix source inline transformation breaking program semantics. Raise it as a bug against go fix?
reply
tapirl 19 hours ago
As I have mentioned, no ways to fix it. Because it is hard to know whether or not the handle function is called in a deferred call.
reply
arccy 19 hours ago
Or: your buggy code is no longer buggy.
reply
tapirl 19 hours ago
You claim listens right for this specified example. :D

It is just a demo.

reply
tapirl 18 hours ago
Another example (fixable):

    package main

    import "unsafe"

    //go:fix inline
    func foo[T any]() {
        var t T
        _ = 1 / unsafe.Sizeof(t)
    }

    func main() {
        foo[struct{}]()
    }
Go is a language full of details: https://go101.org/details-and-tips/101.html
reply
tapirl 14 hours ago
another:

   package main

   type T = [8]byte
   var a T

   //go:fix inline
   func foo() T {
      return T{}
   }

   func main() {
      if foo() == a {
      }
   }
filed: https://github.com/golang/go/issues/78170 and https://github.com/golang/go/issues/78169
reply
tapirl 16 hours ago
similar:

    package main

    //go:fix inline
    func foo[T [8]byte | [4]uint16]() {
        var v T
        var n byte = 1 << len(v) >> len(v)
        if n == 0 {
            println("T is [8]byte")
        } else {
            println("T is [4]uint16]")
        }
    }

    func main() {
        foo[[8]byte]()
    }
reply
vismit2000 16 hours ago
reply
gnabgib 16 hours ago
Far later submission. Check the ID again.. you were 2 days later.

There was even a more upvoted post between your triple dupe and this https://news.ycombinator.com/item?id=47347322 #scp

reply
useftmly 5 hours ago
[dead]
reply