← 输入框|github.com|刷新
Skip to content

Add scijava-code-lsp et al - #294

Draft
gselzer wants to merge 67 commits into
mainfrom
code-lsp
Draft

gselzer wants to merge 67 commits into
mainfrom
code-lsp

Conversation

@gselzer

@gselzer gselzer commented Oct 21, 2026

Copy link
Copy Markdown
Member

This PR introduces a few new modules to the monorepo:

scijava-code-lsp wrangles LanguageServers

The central package, scijava-code-lsp is designed to assist with the management of Language Servers, following the Language Server Protocol, using lsp4j. A LanguageServerService offers the ability to launch a LanguageServer based on a desired programming language and Environment (interpreter path, module search paths, existing bindings) for general usage.

As the central use case is assisting the development of SciJava scripts, a ScriptSession class is also offered. Users provide (1) the text of a SciJava script to a ScriptSession and (2) a LanguageServerService, and can then query for autocomplete, hover info, etc. as well as listen to diagnostic feedback.

Of course, LanguageServers (can/should) not understand SciJava scripts as written, so scijava-code-lsp offers a ScriptDialect plugin describing how to convert text specific to SciJava scripts (e.g. script parameters) to headers that a language server could under stand and reason about. The scijava/scripting-X repositories are where I'm planning for these implementations to live (PRs to follow shortly)

Finally, scijava-code-lsp contains utilities common to multiple language servers, which leads us into:

scijava-code-lsp-groovy offers a GroovyLanguageServer

This language server is fully in-process and uses Groovy's compiler process to develop a typed AST that is used to answer LSP queries.

scijava-code-lsp-jedi offers a PythonLanguageServer

This language server is inter-process and uses jedi-language-server to answer LSP queries.

scijava-code-lsp-jython offers a JythonLanguageServer

This language server is inter-process and uses PythonLanguageServer indirectly. It also autogenerates type stubs to enable java types to be handled by the Python language server.

Points of Discussion

  • lsp4j is a significant dependency to bring in (~640KB) - I'd be receptive to the case that it's not worth it to use the existing work, and we could consider a scijava-code-lsp-api module, because it's unfortunate to bring this in because a library that just wants to declare a ScriptDialect implementation now gets this dependency.
  • jedi was chosen by Claude for the Python language server, but that is apparently in maintenance mode. We probably ought to choose a different server - options include ty, zuban, and pyright - should be very easy to switch.

gselzer and others added 30 commits October 21, 2026 16:15
LspServers starts language servers (one per environment scripts run in) in the background, restarts them if they stop, and stops them when idle or too many. LspClient asks them about ScriptDocuments in code-api terms: completions (the first few resolved for their signatures, the others for documentation on demand) and signature help, at positions of the original script. Answers are reused while the same question is asked, and a pending question is cancelled ($/cancelRequest) once another is asked about the same document.

code-api: CodeCompleter#closed, so completers can release what they keep for a script.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
code-api: CodeCompleter#hover (what is known about the code at a position, e.g. documentation; with updates, like signature help) and CodeCompleter#diagnose (problems in a script, e.g. syntax errors, once found).

code-lsp: LspClient#hover (textDocument/hover) and LspClient#diagnose, from the diagnostics the server publishes, awaited for the version of the document sent; positions mapped back to the original script, problems in the added lines left out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LspServerPlugins offer a language server for a language (e.g. jedi-language-server for Python); DefaultLspService keeps one LspServers per language, shared by the completers that want one, and stops them when the context is disposed. PythonLspClient (reading Python signatures) moves here from scripting-appose-python, for Jython's use too. Gson is declared directly, so that consumers need not resolve lsp4j's range.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SciJavaLanguageServer serves the completions, signature help, hover and diagnostics of the context's code completers over the Language Server Protocol, for editors outside SciJava (e.g. VS Code); Main runs it over standard input and output, keeping SciJava's output off the protocol's stream.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every source of code intelligence becomes an lsp4j LanguageServer: in-process, or a server process (LaunchedLanguageServer, over LspServers). LanguageServerPlugins offer them per SciJava script language; LanguageServerService keeps them, one per plugin and Environment, and hands callers a MergedLanguageServer per language, which keeps each document in sync on every server for it (moving it as its environment changes) and merges their answers: completions in priority order without duplicates, slower ones as an UpdatingCompletionList; the first signature help and hover; all diagnostics. RatedSignatureInformation keeps overload fits; TransformingLanguageServer shows a server SciJava scripts as plain code (ScriptDocument).

Until each language has a server of its own, CodeCompleterServerPlugin offers code-api's CodeCompleters as one. The default completer reads live bindings from a ScriptContext too, as servers get no engine.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The adapter serves languages with a completer of their own (so editors can tell which are served), can serve a single completer (e.g. in tests), and looks for problems a moment after the last change, as editors send every keystroke. Merging keeps overloads: completions are the same only if they insert the same name with the same parameters (label details).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Merging dropped completions inserting the same name with the same parameters even within one server's list, e.g. Groovy's each() from Iterable, Collection, List and Object. Only those duplicating an earlier server's are dropped now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ScriptLanguageServer: a base class keeping the open documents, answering requests by asking the subclass about a document and offset, resolving documentation on demand, and publishing problems a moment after each change. Callable: something to call, as a snippet completion with typed label details, or as a rated signature. Members: completions for Java classes' members, by reflection. Signatures: rating calls' arguments, on Callables.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Callable.of(item) reads a callable from a completion's label details. Merging compares completions without an edit (e.g. jedi's "length") by the word before the caret, so that they match those replacing more (e.g. Jython's "s.length").

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PythonCompletionServer wraps a Python language server (e.g.
jedi-language-server) as editors want it: it asks at a word's first
character and caches the answer, cancels superseded requests, filters by
the word typed, and gives the top completions their signatures.
DelegatingLanguageServer forwards everything, for such decorators.
Cancelling a completion now cancels the request it waits on, through
TransformingLanguageServer and LaunchedLanguageServer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Code intelligence is now lsp4j language servers throughout. What
code-api had that servers still need moves into code-lsp: ScriptDocument,
ScriptDialect, PositionMap and ClassIndex; script parameter parsing, as
LanguageServerService.scriptParameters; and Environment in place of
EnvironmentContext. Positions replaces LspClient's offset conversions,
and Python signatures are parsed by PythonCompletionServer.

Gone: the code-api adapter, LspClient and PythonLspClient. In their
place, BindingsLanguageServer completes a running interpreter's
variables, for languages without servers. The outside-editor server
(SciJavaLanguageServer) is now a front for the service's merged servers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A ScriptSession is where a client (an editor, a REPL, a tool checking a
script) asks about a SciJava script, in terms of its text and offsets:
it keeps the servers' copy up to date, names the script (its file's URI,
or an untitled one; following Save As), and receives its problems. Each
request carries the text it is about. The LanguageServerService makes
sessions, each owning a merged server of its own; for languages no
plugin serves, a session completes a running interpreter's variables.
diagnose is now a session asked once.

Where scripts run can change without them changing (e.g. an environment
is built): plugins now say so via environmentsChanged(), which moves
open scripts to the servers of where they run now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Nothing runs it: editors in the SciJava process ask the service for
servers directly, and none outside it has been wired to this one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
It is specific to Python (completing Java members on objects a Python
server cannot see into), so it belongs with the one plugin using it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The module does several things, each now in a package of its own:

- org.scijava.code.lsp: brokering between scripts and servers (the
  service, plugins, script sessions, environments), plus SciJava's
  additions to LSP (updating completion lists, rated signatures).
- .launch: running language server processes.
- .script: SciJava scripts as plain code (#@ parameters declared).
- .jvm: a toolkit for in-process servers of JVM languages.
- .impl: the implementation, not exported.

The specialized packages depend only on the base package, never on
each other. A module-info exports the four public packages; it is open,
for SciJava Common's reflection. Package infos say what each is for.

To keep the implementation internal, clients get sessions via
ScriptSession.owning, the service no longer offers its servers directly
(LanguageServerService.server), and data is no longer assumed to be
Gson's (the module no longer reads Gson).

Tests run on the classpath, not the module path, where the tests'
plugin index would shadow the module's own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ScriptSession.server, language, text and uri, and setFile(File), had no
callers outside tests; nor had LanguageServerService.session(File).
Clients set a file supplier, as editors follow Save As.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drop UpdatingCompletionList and RatedSignatureInformation, SciJava's
additions to LSP:

- Merged completions come once all servers have answered, as signature
  help and hover already did; clients that must not wait ask
  asynchronously.
- In-process servers still order overloads best fit first (Fit moves to
  Signatures), but signatures no longer carry their fit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drop MergedLanguageServer: a language is served by the highest-priority
plugin supporting it, and a server combining several sources (e.g. an
in-process analysis and jedi) combines them itself. Each session gets a
server of its own, which it initializes (awaiting the answer, as LSP
says) and shuts down when closed. Server processes stay shared, one per
environment.

With it go the service's shared servers and diagnostics routing,
moving scripts between servers as where they run changes
(LanguageServerPlugin.environment, environmentsChanged), and waiting
for several servers' problems.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both are ordered places imports are resolved from; each server reads
them as its language does. Nothing read the classpath.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A script document maps positions between its script and its text
itself: by offset, and by LSP position and range.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A ScriptSession now transforms each script into plain code of its
language, in the dialect of the plugin offering its server
(LanguageServerPlugin#dialect), and maps positions in the server's
answers back. Servers need know nothing of SciJava scripts.

TransformingLanguageServer, which did this inside some servers, is
gone, and so is ScriptDocument's unused environment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The declarations a session adds are real code: imports of the
parameters' types bind names (e.g. numpy) the script does not. A
dialect now says which name each import binds
(ScriptDialect#imported); unless the script mentions it, the session
leaves completions of it out (see ScriptDocument#added).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ScriptDialect and ScriptDocument are now part of how sessions talk to
servers (LanguageServerPlugin#dialect, ScriptSession), so they move
from org.scijava.code.lsp.script into org.scijava.code.lsp. This also
ends the cycle between the two packages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A server process is now offered by a ServerProcessPlugin, in the core
package: it starts the process for an LSP language and an environment,
and says how to initialize it (options, workspace root). The launch
package (LspServerPlugin, LspServerSpec, LspConnection) and the
separate LspService are gone; LanguageServerService#launch serves the
processes itself.

The implementation follows the new names: ServerProcesses keeps a
language's ServerProcess per environment, and a ProcessLanguageServer
shows one as an lsp4j LanguageServer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LanguageServerService.launch can now take a function from a document to
its environment, which ProcessLanguageServer consults before each
request: a document whose environment changes is closed on its old
server process and opened on its new one. Completions remember which
process answered them, so that they are resolved by it.

Also note in ScriptDialect's javadoc why a dialect describes the
translation of scripts, rather than doing it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A ServerProcessPlugin for python, moved here from scripting-appose-python
so that script languages can use jedi without depending on each other
(e.g. scripting-jython). It runs jedi-language-server in a small Appose
environment of its own (code-lsp-jedi: Python 3.12, Jedi, NumPy), built
the first time it is needed, under a launcher that exits once the JVM
is gone.

JediLifecycleIntegrationTest starts a server via LanguageServerService,
asks it about a script, and checks that its process stops with the
context.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LanguageServerService.launch(languageId, environment) now gets a server
for any LSP language a plugin serves, in-process or not: plugins say so
via supports(String), and supports(ScriptLanguage) for sessions. Server
processes are served by ProcessLanguageServerPlugin (internal), which
owns them and stops them when disposed; LanguageServerService also tells
whether a language is served at all, via supports(String).

Servers analyze scripts in one environment, given when created: plugins
whose scripts say where they run keep a server per environment
themselves. ProcessLanguageServer no longer moves documents.

Also:
- Rename ScriptLanguageServer to AbstractLanguageServer.
- Throw rather than return null when no plugin serves a language.
- Remove LanguageServerService.diagnose: a session does as much.
- Stop started server processes directly on shutdown, rather than via
  the starter, whose shutdown could drop the stop.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gselzer and others added 9 commits October 21, 2026 16:15
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Only JythonLanguageServerPlugin (renamed from JythonServerPlugin) is public:
nothing outside the module used the rest. JythonDev's messages, printed to
System.out by default, are gone; EmptyDotAutocompletions is a lambda, and
JythonImportFormat part of JythonImports.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Jython's own analysis (its parser, scopes and dot completions) is gone:
JythonLanguageServer is jedi-language-server, which knows Java classes
from the stubs of the packages a script imports. Beside its completions
come Java classes not imported yet (importing themselves as accepted);
a live interpreter's variables are declared to it, as a module the
script's text imports first; and names like __name__ are offered only
if Jython has them.

So Jython completion, like Python's, needs jedi's environment, built
the first time it is used; and the tests now run with it. Lost, for
now (see the TODOs): the variables of a parameter's type, first in a
call's arguments; and all of a method's overloads in signature help.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BindingsLanguageServer is offered for plaintext by BindingsServerPlugin,
both back in impl. So a session always asks the service for its servers:
of its dialect's LSP language if served, and otherwise (or if that server
cannot be created) of plaintext, as its script is then shown.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Gone, with nothing calling them (Jython's own analysis was the last to
use the first two): Callable.of, Signatures.arguments, Document.version
and ClassIndex.addRoots. Private, as only their classes use them: most
of ClassIndex's documentation lookup, Members.field, method and
signature, Signatures.rate and loadType, and Callable.parameterList.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gselzer

gselzer commented Oct 21, 2026

Copy link
Copy Markdown
Member Author

Note that each of the scripting-X changesets will depend upon the code implemented here, and the script-editor changeset will depend upon each of the scripting-X changesets in addition to the code implemented here.

gselzer and others added 14 commits October 22, 2026 15:23
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ScriptDialect.header now takes Variables (a type, and a value when known)
rather than types, and ScriptSession gives it a live interpreter's
bindings too, alongside the script's parameters (which win). So each
dialect declares them as its language should, and the servers no longer
read the live bindings themselves: GroovyLanguageServer's own dialect and
JythonLanguageServer's stub module of them are gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A number followed by a dot is the start of a decimal only when nothing,
or a digit, follows the dot: a name means a member access, of the number
(an object in Groovy). Test numbers on their own, not with keywords.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Its bindings change as code runs in it, so they are read anew rather than
kept. And the fields need not repeat their getters' docs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Not with inferred types; and of a typed local and a parameter too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
As e.g. testCompleteStaticMembers, with completing qualified class names
apart from imports. And test where imports are inserted via completion,
so importOffset can be private; explain the snippet test, too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Groovy server's description points to scripting-groovy for SciJava
scripts, rather than saying how they (and live bindings) become Groovy.
The Jedi and Jython READMEs are unwrapped, one line per paragraph, as the
others now are; their words are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The server package's Javadoc was swallowed by its unclosed license header:
close it, and say how to offer a server of your own, with an example.
Link the LSP specification (3.18) and jedi-language-server's options from
ServerLauncher, whose initialization options LSP leaves to each server.
And explain each part of JediLauncher's options: its diagnostics ones are
the server's defaults.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@hinerm

hinerm commented Oct 26, 2026

Copy link
Copy Markdown
Member

@gselzer I would encourage you to skip converting this to a full PR that requires review; once it's working and all implementation questions are resolved, why not just merge it?

lsp4j is a significant dependency to bring in (~640KB)

This is very thoughtful, but I am of the opinion that any dependency wrangling for size is premature optimization. Let's think from an end user's perspective: lsp4j is distributed once and although other libraries inherit the dependency, they don't duplicate its distribution. Also you're quoting a number in KB. That is not a unit of size that deserves discussion without strong compelling evidence to the contrary.

jedi was chosen by Claude for the Python language server, but that is apparently in maintenance mode.

This is legitimate. I suggest choosing for best chance at long-term stability (Pyright). Zuban would be my second choice.

@gselzer

gselzer commented Oct 26, 2026

Copy link
Copy Markdown
Member Author

Thanks @hinerm!

I would encourage you to skip converting this to a full PR that requires review; once it's working and all implementation questions are resolved, why not just merge it?

If nobody else wants to review it then I will certainly do so!

lsp4j is a significant dependency to bring in (~640KB)

This is very thoughtful, but I am of the opinion that any dependency wrangling for size is premature optimization.

Fair enough, let's go with it!

This is legitimate. I suggest choosing for best chance at long-term stability (Pyright). Zuban would be my second choice.

Your preferences will be taken into account 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants