Repository navigation
Conversation
LspServers starts language servers (one per environment scripts run in) in the background, restarts them if they stop, and stops them when idle or too many. LspClient asks them about ScriptDocuments in code-api terms: completions (the first few resolved for their signatures, the others for documentation on demand) and signature help, at positions of the original script. Answers are reused while the same question is asked, and a pending question is cancelled ($/cancelRequest) once another is asked about the same document. code-api: CodeCompleter#closed, so completers can release what they keep for a script. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
code-api: CodeCompleter#hover (what is known about the code at a position, e.g. documentation; with updates, like signature help) and CodeCompleter#diagnose (problems in a script, e.g. syntax errors, once found). code-lsp: LspClient#hover (textDocument/hover) and LspClient#diagnose, from the diagnostics the server publishes, awaited for the version of the document sent; positions mapped back to the original script, problems in the added lines left out. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LspServerPlugins offer a language server for a language (e.g. jedi-language-server for Python); DefaultLspService keeps one LspServers per language, shared by the completers that want one, and stops them when the context is disposed. PythonLspClient (reading Python signatures) moves here from scripting-appose-python, for Jython's use too. Gson is declared directly, so that consumers need not resolve lsp4j's range. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SciJavaLanguageServer serves the completions, signature help, hover and diagnostics of the context's code completers over the Language Server Protocol, for editors outside SciJava (e.g. VS Code); Main runs it over standard input and output, keeping SciJava's output off the protocol's stream. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every source of code intelligence becomes an lsp4j LanguageServer: in-process, or a server process (LaunchedLanguageServer, over LspServers). LanguageServerPlugins offer them per SciJava script language; LanguageServerService keeps them, one per plugin and Environment, and hands callers a MergedLanguageServer per language, which keeps each document in sync on every server for it (moving it as its environment changes) and merges their answers: completions in priority order without duplicates, slower ones as an UpdatingCompletionList; the first signature help and hover; all diagnostics. RatedSignatureInformation keeps overload fits; TransformingLanguageServer shows a server SciJava scripts as plain code (ScriptDocument). Until each language has a server of its own, CodeCompleterServerPlugin offers code-api's CodeCompleters as one. The default completer reads live bindings from a ScriptContext too, as servers get no engine. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The adapter serves languages with a completer of their own (so editors can tell which are served), can serve a single completer (e.g. in tests), and looks for problems a moment after the last change, as editors send every keystroke. Merging keeps overloads: completions are the same only if they insert the same name with the same parameters (label details). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Merging dropped completions inserting the same name with the same parameters even within one server's list, e.g. Groovy's each() from Iterable, Collection, List and Object. Only those duplicating an earlier server's are dropped now. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ScriptLanguageServer: a base class keeping the open documents, answering requests by asking the subclass about a document and offset, resolving documentation on demand, and publishing problems a moment after each change. Callable: something to call, as a snippet completion with typed label details, or as a rated signature. Members: completions for Java classes' members, by reflection. Signatures: rating calls' arguments, on Callables. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Callable.of(item) reads a callable from a completion's label details. Merging compares completions without an edit (e.g. jedi's "length") by the word before the caret, so that they match those replacing more (e.g. Jython's "s.length"). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PythonCompletionServer wraps a Python language server (e.g. jedi-language-server) as editors want it: it asks at a word's first character and caches the answer, cancels superseded requests, filters by the word typed, and gives the top completions their signatures. DelegatingLanguageServer forwards everything, for such decorators. Cancelling a completion now cancels the request it waits on, through TransformingLanguageServer and LaunchedLanguageServer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Code intelligence is now lsp4j language servers throughout. What code-api had that servers still need moves into code-lsp: ScriptDocument, ScriptDialect, PositionMap and ClassIndex; script parameter parsing, as LanguageServerService.scriptParameters; and Environment in place of EnvironmentContext. Positions replaces LspClient's offset conversions, and Python signatures are parsed by PythonCompletionServer. Gone: the code-api adapter, LspClient and PythonLspClient. In their place, BindingsLanguageServer completes a running interpreter's variables, for languages without servers. The outside-editor server (SciJavaLanguageServer) is now a front for the service's merged servers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A ScriptSession is where a client (an editor, a REPL, a tool checking a script) asks about a SciJava script, in terms of its text and offsets: it keeps the servers' copy up to date, names the script (its file's URI, or an untitled one; following Save As), and receives its problems. Each request carries the text it is about. The LanguageServerService makes sessions, each owning a merged server of its own; for languages no plugin serves, a session completes a running interpreter's variables. diagnose is now a session asked once. Where scripts run can change without them changing (e.g. an environment is built): plugins now say so via environmentsChanged(), which moves open scripts to the servers of where they run now. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Nothing runs it: editors in the SciJava process ask the service for servers directly, and none outside it has been wired to this one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
It is specific to Python (completing Java members on objects a Python server cannot see into), so it belongs with the one plugin using it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The module does several things, each now in a package of its own: - org.scijava.code.lsp: brokering between scripts and servers (the service, plugins, script sessions, environments), plus SciJava's additions to LSP (updating completion lists, rated signatures). - .launch: running language server processes. - .script: SciJava scripts as plain code (#@ parameters declared). - .jvm: a toolkit for in-process servers of JVM languages. - .impl: the implementation, not exported. The specialized packages depend only on the base package, never on each other. A module-info exports the four public packages; it is open, for SciJava Common's reflection. Package infos say what each is for. To keep the implementation internal, clients get sessions via ScriptSession.owning, the service no longer offers its servers directly (LanguageServerService.server), and data is no longer assumed to be Gson's (the module no longer reads Gson). Tests run on the classpath, not the module path, where the tests' plugin index would shadow the module's own. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ScriptSession.server, language, text and uri, and setFile(File), had no callers outside tests; nor had LanguageServerService.session(File). Clients set a file supplier, as editors follow Save As. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drop UpdatingCompletionList and RatedSignatureInformation, SciJava's additions to LSP: - Merged completions come once all servers have answered, as signature help and hover already did; clients that must not wait ask asynchronously. - In-process servers still order overloads best fit first (Fit moves to Signatures), but signatures no longer carry their fit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drop MergedLanguageServer: a language is served by the highest-priority plugin supporting it, and a server combining several sources (e.g. an in-process analysis and jedi) combines them itself. Each session gets a server of its own, which it initializes (awaiting the answer, as LSP says) and shuts down when closed. Server processes stay shared, one per environment. With it go the service's shared servers and diagnostics routing, moving scripts between servers as where they run changes (LanguageServerPlugin.environment, environmentsChanged), and waiting for several servers' problems. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both are ordered places imports are resolved from; each server reads them as its language does. Nothing read the classpath. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A script document maps positions between its script and its text itself: by offset, and by LSP position and range. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A ScriptSession now transforms each script into plain code of its language, in the dialect of the plugin offering its server (LanguageServerPlugin#dialect), and maps positions in the server's answers back. Servers need know nothing of SciJava scripts. TransformingLanguageServer, which did this inside some servers, is gone, and so is ScriptDocument's unused environment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The declarations a session adds are real code: imports of the parameters' types bind names (e.g. numpy) the script does not. A dialect now says which name each import binds (ScriptDialect#imported); unless the script mentions it, the session leaves completions of it out (see ScriptDocument#added). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ScriptDialect and ScriptDocument are now part of how sessions talk to servers (LanguageServerPlugin#dialect, ScriptSession), so they move from org.scijava.code.lsp.script into org.scijava.code.lsp. This also ends the cycle between the two packages. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A server process is now offered by a ServerProcessPlugin, in the core package: it starts the process for an LSP language and an environment, and says how to initialize it (options, workspace root). The launch package (LspServerPlugin, LspServerSpec, LspConnection) and the separate LspService are gone; LanguageServerService#launch serves the processes itself. The implementation follows the new names: ServerProcesses keeps a language's ServerProcess per environment, and a ProcessLanguageServer shows one as an lsp4j LanguageServer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LanguageServerService.launch can now take a function from a document to its environment, which ProcessLanguageServer consults before each request: a document whose environment changes is closed on its old server process and opened on its new one. Completions remember which process answered them, so that they are resolved by it. Also note in ScriptDialect's javadoc why a dialect describes the translation of scripts, rather than doing it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A ServerProcessPlugin for python, moved here from scripting-appose-python so that script languages can use jedi without depending on each other (e.g. scripting-jython). It runs jedi-language-server in a small Appose environment of its own (code-lsp-jedi: Python 3.12, Jedi, NumPy), built the first time it is needed, under a launcher that exits once the JVM is gone. JediLifecycleIntegrationTest starts a server via LanguageServerService, asks it about a script, and checks that its process stops with the context. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LanguageServerService.launch(languageId, environment) now gets a server for any LSP language a plugin serves, in-process or not: plugins say so via supports(String), and supports(ScriptLanguage) for sessions. Server processes are served by ProcessLanguageServerPlugin (internal), which owns them and stops them when disposed; LanguageServerService also tells whether a language is served at all, via supports(String). Servers analyze scripts in one environment, given when created: plugins whose scripts say where they run keep a server per environment themselves. ProcessLanguageServer no longer moves documents. Also: - Rename ScriptLanguageServer to AbstractLanguageServer. - Throw rather than return null when no plugin serves a language. - Remove LanguageServerService.diagnose: a session does as much. - Stop started server processes directly on shutdown, rather than via the starter, whose shutdown could drop the stop. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Only JythonLanguageServerPlugin (renamed from JythonServerPlugin) is public: nothing outside the module used the rest. JythonDev's messages, printed to System.out by default, are gone; EmptyDotAutocompletions is a lambda, and JythonImportFormat part of JythonImports. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Jython's own analysis (its parser, scopes and dot completions) is gone: JythonLanguageServer is jedi-language-server, which knows Java classes from the stubs of the packages a script imports. Beside its completions come Java classes not imported yet (importing themselves as accepted); a live interpreter's variables are declared to it, as a module the script's text imports first; and names like __name__ are offered only if Jython has them. So Jython completion, like Python's, needs jedi's environment, built the first time it is used; and the tests now run with it. Lost, for now (see the TODOs): the variables of a parameter's type, first in a call's arguments; and all of a method's overloads in signature help. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BindingsLanguageServer is offered for plaintext by BindingsServerPlugin, both back in impl. So a session always asks the service for its servers: of its dialect's LSP language if served, and otherwise (or if that server cannot be created) of plaintext, as its script is then shown. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Gone, with nothing calling them (Jython's own analysis was the last to use the first two): Callable.of, Signatures.arguments, Document.version and ClassIndex.addRoots. Private, as only their classes use them: most of ClassIndex's documentation lookup, Members.field, method and signature, Signatures.rate and loadType, and Callable.parameterList. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Note that each of the |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ScriptDialect.header now takes Variables (a type, and a value when known) rather than types, and ScriptSession gives it a live interpreter's bindings too, alongside the script's parameters (which win). So each dialect declares them as its language should, and the servers no longer read the live bindings themselves: GroovyLanguageServer's own dialect and JythonLanguageServer's stub module of them are gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A number followed by a dot is the start of a decimal only when nothing, or a digit, follows the dot: a name means a member access, of the number (an object in Groovy). Test numbers on their own, not with keywords. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Its bindings change as code runs in it, so they are read anew rather than kept. And the fields need not repeat their getters' docs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Not with inferred types; and of a typed local and a parameter too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
As e.g. testCompleteStaticMembers, with completing qualified class names apart from imports. And test where imports are inserted via completion, so importOffset can be private; explain the snippet test, too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Groovy server's description points to scripting-groovy for SciJava scripts, rather than saying how they (and live bindings) become Groovy. The Jedi and Jython READMEs are unwrapped, one line per paragraph, as the others now are; their words are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The server package's Javadoc was swallowed by its unclosed license header: close it, and say how to offer a server of your own, with an example. Link the LSP specification (3.18) and jedi-language-server's options from ServerLauncher, whose initialization options LSP leaves to each server. And explain each part of JediLauncher's options: its diagnostics ones are the server's defaults. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@gselzer I would encourage you to skip converting this to a full PR that requires review; once it's working and all implementation questions are resolved, why not just merge it?
This is very thoughtful, but I am of the opinion that any dependency wrangling for size is premature optimization. Let's think from an end user's perspective: lsp4j is distributed once and although other libraries inherit the dependency, they don't duplicate its distribution. Also you're quoting a number in KB. That is not a unit of size that deserves discussion without strong compelling evidence to the contrary.
This is legitimate. I suggest choosing for best chance at long-term stability (Pyright). Zuban would be my second choice. |
|
Thanks @hinerm!
If nobody else wants to review it then I will certainly do so!
Fair enough, let's go with it!
Your preferences will be taken into account 😄 |
This PR introduces a few new modules to the monorepo:
scijava-code-lspwranglesLanguageServersThe central package,
scijava-code-lspis designed to assist with the management of Language Servers, following the Language Server Protocol, usinglsp4j. ALanguageServerServiceoffers the ability to launch aLanguageServerbased on a desired programming language andEnvironment(interpreter path, module search paths, existing bindings) for general usage.As the central use case is assisting the development of SciJava scripts, a
ScriptSessionclass is also offered. Users provide (1) the text of a SciJava script to aScriptSessionand (2) aLanguageServerService, and can then query for autocomplete, hover info, etc. as well as listen to diagnostic feedback.Of course,
LanguageServers (can/should) not understand SciJava scripts as written, soscijava-code-lspoffers aScriptDialectplugin describing how to convert text specific to SciJava scripts (e.g. script parameters) to headers that a language server could under stand and reason about. Thescijava/scripting-Xrepositories are where I'm planning for these implementations to live (PRs to follow shortly)Finally,
scijava-code-lspcontains utilities common to multiple language servers, which leads us into:scijava-code-lsp-groovyoffers aGroovyLanguageServerThis language server is fully in-process and uses Groovy's compiler process to develop a typed AST that is used to answer LSP queries.
scijava-code-lsp-jedioffers aPythonLanguageServerThis language server is inter-process and uses
jedi-language-serverto answer LSP queries.scijava-code-lsp-jythonoffers aJythonLanguageServerThis language server is inter-process and uses
PythonLanguageServerindirectly. It also autogenerates type stubs to enable java types to be handled by the Python language server.Points of Discussion
lsp4jis a significant dependency to bring in (~640KB) - I'd be receptive to the case that it's not worth it to use the existing work, and we could consider ascijava-code-lsp-apimodule, because it's unfortunate to bring this in because a library that just wants to declare aScriptDialectimplementation now gets this dependency.jediwas chosen by Claude for the Python language server, but that is apparently in maintenance mode. We probably ought to choose a different server - options includety,zuban, andpyright- should be very easy to switch.