Skip to content

Introduce Cursor/VS Code support #655

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
cloud-fan merged 4 commits into from
Dec 29, 2025
Merged

Introduce Cursor/VS Code support #655

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
2c572cf
Introduce Cursor/VS Code support
cloud-fan Dec 27, 2025
e7b197d
improve
cloud-fan Dec 27, 2025
12092cd
Apply suggestions from code review
cloud-fan Dec 28, 2025
5fc9a7d
Apply suggestions from code review
cloud-fan Dec 28, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions developer-tools.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -470,6 +470,26 @@ debug in IntelliJ as usual.
To exit remote debug mode (so that you don't have to keep starting the remote debugger),
type "session clear" in SBT console while you're in a project.


<h4>Cursor/VS Code</h4>

You can leverage the power of *Cursor* or *VS Code* for Spark development by using the Metals extension.
Metals provides robust Scala support through the Build Server Protocol (BSP).

Initial setup:
- *Install Metals*: Search for and install the Scala (Metals) extension from the Marketplace.
- *Open Project*: Open your Spark root directory.
- *Select Build Tool*: When prompted by the popup in the bottom-left corner, select sbt to read build definition.
- *Confirm Build Server*: Metals defaults to using Bloop as the build server. Do not change this setting, as Bloop provides the fastest compilation and best integration for local development. Open the Command Palette (Cmd+Shift+P or Ctrl+Shift+P), run `Metals: Run doctor`, and verify the status dashboard. Ensure it shows: `Build definition is coming from sbt` and `Build server currently being used is Bloop`.

If you encounter compilation errors or the IDE fails to recognize symbols, follow these steps to perform a clean import:
1. *Stop Bloop Server*: Run the following command in your terminal to stop the bloop server: `pkill -f "bloop.BloopServer"`. If you have installed bloop CLI you can run `bloop exit` instead.
Copy link
Member

@yaooqinn yaooqinn Dec 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this equivalent to Metals: Restart server?

Copy link
Contributor Author

@cloud-fan cloud-fan Dec 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

metals server is different from bloop server.

2. *Clean Workspace*: Run the following command in your terminal to remove cached build metadata: `rm -rf .bloop/ .metals/ .bsp/`.
Copy link
Member

@yaooqinn yaooqinn Dec 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to rm ./project/.bloop too?

Copy link
Contributor Author

@cloud-fan cloud-fan Dec 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't remove it and there is no issue. I checked a bit and this .bloop is only for building the sbt build definition, which is simply the SparkBuild.scala. It's unlikely to cause issues.

3. *Reload Window*: Run `Developer: Reload Window` from the Command Palette, and select sbt when the prompt reappears.
4. *Import Build*: Run `Metals: Import build` and wait for the process to complete.
5. *Connect to Server*: Run `Metals: Connect to build server` and wait for the process to complete.
6. *Confirm*: Run `Metals: Run doctor` and verify the status dashboard.

<h4>Eclipse</h4>

Eclipse can be used to develop and test Spark. The following configuration is known to work:
Expand Down
23 changes: 23 additions & 0 deletions site/developer-tools.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -606,6 +606,29 @@ <h5>Trigger the remote debugging</h5>
<p>To exit remote debug mode (so that you don&#8217;t have to keep starting the remote debugger),
type &#8220;session clear&#8221; in SBT console while you&#8217;re in a project.</p>

<h4>Cursor/VS Code</h4>

<p>You can leverage the power of <em>Cursor</em> or <em>VS Code</em> for Spark development by using the Metals extension.
Metals provides robust Scala support through the Build Server Protocol (BSP).</p>

<p>Initial setup:</p>
<ul>
<li><em>Install Metals</em>: Search for and install the Scala (Metals) extension from the Marketplace.</li>
<li><em>Open Project</em>: Open your Spark root directory.</li>
<li><em>Select Build Tool</em>: When prompted by the popup in the bottom-left corner, select sbt to read build definition.</li>
<li><em>Confirm Build Server</em>: Metals defaults to using Bloop as the build server. Do not change this setting, as Bloop provides the fastest compilation and best integration for local development. Open the Command Palette (Cmd+Shift+P or Ctrl+Shift+P), run <code class="language-plaintext highlighter-rouge">Metals: Run doctor</code>, and verify the status dashboard. Ensure it shows: <code class="language-plaintext highlighter-rouge">Build definition is coming from sbt</code> and <code class="language-plaintext highlighter-rouge">Build server currently being used is Bloop</code>.</li>
</ul>

<p>If you encounter compilation errors or the IDE fails to recognize symbols, follow these steps to perform a clean import:</p>
<ol>
<li><em>Stop Bloop Server</em>: Run the following command in your terminal to stop the bloop server: <code class="language-plaintext highlighter-rouge">pkill -f "bloop.BloopServer"</code>. If you have installed bloop CLI you can run <code class="language-plaintext highlighter-rouge">bloop exit</code> instead.</li>
<li><em>Clean Workspace</em>: Run the following command in your terminal to remove cached build metadata: <code class="language-plaintext highlighter-rouge">rm -rf .bloop/ .metals/ .bsp/</code>.</li>
<li><em>Reload Window</em>: Run <code class="language-plaintext highlighter-rouge">Developer: Reload Window</code> from the Command Palette, and select sbt when the prompt reappears.</li>
<li><em>Import Build</em>: Run <code class="language-plaintext highlighter-rouge">Metals: Import build</code> and wait for the process to complete.</li>
<li><em>Connect to Server</em>: Run <code class="language-plaintext highlighter-rouge">Metals: Connect to build server</code> and wait for the process to complete.</li>
<li><em>Confirm</em>: Run <code class="language-plaintext highlighter-rouge">Metals: Run doctor</code> and verify the status dashboard.</li>
</ol>

<h4>Eclipse</h4>

<p>Eclipse can be used to develop and test Spark. The following configuration is known to work:</p>
Expand Down