[tools]Import Wizard for Installation Setup

[JinkunLiu]

Linked issue: #586

Purpose of change

Import Wizard for Installation Setup

Install Flink & Install PyFlink Environment

Tests

I have test shell on Ubuntu 22.0 and macOS 15.0

API

not

Documentation

• doc-needed
• doc-not-needed
• doc-included

[wenjin272]

Thanks for your work @JinkunLiu. I tried using this script and found some issues, which I've listed in the comments.

From a user experience perspective, I feel there is a significant issue: users are required to understand too many cli flag and environment variables, such as --install-flink, --enable-pyflink, INSTALL_FLINK, FLINK_VERSION, and FLINK_AGENTS_VERSION. Even the value of INSTALL_FLINK can be set to ask.

I'm wondering if we could retain only a single CLI flag, --default, to indicate that all operations should follow the default path, while using an interactive mode for user selection in all other cases. Users can interactively choose whether to install Flink, select the Flink version (1.20.3, 2.0.1, 2.1.1, 2.2.0), and specify the flink-agents version (0.1.1, 0.2.1, 0.3). Additionally, users can provide interactive responses regarding the location of an existing Flink installation and the target installation directory.

We can retain these environment variables; when they are detected as set, we will print a log and skip the prompt.

[JinkunLiu]

Hello @wenjin272 thanks for you review

users are required to understand too many cli flag and environment variables.

CLI flags are reserved for non-interactive installation mode

I'm wondering if we could retain only a single CLI flag, --default, to indicate that all operations should follow the default path

Actually, the code already does this — you can try running bash install.sh directly and the interactive mode for user selection will appear.

Users can interactively choose whether to install Flink, select the Flink version (1.20.3, 2.0.1, 2.1.1, 2.2.0), and specify the flink-agents version (0.1.1, 0.2.1, 0.3).

Now the install shell provides the latest version as default. Users can use all the features we provide in the latest version.If they need special version about Flink or Flink-agents they can use cli flag.

[wenjin272]

Hi, @JinkunLiu, thanks for your update.

Since flink-agents provides multi-version support, and considering that some developers' clusters may not yet be compatible with the latest Flink version, I believe it might be too inflexible to always download the latest Flink version.

Now the install shell provides the latest version as default. Users can use all the features we provide in the latest version.If they need special version about Flink or Flink-agents they can use cli flag.

This is what I mean when I say that users need to understand many CLI flags. Why can't users be allowed to select a specific Flink version, rather than being required to specify it via a CLI flag? We provide a GUI-like installation wizard, but users are still required to manually specify certain cli flag to achieve specific goals. I believe this creates a fragmented user experience.

I think for some less frequent requirements, such as dry-run and verbose, we can simply only provide CLI flags. However, for some high-frequency requirements, I believe it would be a more user-friendly experience to allow users to select options during the installation guide, rather than requiring them to read and understand the information returned by help and setting CLI flags or environment variables.

I think these requirements contains:

1. intall dir: we can provide two options: default vs. custom, and if user choose custom, they need input the custom dir.
2. install flink: yes vs. no, and check the FLINK_HOME when user choose no.
3. flink-version: 4 options, 1.20.3, 2.0.1, 2.1.1, 2.2.0(recommend)
4. install pyflink: yes vs. no

After the user makes the above selections, we will then output the Install Plan.

Additionally, when running the script on my Mac, I encountered the error die: command not found. You may need to replace it with a more universally available command.

[wenjin272]

Hi, @JinkunLiu. I have verified the install.sh script locally and successfully submitted the example to the Flink standalone cluster. LGTM. Please take a look at your convenience @xintongsong.

The only issue I encountered was being unable to complete the download of Flink within the timeout period. I'm not sure whether this is a personal network problem or a common occurrence.

Additionally, now that an install script has been provided, we need to refactor the Installation section in the documentation. I think this can be done either in this PR or in a separate one.

[wenjin272]

The flink-2.2.0-bin-scala_2.12.tgz file is 544 MB in size, and setting the max-time to 30 seconds imposes high demands on network performance.
Based on my actual tests, it takes approximately 500 seconds in my office network. I recommend setting it to 600 seconds.

[JinkunLiu]

Good catch. This is a mistake which introduced when i fix another issue. Fixed it.

[xintongsong]

Thanks for working on this, @JinkunLiu and @wenjin272 .

I did not closely check all the codes, but have tried the script out instead. Here are a few issues that I run into, which might be improved.

1. An extra -- in line 68 causes gum downloading failure. However, the script did fail and continued to execute without gum, leading to a poor interaction experience. I think we should make gum required, and fail the script if it's not available.

2. There's no downloading progress. It takes a couple of minutes to download the Flink binary release, and during that time the script acts like dead. Might be better to show some progress, letting users know that it's actually downloading.

3. Setting up the python environment produces too many logs, which I believe is from pip install. Can we make the pip install output rolling, like it always shows the latest N lines of outputs, taking at most N lines of the screen space?

4. The plan shows FLINK_HOME .//flink-2.2.0 on my screen. There's an extra /.

5. It's like a one-way road. There's no way to go back.

• There's no way to go back to the previous step and change the settings.
• At the stage of confirming the installation plan, if you choose No, the script exits, without giving you a chance to modify the plan.
• When I input a wrong path of exiting flink installation, the script exits, without giving me a second chance to re-input it.

6. I tried ctrl+c to interrupt the script. It didn't exit the script, but instead skipped the configuration and directly started the installation.

In addition to the above experience issues, I think for this over 1k loc script, we need test cases to verify it. The -- bug, introduced by fixing the download time limit, is a strong indicator that the script lacks protection that any simple maintenance update may break things.

[xintongsong]

There's an extra -- that causes the download failure.

[xintongsong]

Suggested change

	ui_section "Install plan"
	ui_section "Installation plan"