Fix: pdflatex shell escape not launching Cygwin mintty

When you use pdflatex with shell escape to launch Cygwin’s mintty.exe, Windows command prompt opens instead of the Cygwin terminal because pdflatex’s shell escape mechanism executes commands through Windows’ default command processor (cmd.exe), which doesn’t maintain the proper Cygwin execution context. This causes mintty.exe to be interpreted as launching cmd.exe rather than as a Cygwin terminal.

Contents

• Understanding the Shell Escape Mechanism
• Why mintty Opens Command Prompt Instead of Cygwin Terminal
• Solutions for Proper mintty Launch
• Alternative Approaches
• Verification and Testing

Understanding the Shell Escape Mechanism

LaTeX’s shell escape feature allows documents to execute external commands through the \write18 command or the expl3 package’s \sys_shell_now:n. When you use pdflatex with the -shell-escape option, these commands are executed through the system’s default shell.

On Windows, the default shell is typically cmd.exe, which means that when your LaTeX document calls C:/cygwin64/bin/mintty.exe -, the command processor interprets this differently than when you run it directly from a Cygwin environment.

According to the LaTeX2e unofficial reference manual, shell escape commands are subject to security restrictions and may behave differently depending on how they’re invoked.

Why mintty Opens Command Prompt Instead of Cygwin Terminal

The issue occurs because of how Windows handles the execution of mintty.exe through cmd.exe:

1. Command Processor Interception: When pdflatex executes the shell escape command, it goes through Windows cmd.exe, which may interpret mintty.exe as a command to launch a new Windows command prompt rather than as a Cygwin terminal emulator.

2. Missing Cygwin Context: The command C:/cygwin64/bin/mintty.exe - works when run from a Cygwin bash shell because the environment is already set up correctly. However, when executed from cmd.exe (which is what pdflatex uses), the command lacks the proper Cygwin context.

3. Path Resolution Issues: Windows command processor may not properly resolve the Cygwin paths or understand the mintty.exe executable’s purpose without additional parameters.

As noted in the Stack Overflow discussion, this is a common issue when trying to launch Cygwin applications from LaTeX’s shell escape mechanism.

Solutions for Proper mintty Launch

Solution 1: Use the -e Parameter with mintty

The most reliable solution is to use mintty’s -e parameter to specify what command should be executed, ensuring proper context:

\documentclass{article}

\begin{document}
\ExplSyntaxOn
\sys_shell_now:n {C:/cygwin64/bin/mintty.exe~-e~/bin/bash~-l~}
\ExplSyntaxOff
\end{document}

This approach explicitly tells mintty to execute bash with login shell parameters, which forces it to maintain the Cygwin environment.

Solution 2: Use cygpath for Path Conversion

When working with mixed Windows and Cygwin paths, use the cygpath utility to ensure proper path resolution:

\documentclass{article}

\begin{document}
\ExplSyntaxOn
\sys_shell_now:n {bash~c~"$(cygpath~-u~'C:/cygwin64/bin/mintty.exe')~-e~/bin/bash~-l~"}
\ExplSyntaxOff
\end{document}

Solution 3: Create a Batch File Wrapper

Create a simple batch file that properly launches mintty and call it from LaTeX:

Create launch_mintty.bat:

@echo off
C:\cygwin64\bin\mintty.exe -e /bin/bash --login -i

Then call it from LaTeX:

\documentclass{article}

\begin{document}
\ExplSyntaxOn
\sys_shell_now:n {launch_mintty.bat}
\ExplSyntaxOff
\end{document}

Alternative Approaches

Using the Default Cygwin Bash Shell

Instead of trying to launch mintty directly, consider using the Cygwin bash shell through shell escape:

\documentclass{article}

\begin{document}
\ExplSyntaxOn
\sys_shell_now:n {C:/cygwin64/bin/bash~--login~-c~"echo~'Hello~from~Cygwin'"}
\ExplSyntaxOff
\end{document}

Setting Up mintty as Default Terminal

According to the Unix Stack Exchange discussion, you can configure mintty as the default terminal by modifying the Cygwin /etc/passwd file or creating a batch file that properly initializes the environment.

Create a batch file with:

@echo off
C:\cygwin64\bin\mintty.exe /bin/bash --login -i

Using Different Invocation Methods

From the Super User solution, you can use various invocation methods:

\documentclass{article}

\begin{document}
\ExplSyntaxOn
% Simple execution
\sys_shell_now:n {C:/cygwin64/bin/mintty.exe~-e~/bin/bash~-l~-c~'echo~Hello~World'}

% With interactive shell after execution
\sys_shell_now:n {C:/cygwin64/bin/mintty.exe~-e~/bin/bash~-l~-c~'echo~Hello~World;~bash'}
\ExplSyntaxOff
\end{document}

Verification and Testing

After implementing any of these solutions, verify that mintty launches correctly by:

1. Checking the LaTeX Log File: Look for any error messages or warnings related to the shell escape command execution.

2. Testing Commands Individually: Test each command component separately to isolate the issue.

3. Using Debug Output: Add debug information to your LaTeX document to see exactly what command is being executed.

4. Checking Environment Variables: Ensure that necessary environment variables are properly set when the command executes.

Remember to always compile your LaTeX document with the -shell-escape option:

pdflatex -shell-escape your_document.tex

As mentioned in the LaTeX.org forum, for MiKTeX systems, you might need to use --enable-write18 instead of -shell-escape in some cases.

Sources

1. Launching Cygwin (mintty.exe) from Windows pdflatex (via shell escape) actually launches Windows’ command prompt? - Stack Overflow

2. pdflatex + Cygwin bash: shell escape is not working? - TeX - LaTeX Stack Exchange

3. Using mintty as default terminal on Cygwin and default shell as bash - Unix & Linux Stack Exchange

4. Command line options (LaTeX2e unofficial reference manual)

5. How can I enable shell-escape? - TeX - LaTeX Stack Exchange

6. Cygwin: Run a script in mintty/bash - Super User

7. Enable shell escape for epstopdf - LaTeX.org

Conclusion

The behavior you’re experiencing occurs because pdflatex’s shell escape mechanism executes commands through Windows cmd.exe, which doesn’t properly maintain the Cygwin execution context needed for mintty to function as intended. To resolve this issue:

1. Use the -e parameter with mintty to explicitly specify the command to execute
2. Leverage cygpath for proper path conversion between Windows and Cygwin formats
3. Consider creating a batch file wrapper that properly initializes the Cygwin environment
4. Test different invocation methods to find the one that works best for your setup

The most reliable solution is typically using C:/cygwin64/bin/mintty.exe -e /bin/bash --login -i in your LaTeX shell escape command, as this explicitly tells mintty to launch a bash login shell, ensuring the proper Cygwin context is maintained.