PyCharm debugging with separate console

In PyCharm, debugging a Python script puts its I/O in the PyCharm console. Unfortunately, it is not a complete console, so if your script uses advanced I/O features, such as waiting for key presses or colored output, it will not work correctly.

In order to have it work, you need to run the script in a separate console window, and connect to the PyCharm debugger. This is done as follows.

First of all, create an external tool which will run the script in a dedicated console.

Name: gnome-terminal-<myprogram>
Program: gnome-terminal
Arguments: — /bin/bash -c “sleep 2; venv/bin/python3 pycharm; sleep 2”
Working Directory: $ProjectFileDir$

This runs <myprogram> with the Python executable in the venv/bin environment, in a dedicated gnome-terminal.

Then, create a new Run/debug configuration “Python Debug Server”.
Name: Remote <myprogram>
IDE host name: localhost
Port: 45257
Before launch: Add New Task -> Run external tool “gnome-terminal-<myprogram>”

At the end, in the program code add the following:

try:
    if sys.argv[1] == "pycharm":
        # force keeping the original stdin.
        original_stdin = sys.stdin
        import pydevd_pycharm
        pydevd_pycharm.settrace('localhost', port=45257, suspend=False, trace_only_current_thread=False)
        sys.stdin = original_stdin
        sys.argv = sys.argv[0:1] + sys.argv[2:]
except IndexError:
    pass

If the first argument passed to the program is “pycharm”, it will be erased and the program will connect to the debug server started by the debug configuration.


Wireguard MTU

Wireguard causes strange problems if its MTU setting is the “default”. Lower it to 1280, because otherwise you have things like SSH hanging while fetching from a Git repository!

If you have to do it, remember also to check that the configurtation survives a reboot.


Conversion of a Mercurial repository to Git

These instructions apply to Windows.

We use hg-git, which is installed by default with TortoiseHg version 7.0.1. Note that with a previous version we had problems, so it is necessary to upgrade first.

Remember that everything must be committed into the original Hg repository to be migrated. If you don’t commit something, it will be lost.

We will call the folder in which the Hg repository resides hg-repo, and the new folder in which the Git repository will be created git-repo.

First of all we need an empty Git repository.

> md git-repo
> cd git-repo
> git init

Then we push the Hg repository into the Git repository, using a bookmark as a reference.

> cd hg-repo
> hg bookmarks hg
> hg push git-repo

Now we can get a working copy in the Git repository folder.

> cd git-repo
> git checkout -b master hg

Then we push the Git repository to a new repository in Gitea. In Gitea, we create a new empty repository. We also choose the default branch to be “master” instead of “main”, but any choice will do. The gitea creation process will tell you how you can push to the repository, but we use HTTPS and not the GIT protocol. So:

> cd git-repo
> git remote add origin https://gitea-server/user/repository.git
> git push -u origin master

This pushes the master branch. Everything can be pushed, and the previous history is preserved.


VirtualBox on Windows 11

VirtualBox on Windows 11 will run VMs with the “Green Turtle” icon, at a very reduced speed, unless some steps are made (which may also reduce Windows 11 security).

Having W11 24H2 installed, the following set of actions was necessary.

  1. In Windows features, disable: Containers – Hyper -V – Virtual Machine Platform – Windows Hypervisor – Windows Sandbox – Windows Subsystem for Linux (some were already disabled)
  2. In the “Core Isolation” Settings, disable “Memory Integrity”.
  3. Run in the administrator command prompt:
    bcdedit /set hypervisorlaunchtype off
  4. Download the Device Guard and Credential Guard hardware readiness tool from Microsoft, uncompress the .zip file, open an administrator PowerShell and run from the uncompressed directory:
    .\DG_Readiness_Tool_v3.6.ps1 -Disable
  5. Reboot the PC. You will be asked whether to disable Credential Guard (say NO) and to disable Device Guard (say YES). You’ll lose the ability to log in with the fingerprint or PIN, and you will need the password.

Trying to re-enable the fingerprint log in with the command
Set-ItemProperty -Path "HKLM:\System\CurrentControlSet\Control\Lsa" -Name LsaCfgFlags -Value 1"
did not work.

Sources:

https://www.youtube.com/watch?v=cTyjXy26mqs


EOS R8 on external power

Here are a few notes about supplying external power to the Canon EOS R8 mirrorles camera.

Canon sells a “fake battery” adapter which uses a 220V power supply, but the R8 has also a USB-C connector that can be used to power it and also recharge the battery without removing it.

Since there is plenty of external power packs available, this can be a much better solution if you must keep the camera continuously powered on for a long time, making a battery change not acceptable, and if you don’t have access to mains power.

However there are conflicting reports on the internet, and the user manual is very concise, stating that the camera can be powered by USB with the Canon provided power adapter (which requires mains power as well).

The tests with an external power bank lead to the following results.

Connecting the power bank when the camera is ON will not allow power to the camera. It isn’t enough to switch the camera OFF and ON again. You have to connect the power bank when the camera is OFF. A green light will turn on on the camera. If you switch on the camera, you will see the battery symbol switching to GRAY.

You need a power bank that is able to supply at least 45 W. I tested with a normal USB-A 3.0 power supply (which is able to output 18 W) and it didn’t work. An USB-C power supply, able to provide 45 W, was able to switch on the camera.

It is possible that some power rating between 18 W and 45 W works but I couldn’t test it.

The internal battery of the camera will be charged when the camera is switched off.

I don’t know yet whether the battery is charged when the camera is switched on.

In case the USB supply gets disconnected, you can continue using the camera on the internal battery, but you have to switch off the camera to reconnect the USB supply, otherwise it won’t work.

 


Some benchmarks for VMWare and VirtualBox

Finding the correct configuration for using a virtual machine for running the Yocto builds is not easy – and for other things also.

I have a new laptop, a ThinkPad with a 13th Gen Intel(R) Core(TM) i7-1360P 2.20 GHz processor, 32 GB of RAM, and Windows 11 Pro (23H2).

So I decided to migrate the very old Mandrake Linux VM I use for some old compiling builds, and the new Ubuntu 20.04 LTS VM I use for Yocto builds.

The first issue has been with using the Ubuntu VM with VMWare 17.5. It simply does not work (see here), and I had to revert to VMWare 17.0 to have a working machine.

The second issue has been with the Mandrake Linux VM. It is simply incredibly slow – the kernel loading time is on the order of minutes.

Searching on the Internet finds that there is a potential issue with the new Windows hypervisor settings. See here. The problem seemed to be that, since in the log I had the line

2023-01-22T09:26:16.952Z In(05) vmx Monitor Mode: ULM

I switched the memory isolation off and set the hypervisorlaunchtype to off. It did nothing and the performance did not change.

So I migrated the VM to Virtual Box, which worked (after some fighting to install the linux source package of the very old Mandrake 9.2 distribution, and compiling the virtual box Guest Additions).

At this point I decided to test the performance of VMWare and VirtualBox with the same VM.

I created two virtual machines with 8 processor cores, 8 GB of RAM and 20 GB of HD, with Ubuntu 20.04 LTS. In VMWare, the mitigation against the side-channel attacks was switched to off putting in the .vmx file the line:

ulm.disableMitigations="TRUE"

The performance benchmark was measured with hardinfo and sysbench.

After the first run with VirtualBox, done with the memory isolation off and hypervisorlaunchtype off (“modded” Windows cfg), I decided to switch back the memory isolation and hypervisorlaunchtype configuration to the standard one (“default” Windows cfg).

Machine VirtualBox VirtualBox VMWare VirtualBox (VMWare HD) WSL2 Ubuntu
Windows cfg modded default default default default
sysbench cpu
(events/s)
Threads=1 higher is better n.a. 3990.63 1730.44 4103.91 4030.06
Threads=2 higher is better n.a. 7580.10 2840.87 7991.29 7791.52
Threads=8 higher is better n.a. 20868.16 11522.57 23974.51 26679.99
Threads=16 higher is better n.a. 20990.99 11574.03 25237.46 38108.54
hardinfo CPU Blowfish lower is better 0.62 0.54 1.11 0.50 0.31
CPU CryptoHash higher is better 1509.54 1103.02 371.26 1520.22 2006.11
CPU Fibonacci lower is better 0.33 0.35 1.01 0.33 0.31
CPU N-Queens lower is better 7.99 6.98 28.60 6.96 9.72
CPU Zlib higher is better 0.05 0.04 0.56 0.05 3.18
FPU FFT lower is better 0.44 0.47 1.88 0.52 0.41
FPU Raytracing lower is better 1.06 1.19 4.71 1.08 1.01
GPU Drawing higher is better 20098.72 20475.80 n.a. 20555.37 24410.46

 

VirtualBox beats VMWare in all tests, though the CPU load measured by the Windows Resource Monitor is 40% with VMWare and 80% with VirtualBox. However, we also observed that running with VMWare and more than 8 processor cores “hangs” the Windows host.

The last column contains the same benchmark made with the Windows Subsystem for Linux and an Ubuntu 20.04 VM. This is not limited to 8 processor cores, so the performance can be higher. However, the VirtualBox VM is closer to the WSL2 VM.

Unexpectedly, VirtualBox seems to be better than VMWare in my current environment.


Convert a data disk from MBR to GPT (without losing data)

The reason for this is very simple. MBR partitions do not work above 2 TB. I had (long story…) a 2 TB NTFS partition on a 4 TB USB hard drive. Windows provides a command-line tool, MBR2GPT.EXE, but it works only on bootable disks with Windows installed. And a request for making it work with data disks has been dismissed from Microsoft.

The prevailing wisdom is “do a backup”. But even if I do – or already have – a backup, if I want to enlarge a 2 TB partition it will take a long time to rewrite all data to the disk. So it is much better to convert the disk in-place.

You can find procedures on the web, but they focus on system disks. Actually, the procedure is extremely simple if you have only a data disk, but you have to use non-Microsoft tools.

I used Ubuntu 22.04, as a VMWare virtual machine on a Windows 10 host. It has the ability to connect an USB device to the virtual machine, but I had to replace the line

usb.restrictions.defaultAllow = "TRUE"

with

usb.restrictions.defaultAllow = "TRUE"

in the .vmx file.

Afterwards, I connected the USB drive to the VM. It appeared as /dev/sdc. The procedure is then as simple as:

# umount /dev/sdc1 (to remove the mount for the existing partition)
# gdisk /dev/sdc

This automatically converts the partition to GPT, you have only to write it to the disk with [w] and confirm.

Just to read again the disk in Linux, simply do the following:

# partprobe

In Ubuntu, this will also make the partition to mount again (and you can check everything is there).

That’s all! In my case, I then went back to windows and enlarged the 2 TB partition with the Disk Administration tool.


LibreOffice dialog speed

Short version: do not use a default network printer.

LibreOffice is very slow opening UI dialogs, such as “Format”. I am using version 7.4.1.2 (x64) on Windows 7, but it was the same with many previous versions.

This is also reported by many others on the Internet. Someone suggested that clearing the user directory solves the issue, but this didn’t happen in my case.

Someone suggested that the issue is linked to the network being available or not, so I made some tests disabling the network on my PC.

Before doing anything, opening the “Format” dialog takes about 10 seconds.

After disabling the network, opening the “Format” dialog becomes very slow, taking 40 seconds. However, the second time I open the “Format” dialog it becomes very fast. Something has happened.

If I re-enable the network, the “Format” dialog opens again in about 10 seconds.

Somebody else on the Internet suggested that this is linked to the default printer being on the network, even if this was for an Ubuntu system. However this is exactly my configuration. I set the default printer to the local PDF writer, and the “Format” dialog becames very quick.

P.S. This seems to be a known issue, even if this bug is marked as “hangs opening some files” we have the comment: “Sometimes (not always) that action can freeze for 30 seconds before the Format cells dialog comes up. Again this won’t happen if my network printer is ONLINE only when it is OFFLINE.” The bug is almost 11 years old.

https://bugs.documentfoundation.org/show_bug.cgi?id=42673


Encoding URL query parameters in Javascript + Django

When passing arbitrary strings as parameters for some function, you may be tempted to do it using a path component, which can be decoded easily with Django.

Do not do it. Forward slashes will be encoded in arbitrary ways (and you should note that the “string” path component in Django excludes forward slashes). Use query parameters instead, as follows.

In Javascript:

searchParams = new URLSearchParams({'parameter': 'value'});
URLparams = searchParams.toString();
// use this string to build your URL
URL = "http://<server>/endpoint" + URLparams;

In Django:

# url
    path('endpoint', views.endpoints.endpoint_function),
# view function in endpoint
def endpoint_function(request):
    parameter = request.query_params.get('parameter', DEFAULT_VALUE)

This will ensure correct encoding and decoding of parameters.

 


Guide to identifying C2000 processors

In case you have code running on a Texas Instruments C2000 processor, and you have to identify the specific processor in the code itself.

1. Read register 0x0882. This register is PARTID in F280x and C280x processors, and CLASSID in other processors.

0x002C – part is F2801
0x0024 – part is F2802
0x0034 – part is F2806
0x003C – part is F2808
0x00FE – part is F2809
0x0014 – part is F28016
0x001C – part is F28015
0xFF2C – part is C2801
0xFF24 – part is C2802
0x00C7 or 0x00CF – part is F2802x, go to 2.
0x00AB – part is TMS320F28030 – go to 3. for more details
0x00AF – part is TMS320F28031 – go to 3. for more details
0x00B3 – part is TMS320F28032 – go to 3. for more details
0x00B7 – part is TMS320F28033 – go to 3. for more details
0x00BB – part is TMS320F28034 – go to 3. for more details
0x00BF – part is TMS320F28035 – go to 3. for more details

2. Read register 0x3D7FFF. This register is PARTID in F2802x and F2802xx processors.
0x00C1 – part is TMS320F280200PT
0x00C0 – part is TMS320F280200DA
0x00CF – part is TMS320F28027PT
0x00CE – part is TMS320F28027DA
0x00C7 – part is TMS320F28026PT
0x00C6 – part is TMS320F28026DA
0x00CD – part is TMS320F28023PT
0x00CC – part is TMS320F28023DA
0x00C5 – part is TMS320F28022PT
0x00C4 – part is TMS320F28022DA
0x00CB – part is TMS320F28021PT
0x00CA – part is TMS320F28021DA
0x00C3 – part is TMS320F28020PT
0x00C2 – part is TMS320F28020DA

3. Read register 0x3D7E80. This register is PARTID in F2803x.
0x00BF – part is TMS320F28035PN
0x00BE – part is TMS320F28035PAG
0x00BD – part is TMS320F28035RSH
0x00BB – part is TMS320F28034PN
0x00BA – part is TMS320F28034PAG
0x00B9 – part is TMS320F28034RSH
0x00B7 – part is TMS320F28033PN
0x00B6 – part is TMS320F28033PAG
0x00B5 – part is TMS320F28033RSH
0x00B3 – part is TMS320F28032PN
0x00B2 – part is TMS320F28032PAG
0x00B1 – part is TMS320F28032RSH
0x00AF – part is TMS320F28031PN
0x00AE – part is TMS320F28031PAG
0x00AD – part is TMS320F28031RSH
0x00AB – part is TMS320F28030PN
0x00AA – part is TMS320F28030PAG
0x00A9 – part is TMS320F28030RSH