Merge branch 'dev-parouby' into dev

README.md

@@ -2,7 +2,9 @@
 
 AcouSed for **Acou**stic Backscattering for Concentration of Suspended **Sed**iments in Rivers is a software developped by INRAE, in collaboation with CNR.
 
-![](icons/Logo-INRAE.jpg)
+<p>
+  <img src="logos/AcouSed.png" align="center" width=20% height=20% >
+</p>
 
 It is divided in six tabs:
 - Acoustic data : acoustic raw data are downloaded and visualised
@@ -13,14 +15,39 @@ It is divided in six tabs:
 
 ## Installation
 
+### Standalone software
+
+AcouSed can be launched with python installation. An executable is available on [River Hydraulics](https://riverhydraulics.riverly.inrae.fr/outils/logiciels-pour-la-mesure/acoused) teams website. 
+The user needs to download the folder "acoused-packaging" including :
+- icons and logos folder
+- _internal folder (python packages)
+- executable file
+- calibration constant file
+- documentation
+
+Acoused.exe file must be launched from this folder. 
+Test data can be dowloaded from the [INRAE nextcloud](https://nextcloud.inrae.fr/s/3zZdieztrx7nwYa)
+
+### Python environment
+
 Acoused is developped for Linux and Windows on Python version 3.8 or
 greater. By default, Acoused is developped with Pypi package
 dependencies, but is also possible to use Guix package manager to run
 Acoused.
 
-### **TODO** Windows
+#### Windows
 
-### Linux
+You can use Pypi to get correct software environment and run the
+program.
+
+```shell
+python -m venv env
+env\Scripts\activate.bat
+python -m pip install -U -r ..\virtualenv\requirements.txt
+python main.py
+```
+
+#### Linux
 
 You can use Pypi to get correct software environment and run the
 program.
@@ -32,7 +59,7 @@ program.
   python3 main.py
 ```
 
-### Linux with Guix
+#### Linux with Guix
 
 To run Acoused within a [GNU Guix](https://guix.gnu.org/) software
 environment, you need Guix installed on your computer and run the
@@ -62,15 +89,44 @@ script `guix.sh` to run the program.
 
 If you have any questions or suggestions, please contact us to celine.berni@inrae.fr and/or jerome.lecoz@inrae.fr.
 
-## Acknowledgment (Funding)
+## Acknowledgment
+
+This study was conducted within the [Rhône Sediment Observatory](https://observatoire-sediments-rhone.fr/) (OSR), a multi-partner research program funded through the Plan Rhône by the European Regional Development Fund (ERDF), Agence de l’Eau RMC, CNR, EDF and three regional councils (Auvergne-Rhône-Alpes, PACA and Occitanie). 
+
+<p>
+  <img src="logos/OSR.png" align="center" width=10% height=10% >
+</p>
+
+## Industrial partners
+
+[CNR](https://www.cnr.tm.fr/)
+
+<p>
+  <img src="logos/CNR.png" align="center" width=10% height=10% >
+</p>
+
+[UBERTONE](https://ubertone.com/)
+
+<p>
+  <img src="logos/Ubertone.jpeg" align="center" width=10% height=10% >
+</p>
+
+
+[EDF](https://www.edf.fr/hydraulique-isere-drome)
+<p>
+  <img src="logos/EDF.png" align="center" width=10% height=10% >
+</p>
 
-This study was conducted within the [Rhône Sediment Observatory](https://observatoire-sediments-rhone.fr/) (OSR), a multi-partner research program funded through the Plan Rhône by the European Regional Development Fund (ERDF), Agence de l’Eau RMC, CNR, EDF and three regional councils (Auvergne-Rhône-Alpes, PACA and Occitanie). It was also support by CNR.
 
 ## License
 
 AcouSed
 Copyright (C) 2024-2025 - INRAE
 
+<p>
+  <img src="logos/BlocMarque-INRAE-Inter.jpg" align="center" width=10% height=10% >
+</p>
+
 This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
 
 This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.

View/sediment_calibration_tab.py

@@ -1569,6 +1569,10 @@ class SedimentCalibrationTab(QWidget):
             self._data_validity_message_box()
             return
 
+        if len(stg.fine_sample_profile) == 0:
+            self._data_validity_message_box()
+            return
+
         data_choice = self.combobox_acoustic_data_choice\
                           .currentIndex()
 
@@ -1659,8 +1663,8 @@ class SedimentCalibrationTab(QWidget):
             .scaledToHeight(32, Qt.SmoothTransformation)
         )
         msgBox.setText(
-            "Please select and valid the sample data with"
+            "Please select and valid the find and sand"
-            + "click on 'sample plot' button"
+            + "sample data with click on 'sample plot' button"
         )
         msgBox.setStandardButtons(QMessageBox.Ok)
         msgBox.exec()

View/signal_processing_tab.py

@@ -475,6 +475,7 @@ class SignalProcessingTab(QWidget):
 
         self.lineEdit_horizontal_average.returnPressed.connect(self.update_label_cells_sec)
         self.pushbutton_average.clicked.connect(self.compute_averaged_BS_data)
+        self.pushbutton_average.clicked.connect(self.replot)
 
         self.pushbutton_slider_left_to_begin.clicked.connect(self.slide_profile_number_to_begin)
 
@@ -561,8 +562,7 @@ class SignalProcessingTab(QWidget):
             self.combobox_freq_noise_from_profile_tail.blockSignals(False)
             self.combobox_acoustic_data_choice.blockSignals(False)
 
-    @trace
+    def recompute(self):
-    def recompute(self, *args):
         data_id = self.combobox_acoustic_data_choice.currentIndex()
 
         self.compute_average_profile_tail()
@@ -573,10 +573,10 @@ class SignalProcessingTab(QWidget):
         elif stg.noise_method[data_id] == 1:
             self.compute_noise_from_profile_tail_value()
 
+        self.remove_point_with_snr_filter()
         self.compute_averaged_BS_data()
 
-    @trace
+    def replot(self):
-    def replot(self, *args):
         self.plot_averaged_profile_tail()
         self.plot_transect_with_SNR_data()
         self.plot_pre_processed_BS_signal()
@@ -646,8 +646,7 @@ class SignalProcessingTab(QWidget):
         self.animation_groupbox_option_profile_tail.start()
 
     # ------------------------------------------------------
-    @trace
+    def compute_average_profile_tail(self):
-    def compute_average_profile_tail(self, *args):
         data_id = self.combobox_acoustic_data_choice.currentIndex()
         freq_noise_id = self.combobox_freq_noise_from_profile_tail.currentIndex()
 
@@ -712,8 +711,7 @@ class SignalProcessingTab(QWidget):
             )
         )
 
-    @trace
+    def plot_averaged_profile_tail(self):
-    def plot_averaged_profile_tail(self, *args):
 
         # --- Plot averaged signal ---
 
@@ -998,8 +996,7 @@ class SignalProcessingTab(QWidget):
         pnw = PlotNoiseWindow()
         pnw.exec()
 
-    @trace
+    def compute_noise_from_profile_tail_value(self):
-    def compute_noise_from_profile_tail_value(self, *args):
 
         stg.noise_method[self.combobox_acoustic_data_choice.currentIndex()] = 1
 
@@ -1080,8 +1077,7 @@ class SignalProcessingTab(QWidget):
 
         # self.activate_list_of_pre_processed_data()
 
-    @trace
+    def plot_noise(self):
-    def plot_noise(self, *args):
         self.horizontalLayout_groupbox_plot_noise_data.removeWidget(self.canvas_noise)
 
         self.fig_noise, self.axis_noise = plt.subplots(nrows=1, ncols=1, layout="constrained")
@@ -1112,8 +1108,7 @@ class SignalProcessingTab(QWidget):
 
         self.axis_noise.tick_params(axis='both', which='minor', labelsize=10)
 
-    @trace
+    def plot_transect_with_SNR_data(self):
-    def plot_transect_with_SNR_data(self, *args):
         # --- Condition if table is not filled ---
         # if not self.lineEdit_noise_file.text():
         if len(stg.BS_noise_raw_data) == 0:
@@ -1305,6 +1300,8 @@ class SignalProcessingTab(QWidget):
                 self.verticalLayout_groupbox_plot_SNR.addWidget(self.scroll_SNR)
 
     def remove_point_with_snr_filter(self):
+        if self.lineEdit_SNR_criterion.text() == '':
+            return
 
         if len(stg.filename_BS_raw_data) == 0:
             msgBox = QMessageBox()
@@ -1369,8 +1366,7 @@ class SignalProcessingTab(QWidget):
 
             self.compute_averaged_BS_data()
 
-    @trace
+    def plot_pre_processed_BS_signal(self):
-    def plot_pre_processed_BS_signal(self, *args):
         data_id = self.combobox_acoustic_data_choice.currentIndex()
 
         self.lineEdit_horizontal_average.setText(
@@ -1400,7 +1396,6 @@ class SignalProcessingTab(QWidget):
                 .addWidget(self.scroll_BS)
 
             if stg.time_cross_section[data_id].shape != (0,):
-                logger.info("time_cross_section")
                 if stg.depth_cross_section[data_id].shape != (0,):
                     x_time = stg.time_cross_section[data_id]
                     y_depth = stg.depth_cross_section[data_id]
@@ -1408,7 +1403,6 @@ class SignalProcessingTab(QWidget):
                     x_time = stg.time_cross_section[data_id]
                     y_depth = stg.depth[data_id]
             else:
-                logger.info("time")
                 if stg.depth_cross_section[data_id].shape != (0,):
                     x_time = stg.time[data_id]
                     y_depth = stg.depth_cross_section[data_id]
@@ -1416,31 +1410,31 @@ class SignalProcessingTab(QWidget):
                     x_time = stg.time[data_id]
                     y_depth = stg.depth[data_id]
 
-            for f, _ in enumerate(stg.freq[data_id]):
+            logger.debug(f"x_time: {x_time[data_id].shape}")
-                bed = False
+            logger.debug(f"y_depth: {y_depth[data_id].shape}")
 
+            for f, _ in enumerate(stg.freq[data_id]):
                 if stg.BS_stream_bed_pre_process_average[data_id].shape != (0,):
                     BS_data = stg.BS_stream_bed_pre_process_average
-                    bed = True
                 elif stg.BS_cross_section_pre_process_average[data_id].shape != (0,):
                     BS_data = stg.BS_cross_section_pre_process_average
                 elif stg.BS_raw_data_pre_process_average[data_id].shape != (0,):
                     BS_data = stg.BS_raw_data_pre_process_average
                 elif stg.BS_stream_bed_pre_process_SNR[data_id].shape != (0,):
                     BS_data = stg.BS_stream_bed_pre_process_SNR
-                    bed = True
                 elif stg.BS_cross_section_pre_process_SNR[data_id].shape != (0,):
                     BS_data = stg.BS_cross_section_pre_process_SNR
                 elif stg.BS_raw_data_pre_process_SNR[data_id].shape != (0,):
                     BS_data = stg.BS_raw_data_pre_process_SNR
                 elif stg.BS_stream_bed[data_id].shape != (0,):
                     BS_data = stg.BS_stream_bed
-                    bed = True
                 elif stg.BS_cross_section[data_id].shape != (0,):
                     BS_data = stg.BS_cross_section
                 elif stg.BS_raw_data[data_id].shape != (0,):
                     BS_data = stg.BS_raw_data
 
+                logger.debug(f"BS_data: {BS_data[data_id].shape}")
+
                 val_min = np.nanmin(
                     BS_data[data_id][f, :, :]
                 )
@@ -1457,7 +1451,7 @@ class SignalProcessingTab(QWidget):
                     cmap='viridis', norm=LogNorm(vmin=val_min, vmax=val_max)
                 )
 
-                if bed:
+                if stg.depth_bottom[data_id].shape != (0,):
                     self.axis_BS[f].plot(
                         x_time[f, :], -stg.depth_bottom[data_id],
                         color='black', linewidth=1, linestyle="solid"
@@ -1516,8 +1510,7 @@ class SignalProcessingTab(QWidget):
                 (1 / stg.nb_profiles_per_sec[self.combobox_acoustic_data_choice.currentIndex()][0])) +
             " sec")
 
-    @trace
+    def compute_averaged_BS_data(self):
-    def compute_averaged_BS_data(self, *args):
         if len(stg.filename_BS_raw_data) == 0:
             msgBox = QMessageBox()
             msgBox.setWindowTitle("Compute noise from profile tail error")
@@ -1525,7 +1518,6 @@ class SignalProcessingTab(QWidget):
             msgBox.setText("Download acoustic data in previous tab before applying SNR filter")
             msgBox.setStandardButtons(QMessageBox.Ok)
             msgBox.exec()
-
         elif len(stg.BS_noise_raw_data) == 0:
             msgBox = QMessageBox()
             msgBox.setWindowTitle("SNR filter Error")
@@ -1533,108 +1525,96 @@ class SignalProcessingTab(QWidget):
             msgBox.setText("Define noise data (file or profile tail) before using SNR filter")
             msgBox.setStandardButtons(QMessageBox.Ok)
             msgBox.exec()
-
         else:
             data_id = self.combobox_acoustic_data_choice.currentIndex()
-            kernel_avg = np.ones(2 * int(float(self.lineEdit_horizontal_average.text().replace(",", "."))) + 1)
+            kernel_avg = np.ones(
-            print(kernel_avg)
+                2 * int(
+                    float(
+                        self.lineEdit_horizontal_average\
+                            .text()\
+                            .replace(",", ".")
+                    )
+                ) + 1
+            )
+            logger.debug(f"kernel_avg: {kernel_avg}")
 
             stg.Nb_cells_to_average_BS_signal[data_id] = (
-                float(self.lineEdit_horizontal_average.text().replace(",", ".")))
+                float(
+                    self.lineEdit_horizontal_average\
+                        .text().replace(",", ".")
+                )
+            )
 
             if stg.time_cross_section[data_id].shape != (0,):
-
                 if stg.depth_cross_section[data_id].shape != (0,):
-
                     x_time = stg.time_cross_section[data_id]
                     y_depth = stg.depth_cross_section[data_id]
-
                 elif stg.depth[data_id].shape != (0,):
-
                     x_time = stg.time_cross_section[data_id]
                     y_depth = stg.depth[data_id]
-
             else:
-
                 if stg.depth_cross_section[data_id].shape != (0,):
-
                     x_time = stg.time[data_id]
                     y_depth = stg.depth_cross_section[data_id]
-
                 elif stg.depth[data_id].shape != (0,):
-
                     x_time = stg.time[data_id]
                     y_depth = stg.depth[data_id]
 
-            if stg.BS_stream_bed_pre_process_SNR[data_id].shape != (0,):
+            BS = [
+                stg.BS_stream_bed_pre_process_SNR,
+                stg.BS_cross_section_pre_process_SNR,
+                stg.BS_raw_data_pre_process_SNR,
+                stg.BS_stream_bed,
+                stg.BS_cross_section,
+                stg.BS_raw_data,
+            ]
 
-                stg.BS_stream_bed_pre_process_average[data_id] = (deepcopy(
+            BS_ppa = [
-                    stg.BS_stream_bed_pre_process_SNR[data_id]))
+                stg.BS_stream_bed_pre_process_average,
+                stg.BS_cross_section_pre_process_average,
+                stg.BS_raw_data_pre_process_average,
+                stg.BS_stream_bed_pre_process_average,
+                stg.BS_cross_section_pre_process_average,
+                stg.BS_raw_data_pre_process_average,
+            ]
+
+            time_shape, = x_time[data_id].shape
+            depth_shape, = y_depth[data_id].shape
+            logger.debug(f"time_shape: {time_shape}")
+            logger.debug(f"depth_shape: {depth_shape}")
+
+            BS_data = stg.BS_raw_data
+            BS_data_ppa = stg.BS_raw_data_pre_process_average
+            for i in range(len(BS)):
+                bs = BS[i]
+                logger.debug(f"BS data shape {bs[data_id].shape}")
+                if bs[data_id].shape == (0,):
+                    continue
+
+                x, y, z = bs[data_id].shape
+                if y == depth_shape and z == time_shape:
+                    BS_data = bs
+                    BS_data_ppa = BS_ppa[i]
+                    break
+
+            logger.debug(f"BS_data: {BS_data[data_id].shape}")
+
+            BS_data_ppa[data_id] = deepcopy(BS_data[data_id])
 
             for f, _ in enumerate(stg.freq[data_id]):
                 for i in range(y_depth.shape[1]):
+                    BS_data_ppa[data_id][f, i, :] = (
+                        convolve(
+                            BS_data[data_id][f, i, :],
+                            kernel_avg
+                        )
+                    )
 
-                        stg.BS_stream_bed_pre_process_average[data_id][f, i, :] = (
+            logger.debug(
-                            convolve(stg.BS_stream_bed_pre_process_SNR[data_id][f, i, :],
+                f"BS_data_ppa: {BS_data_ppa[data_id].shape}"
-                                     kernel_avg))
+            )
 
-            elif stg.BS_cross_section_pre_process_SNR[data_id].shape != (0,):
+    def plot_pre_processed_profile(self):
-
-                stg.BS_cross_section_pre_process_average[data_id] = (deepcopy(
-                    stg.BS_cross_section_pre_process_SNR[data_id]))
-
-                for f, _ in enumerate(stg.freq[data_id]):
-                    for i in range(y_depth.shape[1]):
-
-                        stg.BS_cross_section_pre_process_average[data_id][f, i, :] = (
-                            convolve(stg.BS_cross_section_pre_process_SNR[data_id][f, i, :],
-                                     kernel_avg))
-
-            elif stg.BS_raw_data_pre_process_SNR[data_id].shape != (0,):
-
-                stg.BS_raw_data_pre_process_average[data_id] = (deepcopy(
-                    stg.BS_raw_data_pre_process_SNR[data_id]))
-
-                for f, _ in enumerate(stg.freq[data_id]):
-                    for i in range(y_depth.shape[1]):
-
-                        stg.BS_raw_data_pre_process_average[data_id][f, i, :] = (
-                            convolve(stg.BS_raw_data_pre_process_SNR[data_id][f, i, :],
-                                     kernel_avg))
-
-            elif stg.BS_stream_bed[data_id].shape != (0,):
-
-                stg.BS_stream_bed_pre_process_average[data_id] = (deepcopy(
-                    stg.BS_stream_bed[data_id]))
-
-                for f, _ in enumerate(stg.freq[data_id]):
-                    for i in range(y_depth.shape[1]):
-                        stg.BS_stream_bed_pre_process_average[data_id][f, i, :] = (
-                            convolve(stg.BS_stream_bed[data_id][f, i, :], kernel_avg))
-
-            elif stg.BS_cross_section[data_id].shape != (0,):
-
-                stg.BS_cross_section_pre_process_average[data_id] = (deepcopy(
-                    stg.BS_cross_section[data_id]))
-
-                for f, _ in enumerate(stg.freq[data_id]):
-                    for i in range(y_depth.shape[1]):
-                        stg.BS_cross_section_pre_process_average[data_id][f, i, :] = (
-                            convolve(stg.BS_cross_section[data_id][f, i, :],
-                                     kernel_avg))
-
-            elif stg.BS_raw_data[data_id].shape != (0,):
-
-                stg.BS_raw_data_pre_process_average[data_id] = (deepcopy(
-                    stg.BS_raw_data[data_id]))
-
-                for f, _ in enumerate(stg.freq[data_id]):
-                    for i in range(y_depth.shape[1]):
-                        stg.BS_raw_data_pre_process_average[data_id][f, i, :] = (
-                            convolve(stg.BS_raw_data[data_id][f, i, :], kernel_avg))
-
-    @trace
-    def plot_pre_processed_profile(self, *args):
         data_id = self.combobox_acoustic_data_choice.currentIndex()
 
         if ((data_id != -1) and

packages/debug.bat

@@ -0,0 +1,5 @@
+
+@ECHO OFF
+
+start cmd /c test3\Acoused.exe
+

packages/windows.bat

@@ -0,0 +1,32 @@
+
+@ECHO OFF
+
+rem Python environment (-U = update python packages / -r = texte file)
+python -m pip install -U -r ..\virtualenv\requirements.txt
+
+rem Build windows version
+mkdir acoused_packaging
+pyinstaller --name "acoused" ..\main.py -y
+
+rem Icons
+mkdir acoused_packaging\icons
+copy /y ..\icons\*.png acoused_packaging\icons
+
+rem Logos
+mkdir acoused_packaging\logos
+copy /y ..\logos\* acoused_packaging\logos
+
+rem Doc
+copy /y ..\ABS_calibration_constant_kt.xlsx acoused_packaging
+copy /y ..\AcouSed_UserManual.pdf acoused_packaging
+copy /y ..\Acoustic_Inversion_theory.pdf acoused_packaging
+copy /y ..\Tutorial_AQUAscat_software.pdf acoused_packaging
+
+rem move exe
+move /y dist\AcouSed\acoused.exe acoused_packaging
+move /y dist\acoused\_internal acoused_packaging
+copy debug.bat acoused_packaging
+rmdir /s /q build
+rmdir /s /q dist
+del /q AcouSed.spec
+

requirements.txt

@@ -1,13 +0,0 @@
-matplotlib==3.6.3
-numpy==1.23.5
-pandas==1.5.3
-PyQt5==5.15.9
-PyQt5-Qt5==5.15.2
-PyQt5-sip==12.11.0
-python-dateutil==2.8.2
-scikit-learn==1.2.1
-scipy==1.10.0
-pyqt-file-list-widget==0.0.1
-qtrangeslider==0.1.5
-astropy==6.1.7
-odfpy==1.4.1

virtual_env/requirements.txt

@@ -1,33 +0,0 @@
-contourpy==1.0.7
-cycler==0.11.0
-defusedxml==0.7.1
-et-xmlfile==1.1.0
-fonttools==4.38.0
-importlib-resources==5.12.0
-joblib==1.2.0
-kiwisolver==1.4.4
-llvmlite==0.39.1
-matplotlib==3.6.3
-numba==0.56.4
-numpy==1.23.5
-odfpy==1.4.1
-openpyxl==3.0.10
-packaging==23.0
-pandas==1.5.3
-Pillow==9.4.0
-profilehooks==1.12.0
-pyparsing==3.0.9
-pyqt-checkbox-table-widget==0.0.14
-PyQt5==5.15.9
-PyQt5-Qt5==5.15.2
-PyQt5-sip==12.11.0
-python-dateutil==2.8.2
-pytz==2022.7.1
-scikit-learn==1.2.1
-scipy==1.10.0
-six==1.16.0
-threadpoolctl==3.1.0
-utm==0.7.0
-xlrd==2.0.1
-xmltodict==0.13.0
-zipp==3.15.0