Compare commits

...

118 Commits

Author SHA1 Message Date
brahim 60b367b7ef correction fusion 2025-03-20 13:23:15 +01:00
brahim ac1ffcef58 windows comand to zip the file is added 2025-03-20 13:20:23 +01:00
Pierre-Antoine dea9042ed1 Merge branch 'dev-parouby' into dev 2025-03-20 11:56:28 +01:00
Pierre-Antoine b52acafd29 Sediment calibration: Add test on fine data selection for interpolate. 2025-03-20 11:55:26 +01:00
brahim e87bb9731d fusion repared 2025-03-20 11:06:08 +01:00
brahim d076054375 fusion finished 2025-03-20 11:01:14 +01:00
Pierre-Antoine 998e1e7f8f Signal processing: Fix #8 and clean debug display code. 2025-03-20 10:33:16 +01:00
brahim 1e9116bd0d File requirements.txt and folder vritual_env are deleted 2025-03-20 10:06:00 +01:00
Pierre-Antoine be7627d93f Signal processing: Refactoring with recompute and replot method. 2025-03-19 18:08:33 +01:00
brahim 1fffdc714e scripts to generate acoused release 2025-03-19 16:14:25 +01:00
Pierre-Antoine 225c40c6c1 Signal processing: Some minor refactoring. 2025-03-19 16:13:15 +01:00
MOUDJED Brahim fbc46f3344 Update README.md 2025-03-19 15:59:21 +01:00
MOUDJED Brahim 43d586bf05 Update README.md 2025-03-19 15:58:55 +01:00
MOUDJED Brahim 089a7657ab Update README.md 2025-03-19 15:57:59 +01:00
MOUDJED Brahim 0eb10fea10 Update README.md 2025-03-19 15:55:03 +01:00
MOUDJED Brahim d3afd2900e Update README.md 2025-03-19 15:51:59 +01:00
MOUDJED Brahim f1b196dba2 Update README.md 2025-03-19 15:50:41 +01:00
MOUDJED Brahim a877cf86d7 Update README.md 2025-03-19 15:50:12 +01:00
MOUDJED Brahim 9e7f34238e Update README.md 2025-03-19 15:49:25 +01:00
MOUDJED Brahim 887a152fff Update README.md 2025-03-19 15:47:37 +01:00
Pierre-Antoine 8a39bba7b1 Signal processing: Some refactoring. 2025-03-19 15:31:06 +01:00
Pierre-Antoine ad865e2829 tools: Add new file. 2025-03-19 15:30:56 +01:00
Pierre-Antoine 99ff7c5fed Acoused: Use a specific logger. 2025-03-19 14:12:35 +01:00
Pierre-Antoine 64230540c7 Signal processing: Change plot font. 2025-03-19 13:44:53 +01:00
Pierre-Antoine 44bf348ee5 Signal processing: Block signals during tab update. 2025-03-19 13:43:37 +01:00
Pierre-Antoine c521738567 Acoutic data: Minor change. 2025-03-17 18:06:09 +01:00
Pierre-Antoine b5338366cf Merge branch 'dev-parouby' into dev 2025-03-17 17:34:25 +01:00
Pierre-Antoine 752697db86 gitignore: Add '.exe' files. 2025-03-17 17:26:50 +01:00
brahim b99e950a8e If values of sediment attenuation parameters are infinite, a warning is added and inversion computatio is not allowed #41 2025-03-17 17:25:55 +01:00
Pierre-Antoine b9669d9cbf Acoustic data: Refactorise 'remove_file_from_ListWidget' method. 2025-03-17 17:25:19 +01:00
Pierre-Antoine b9ca307c59 Acoustic data: Remove useless 'eval' or 'exec'. 2025-03-17 17:18:02 +01:00
Pierre-Antoine f0150443e3 Acoustic data: Clear 'clear_files_from_ListWidget' method. 2025-03-17 17:14:00 +01:00
Pierre-Antoine 4c024cbb42 Acoustic data: Disable unused distance from bank (#39). 2025-03-17 16:35:32 +01:00
Pierre-Antoine 496937bde2 Acoustic data: Disable unused groupbox gps. 2025-03-17 16:26:50 +01:00
Pierre-Antoine 11ced0a263 Mainwindow: Minor change. 2025-03-17 10:44:49 +01:00
Pierre-Antoine 5cf87a5e7b Mainwindow: Fix table export. 2025-03-17 10:41:38 +01:00
brahim 477ad00cf8 hyperlink of git deposit is added in about window 2025-03-14 17:22:19 +01:00
brahim 1c409531a1 logger added to print computed values of calibration parameters 2025-03-14 16:42:23 +01:00
brahim 5734f6e614 Clear button works when only one acoustic file is downloaded and crash when more than one acoustic file is downloaded #34 2025-03-14 16:33:08 +01:00
brahim 22cd451b24 Error message is added when user click on import/compute calibration without acoustic data #37 2025-03-14 11:52:27 +01:00
brahim 22be597d75 Correction of Delete acoustic data issue (#34) generates others issues in measurements information box. To be continued ... 2025-03-14 11:40:44 +01:00
brahim 671f66058c the clear button is partially corrected 2025-03-13 17:11:10 +01:00
brahim 2d6c950dfe the delete button is corrected 2025-03-13 17:01:13 +01:00
brahim fe9ac2d65c Some correction : Profile and display option wasn't filled when the user download an acoustic file 2025-03-13 16:03:51 +01:00
brahim bdeca2b44e Correction when the user clicks on the clear button (clear profile tail) #36 2025-03-13 15:41:04 +01:00
brahim 06056d4865 Correction of average filter button (if no acoustic data) #33 2025-03-13 15:22:49 +01:00
brahim df68a862fc Correction of SNR filter button (if no acoustic data) #33 2025-03-13 15:19:30 +01:00
brahim 7a5f6d41f7 Correction of the profile tail button (if no acoustic data) #33 2025-03-13 15:07:31 +01:00
brahim eef048b197 Correction of the red cross button (if no acoustic data) #33 2025-03-13 14:19:44 +01:00
brahim 3c78777179 Correction of reload button (if no acoustic data) #33 2025-03-13 13:34:57 +01:00
brahim b84f58ca2c Resolving confilct 2025-03-13 12:23:30 +01:00
brahim 0df253d70f Resolve merge conflict by incorporating both suggestions 2025-03-13 12:05:27 +01:00
brahim 2715d225af Crash corrected if any calibration file is selected (Click Cancel) 2025-03-13 11:38:21 +01:00
Pierre-Antoine 571ac20d37 Acoustic data: refactoring plot and fix #17. 2025-03-12 16:36:53 +01:00
Pierre-Antoine 5f0fb9ad53 Acoustic data: Refactoring. 2025-03-12 15:51:44 +01:00
Pierre-Antoine cb94bb3c5d Acoustic data: Refactoring. 2025-03-12 15:02:02 +01:00
Pierre-Antoine 56a26c04b6 Acoustic data: Minor refactoring. 2025-03-12 11:45:03 +01:00
Pierre-Antoine de54303e0b README.md: Minor change. 2025-03-12 11:17:16 +01:00
Pierre-Antoine 43ed93584f Mainwindow: Set translate button to disabled. 2025-03-12 11:05:30 +01:00
Pierre-Antoine d1b311d1a3 Merge branch 'dev-parouby' into dev 2025-03-12 10:51:32 +01:00
Pierre-Antoine 607aed3f11 Mainwindow: 'save' call 'save_as' when study are never saved before. 2025-03-12 10:49:04 +01:00
Pierre-Antoine 487f5e9ac8 Sediment calibration: Code refactoring and fix #14. 2025-03-12 10:17:38 +01:00
brahim 761dc9008c compute interpolation can be done without detect bottom 2025-03-11 17:32:00 +01:00
brahim 372d002277 resolve conflict 2025-03-11 17:10:56 +01:00
brahim b28978311c correction merge and correction interpolate fine profile 2025-03-11 16:44:14 +01:00
Pierre-Antoine 8c7c4cafbc Inversion: Excel export: Fix depth column shape and filename extention. 2025-03-11 16:15:06 +01:00
brahim baebe2e8cc interpolate Mifine profile is corrected with depth_bottom 2025-03-11 15:32:39 +01:00
Pierre-Antoine 8dd50f9373 Inversion: Fix #13. 2025-03-11 15:12:31 +01:00
Pierre-Antoine bcdae43d10 SQL: Remove unused logs. 2025-03-11 14:39:22 +01:00
Pierre-Antoine 0c141b8d24 SQL: Minor fix. 2025-03-11 14:28:49 +01:00
Pierre-Antoine ba0557294e Inversion: Refactoring 'save_result_in_excel_file' method. 2025-03-11 14:19:18 +01:00
brahim 73bf3cf9d3 The user can reload signal processing after acoustic data boundaries values modification without crash #8 2025-03-11 11:39:09 +01:00
Pierre-Antoine 12fea4e182 Merge branch 'dev-parouby' into dev 2025-03-11 11:37:22 +01:00
Pierre-Antoine a5268c6214 Sample data: Fix #29. 2025-03-11 11:35:52 +01:00
Pierre-Antoine 2697acddfe Sample data: Refactoring some functions. 2025-03-11 10:20:18 +01:00
Pierre-Antoine f872b625c2 SQL: Minor change. 2025-03-10 17:14:11 +01:00
Pierre-Antoine 87098ff152 SQL: Some refactoring. 2025-03-10 17:12:45 +01:00
Pierre-Antoine f5eb9a18c6 SQL: Fix empty/partiel study save #12. 2025-03-10 17:04:31 +01:00
Pierre-Antoine b7ae9dfe69 Sediment calibration: Some refactoring and fix #6. 2025-03-10 15:48:00 +01:00
Pierre-Antoine 0ce60afd04 Sediment calibration: Skip crash compute button without data. 2025-03-10 15:17:19 +01:00
Pierre-Antoine 86b9f85ec6 Sediment calibration: Skip somme crash and refactoring. 2025-03-10 15:14:09 +01:00
brahim 6de578a07e time noise and depth noise variable are used instead of time and depth to avoid size incompatibility #9 2025-03-10 15:12:49 +01:00
Pierre-Antoine 5eb78b812d Fix interpolate button action without correct data. 2025-03-10 14:43:36 +01:00
brahim 320971160d Plot of SNR field is corrected when the user set a profile tail level #4 2025-03-10 14:30:45 +01:00
Pierre-Antoine 8f7e679598 Sediment calibration: Refactoring method 'interpolate_Mfine_profile'. 2025-03-10 14:23:29 +01:00
Pierre-Antoine 6628d610de Sediment calibration: Refactoring __init__ and minor change. 2025-03-10 12:05:07 +01:00
brahim 190c0f0b80 The readme file is modified and will be completed by Brahim. 2025-03-10 11:38:47 +01:00
brahim 2ffb9a21e6 read table for open cleaned 2025-03-10 10:16:03 +01:00
MOUDJED Brahim 2ec5584860 Update README.md 2025-03-07 17:19:16 +01:00
MOUDJED Brahim 6d4fb5fae6 Update README.md 2025-03-07 12:14:46 +01:00
Pierre-Antoine 2b84fab38d README.md: Add some information and global file scheme #27. 2025-03-06 15:49:48 +01:00
Pierre-Antoine 17619f15ac Fix save method on existing file #16. 2025-03-06 14:33:30 +01:00
Pierre-Antoine b26d8a2906 Fix note tab click on 'edit settings' button #10. 2025-03-06 14:13:35 +01:00
Pierre-Antoine 52b612ba3e Fix acoustic data loading without selection #3. 2025-03-06 10:47:54 +01:00
brahim 65a9422f71 acoustic inversion method high concentration file is cleaned from useless commented lines and useless print 2025-03-06 10:23:27 +01:00
brahim cea9e35498 acoustic data loader file is cleaned from useless commented lines and useless print 2025-03-06 10:18:46 +01:00
Pierre-Antoine 5ebd842346 Fix open and save as method (#2 and #18). 2025-03-06 10:13:00 +01:00
Pierre-Antoine 4af3c4c9ea Fix #7 with exception handler. 2025-03-06 09:18:04 +01:00
Pierre-Antoine b80e0f3c8e Merge branch 'dev-parouby' into dev 2025-03-05 15:40:00 +01:00
Pierre-Antoine 1c3f6fb015 main: Add 'logging' basic config. 2025-03-05 14:53:29 +01:00
brahim 9ecb70b955 plot noise program is cleaned from useless commented lines and useless print 2025-03-05 14:28:00 +01:00
brahim 0879520fd2 mainwindow program is cleaned from useless commented lines and useless print 2025-03-05 14:24:00 +01:00
brahim 030213608d line from PyQt5.QtCore import Qt was removed by mistake 2025-03-05 14:17:17 +01:00
brahim 329639e064 granulo loader program is cleaned from useless commented lines and useless print 2025-03-05 13:40:22 +01:00
brahim 5d16adc521 calibration constant program is cleaned from useless commented lines 2025-03-05 13:37:59 +01:00
brahim 9fcbe0e306 ubsediflow raw extract is cleaned from useless test 2025-03-05 13:36:37 +01:00
brahim 77f28462e6 ubsediflow raw extract is cleaned from useless print 2025-03-05 13:35:13 +01:00
brahim 2e9f9c3a3c acoustic data loader ubsediflow is cleaned from useless commented lines and useless print 2025-03-05 12:56:55 +01:00
brahim 80d570796a acoustic inversion tab is cleaned from useless commented lines and useless print 2025-03-05 12:48:45 +01:00
brahim 40d14a2ffb Sediment calibration tab is cleaned from useless commented lines and useless print 2025-03-05 12:02:59 +01:00
brahim d3234bff5d Sample data tab is cleaned from useless commented lines and useless print 2025-03-05 11:50:38 +01:00
brahim f2db89eef8 signal processing tab is cleaned from useless commented lines and useless print 2025-03-05 11:32:21 +01:00
Pierre-Antoine 3e5ef5d2bc Mainwindow: Revert 'sqlitebrowser' QProcess to Popen (run as async). 2025-03-05 11:22:06 +01:00
Pierre-Antoine 9c5b76e451 Mainwindow: Fix #11 pdf file open, use QDesktopServices instead of open cmd. 2025-03-05 09:40:18 +01:00
Pierre-Antoine 769fd8581e Mainwindow: Fix #7 sqlitebrowser call with QProcess. 2025-03-05 09:27:54 +01:00
brahim 153b136812 acoustic data tab is cleaned from useless commented lines and useless print 2025-03-04 15:38:20 +01:00
brahim 6b22562e57 astropy library is added 2025-03-04 14:39:53 +01:00
Pierre-Antoine dc9264cd65 git: Update project depencencies to fix #5. 2025-03-04 13:35:10 +01:00
33 changed files with 3638 additions and 10401 deletions

3
.gitignore vendored
View File

@ -1,6 +1,9 @@
TAGS
Error_file.txt
# Windows executable file
*.exe
# Created by https://www.toptal.com/developers/gitignore/api/python
# Edit at https://www.toptal.com/developers/gitignore?templates=python

View File

@ -721,14 +721,14 @@ class RawAquascatData:
elif pktType == 41:
chan = st.unpack("H", f.read(2))[0] + 1
# Increase the Index
AbsIndex[chan-1] = AbsIndex[chan-1] + 1
Tmp = [st.unpack(
"h", f.read(2))[0] for ui in np.arange(
1, 2 * AbsNumBins[chan-1]+1)]
sss = [x / 32768 for x in Tmp]
AbsData[:, int(
AbsIndex[chan-1]), int(chan-1)] = np.abs(np.array(sss[0::2])+complex(0,1)*np.array(sss[1::2]))
@ -1097,19 +1097,3 @@ class MeanAquascatProfile:
file.close()
# ------------------------- Test --------------------------------------#
start_time = time.time()
if __name__ == "__main__":
# path1 = r'C:\Users\vergne\Documents\Donnees_aquascat\2017_juillet - experience cuve sediments fins\2017_07_19 - mercredi eau claire\20170719114700.aqa'
path1 = r'//home/brahim.moudjed/Documents/3 Software_Project/river_inversion_project/Data/Aquascat data test/20171213135800.aqa'
data1 = RawAquascatData(path1)
# path2 = r'C:\Users\vergne\Documents\Donnees_aquascat\2017_juillet - experience cuve sediments fins\2017_07_19 - mercredi eau claire\20170719114700.aqa.txt'
path2 = r'//home/brahim.moudjed/Documents/3 Software_Project/river_inversion_project/Data/Aquascat data test/20171213135800.txt'
data2 = RawAquascatData(path2)
print(data1.PingRate)
print(data2.PingRate)
print("Computational time: %.2f min" %((time.time() - start_time)/60) )

View File

@ -5,35 +5,25 @@ import pandas as pd
import matplotlib.pyplot as plt
from matplotlib.colors import LogNorm
# path_BS_raw_data = "/home/bmoudjed/Documents/2 Data/Confluence_Rhône_Isere_2018/Acoustic_data/20180107123500.aqa"
# path_BS_raw_data = "/home/bmoudjed/Documents/3 SSC acoustic meas project/Graphical interface project/" \
# "Data/AcousticNoise_data/20180107121600.aqa"
class AcousticDataLoader:
def __init__(self, path_BS_raw_data: str):
self.path_BS_raw_data = path_BS_raw_data
print(self.path_BS_raw_data)
# --- Load Backscatter acoustic raw data with RawAquascatData class ---
self._data_BS = RawAquascatData(self.path_BS_raw_data)
print(self._data_BS.V.shape)
self._BS_raw_data = np.swapaxes(self._data_BS.V, 0, 1)
print(f"BS raw data shape = {self._BS_raw_data.shape}")
self._freq = self._data_BS.Freq
print(f"freq shape = {self._freq.shape}")
self._freq_text = self._data_BS.freqText
self._r = np.repeat(np.transpose(self._data_BS.r), self._freq.shape[0], axis=0)
print(f"r shape = {self._r.shape}")
self._time = np.repeat(
np.transpose(np.array([t / self._data_BS.PingRate for t in range(self._data_BS.NumProfiles)])[:, np.newaxis]),
self._freq.shape[0], axis=0)
print(f"time shape = {self._time.shape}")
self._date = self._data_BS.date.date()
self._hour = self._data_BS.date.time()
@ -48,97 +38,30 @@ class AcousticDataLoader:
self._gain_rx = self._data_BS.RxGain.tolist()
self._gain_tx = self._data_BS.TxGain.tolist()
# print((self._cell_size))
# print((self._nb_pings_averaged_per_profile))
# print(self._r[0, :][1] - self._r[1, :][0])
# print(type(self._nb_cells), self._nb_cells)
# self._snr = np.array([])
# self._snr_reshape = np.array([])
# self._time_snr = np.array([])
# print(type(self._gain_tx))
# print(["BS - " + f for f in self._freq_text])
# print(self._time.shape[0]*self._r.shape[0]*4)
# print(self._time[np.where(np.floor(self._time) == 175)])
# print(np.where((self._time) == 155)[0][0])
# fig, ax = plt.subplots(nrows=1, ncols=1)
# # ax.pcolormesh(self._time[0, :2200], -self._r[0, :], (self._BS_raw_data[0, :, :2200]),
# # cmap='viridis',
# # norm=LogNorm(vmin=1e-5, vmax=np.max(self._BS_raw_data[0, :, :2200]))) # , shading='gouraud')
# ax.pcolormesh(range(self._BS_raw_data.shape[2]), range(self._BS_raw_data.shape[1]), self._BS_raw_data[2, :, :], cmap='viridis',
# norm=LogNorm(vmin=1e-5, vmax=np.max(self._BS_raw_data[:, 0, :]))) # , shading='gouraud')
# ax.set_xticks([])
# ax.set_yticks([])
# plt.show()
# --- Plot vertical profile for bottom detection ---
# fig2, ax2 = plt.subplots(nrows=1, ncols=1, layout="constrained")
# ax2.plot(self._BS_raw_data[0, :, 1], -self._r[0], "k.-")
# plt.show()
# fig, ax = plt.subplots(nrows=1, ncols=1)
# ax.plot(self._BS_raw_data[:, 0, 100] , self._r)
# ax.set_ylim(2, 20)
# plt.show()
# print(self.reshape_BS_raw_cross_section()[0, 0])
# self.reshape_BS_raw_cross_section()
# self.reshape_r()
# self.reshape_t()
# self.compute_r_2D()
def reshape_BS_raw_data(self):
BS_raw_cross_section = np.reshape(self._BS_raw_data,
(self._r.shape[1] * self._time.shape[1], self._freq.shape[0]),
order="F")
print(BS_raw_cross_section.shape)
return BS_raw_cross_section
def reshape_r(self):
# r = np.reshape(np.repeat(self._r[0, :], self._time.shape[0], axis=1),
# self._r.shape[0]*self._time.shape[0],
# order="F")
r = np.zeros((self._r.shape[1] * self._time.shape[1], self._freq.shape[0]))
for i, _ in enumerate(self._freq):
for j in range(self._time.shape[1]):
r[j*self._r.shape[1]:(j+1)*self._r.shape[1], i] = self._r[i, :]
# r[:, i] = np.repeat(self._r[i, :], self._time.shape[1])
print(r.shape)
return r
def compute_r_2D(self):
r2D = np.zeros((self._freq.shape[0], self._r.shape[1], self._time.shape[1]))
for f, _ in enumerate(self._freq):
r2D[f, :, :] = np.repeat(np.transpose(self._r[f, :])[:, np.newaxis], self._time.shape[1], axis=1)
print(r2D.shape)
return r2D
def reshape_t(self):
# t = np.reshape(np.repeat(self._time, self._r.shape[0]), (self._time.shape[0]*self._r.shape[0], 1))
t = np.zeros((self._r.shape[1] * self._time.shape[1], self._freq.shape[0]))
for i, _ in enumerate(self._freq):
t[:, i] = np.repeat(self._time[i, :], self._r.shape[1])
print(t.shape)
return t
# def concatenate_data(self):
# self.reshape_t()
# self.reshape_BS_raw_cross_section()
# # print(self.reshape_t().shape)
# # print(se.lf.reshape_BS_raw_cross_section().shape)
# df = pd.DataFrame(np.concatenate((self.reshape_t(), self.reshape_BS_raw_cross_section()), axis=1),
# columns=["time"] + self._freq_text)
# return df
# if __name__ == "__main__":
# AcousticDataLoader(path_BS_raw_data)

View File

@ -1,49 +1,35 @@
# ============================================================================== #
# acoustic_data_loder_UBSediFlow.py - AcouSed #
# Copyright (C) 2024 INRAE #
# #
# This program is free software: you can redistribute it and/or modify #
# it under the terms of the GNU General Public License as published by #
# the Free Software Foundation, either version 3 of the License, or #
# (at your option) any later version. #
# #
# This program is distributed in the hope that it will be useful, #
# but WITHOUT ANY WARRANTY; without even the implied warranty of #
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
# GNU General Public License for more details. #
# #
# You should have received a copy of the GNU General Public License #
# along with this program. If not, see <https://www.gnu.org/licenses/>. #
# by Brahim MOUDJED #
# ============================================================================== #
# -*- coding: utf-8 -*-
import numpy as np
import pandas as pd
import datetime
import matplotlib.pyplot as plt
from matplotlib.colors import LogNorm, BoundaryNorm
from copy import deepcopy
from scipy.signal import savgol_filter
from Model.udt_extract.raw_extract import raw_extract
# raw_20210519_102332.udt raw_20210520_135452.udt raw_20210525_092759.udt raw_20210525_080454.udt
# path_BS_raw_data0 = ("/home/bmoudjed/Documents/3 SSC acoustic meas project/Graphical interface project/Data/APAVER_2021/Raw_data_udt/")
# filename0 = "raw_20210519_135400.udt"
# path_BS_raw_data0 = ("/home/bmoudjed/Documents/3 SSC acoustic meas project/Graphical interface project/Data/"
# "APAVER_2021/transect_ubsediflow/01-raw_20210519_115128/Raw_data_udt/")
# filename0 = "raw_20210519_115128.udt"
# path_BS_raw_data0 = ("/home/bmoudjed/Documents/3 SSC acoustic meas project/Graphical interface project/Data/"
# "APAVER_2021/transect_ubsediflow/02-bb0077eda128f3f7887052eb3e8b0884/Raw_data_udt/")
# filename0 = "raw_20210519_161400.udt"
# path_BS_raw_data0 = ("/home/bmoudjed/Documents/3 SSC acoustic meas project/Graphical interface project/Data/"
# "APAVER_2021/transect_ubsediflow/04-fb53d0e92c9c88e2a6cf45e0320fbc76/Raw_data_udt/")
# filename0 = "raw_20210520_133200.udt"
# ("/home/bmoudjed/Documents/3 SSC acoustic meas project/Graphical interface project/Data/APAVER_2021/"
# "Rhone_20210519/Rhone_20210519/record/")
# path_BS_raw_data0 = ("/home/bmoudjed/Documents/3 SSC acoustic meas project/Graphical interface project/Data/Raw_data_udt/")
# filename0 = "raw_20210519_130643.udt"
# path_BS_raw_data0 = ("/home/bmoudjed/Documents/2 Data/APAVER_2021/Raw_data_udt/")
# filename0 = "raw_20210520_085958.udt"
# filename = "raw_20210519_115128.udt"
# "raw_20210526_153310.udt"
class AcousticDataLoaderUBSediFlow:
def __init__(self, path_BS_raw_data: str):
# path_BS_raw_data = path_BS_raw_data0 + filename0
self.path_BS_raw_data = path_BS_raw_data
# --- Extract Backscatter acoustic raw data with class ---
@ -53,23 +39,10 @@ class AcousticDataLoaderUBSediFlow:
device_name, time_begin, time_end, param_us_dicts, data_us_dicts, data_dicts, settings_dict \
= raw_extract(self.path_BS_raw_data)
print(f"device_name : {device_name}")
print(f"date begin : {time_begin.date()}")
print(f"time begin : {time_begin.time()}")
print(f"settings_dict : {settings_dict}")
# # --- Date and Hour of measurements read on udt data file ---
# filename = self.path_BS_raw_data[-23:]
# date_and_time = datetime.datetime(year=int(filename[4:8]),
# month=int(filename[8:10]),
# day=int(filename[10:12]),
# hour=int(filename[13:15]),
# minute=int(filename[15:17]),
# second=int(filename[17:19]))
self._date = time_begin.date()
print(f"date : {self._date}")
self._hour = time_begin.time()
print(f"time : {self._hour}")
self._freq = np.array([[]])
@ -88,19 +61,11 @@ class AcousticDataLoaderUBSediFlow:
self._time = np.array([[]])
self._time_snr = np.array([[]])
self._BS_raw_data = np.array([[[]]])
# self._SNR_data = np.array([[[]]])
time_len = []
time_snr_len = []
for config in param_us_dicts.keys():
# print("-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x")
# print(f"config : {config} \n")
for channel in param_us_dicts[config].keys():
print("-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x")
# print(f"channel : {channel} \n")
# print("param_us_dicts[config][channel] ", param_us_dicts[config][channel])
# print("param_us_dicts ", param_us_dicts)
# print(data_us_dicts[config][channel]['echo_avg_profile'])
# --- Frequencies ---
self._freq = np.append(self._freq, param_us_dicts[config][channel]['f0'])
@ -115,31 +80,19 @@ class AcousticDataLoaderUBSediFlow:
self._gain_tx = np.append(self._gain_tx, param_us_dicts[config][channel]['a1'])
# --- Depth for each frequencies ---
print("r_dcell : ", param_us_dicts[config][channel]['r_dcell'])
print("n_cell : ", param_us_dicts[config][channel]['n_cell'])
depth = [param_us_dicts[config][channel]['r_dcell'] * i
for i in list(range(param_us_dicts[config][channel]['n_cell']))]
print(f"depth : {depth}")
print(f"lenght of depth : {len(depth)}")
if self._r.shape[1] == 0:
self._r = np.array([depth])
else:
if len(depth) == self._r.shape[1]:
print("Je suis là")
print(f"depth lenght : {len(depth)}")
print(f"r shape : {self._r.shape}")
self._r = np.append(self._r, np.array([depth]), axis=0)
print("C'est encore moi")
elif len(depth) < self._r.shape[1]:
print(f"depth lenght : {len(depth)}")
self._r = self._r[:, :len(depth)]
self._r = np.append(self._r, np.array([depth]), axis=0)
print(f"r shape : {self._r.shape}")
elif len(depth) > self._r.shape[1]:
print(f"depth lenght : {len(depth)}")
self._r = np.append(self._r, np.array([depth[:self._r.shape[1]]]), axis=0)
print(f"r shape : {self._r.shape}")
print(f"self._r : {self._r.shape}")
# --- BS Time for each frequencies ---
time = [[(t - data_us_dicts[config][channel]['echo_avg_profile']['time'][0]).total_seconds()
@ -147,69 +100,28 @@ class AcousticDataLoaderUBSediFlow:
time_len = np.append(time_len, len(time[0]))
if len(time_len) == 1:
print(f"1 time length : {len(time[0])}")
self._time = np.array(time)
print(f"self._time.shape {self._time.shape}")
elif self._time.shape[1] == len(time[0]):
print(f"2 time length : {len(time[0])}")
self._time = np.append(self._time, time, axis=0)
print(f"self._time.shape {self._time.shape}")
elif self._time.shape[1] > len(time[0]):
print(f"3 time length : {len(time[0])}")
# print(f"self._time.shape {self._time.shape}")
# print([int(np.min(time_len)) + int(i) - 1 for i in range(1, int(np.max(time_len))-int(np.min(time_len))+1)])
self._time = np.delete(self._time,
[int(np.min(time_len)) + int(i) - 1 for i in range(1, int(np.max(time_len))-int(np.min(time_len))+1)],
axis=1)
self._time = np.append(self._time, time, axis=0)
print(f"self._time.shape {self._time.shape}")
elif self._time.shape[1] < len(time[0]):
print(f"4 time length : {len(time[0])}")
time = time[:int(np.max(time_len)) - (int(np.max(time_len)) - int(np.min(time_len)))]
self._time = np.append(self._time, time, axis=0)
print(f"self._time.shape {self._time.shape}")
self._nb_profiles = np.append(self._nb_profiles, self._time.shape[1])
self._nb_profiles_per_sec = np.append(self._nb_profiles_per_sec,
param_us_dicts[config][channel]['n_avg'])
# --- SNR Time for each frequencies ---
# time_snr = [[(t - data_us_dicts[config][channel]['snr_doppler_avg_profile']['time'][0]).total_seconds()
# for t in data_us_dicts[config][channel]['snr_doppler_avg_profile']['time']]]
# time_snr_len = np.append(time_snr_len, len(time_snr[0]))
#
# if len(time_snr_len) == 1:
# # print(f"1 time length : {len(time[0])}")
# self._time_snr = np.array(time_snr)
# # print(f"self._time.shape {self._time.shape}")
# elif self._time_snr.shape[1] == len(time_snr[0]):
# # print(f"2 time length : {len(time[0])}")
# self._time_snr = np.append(self._time_snr, time_snr, axis=0)
# # print(f"self._time.shape {self._time.shape}")
# elif self._time_snr.shape[1] > len(time_snr[0]):
# # print(f"3 time length : {len(time[0])}")
# # print(f"self._time.shape {self._time.shape}")
# # print([int(np.min(time_len)) + int(i) - 1 for i in range(1, int(np.max(time_len))-int(np.min(time_len))+1)])
# self._time_snr = np.delete(self._time_snr,
# [int(np.min(time_snr_len)) + int(i) - 1 for i in
# range(1, int(np.max(time_snr_len)) - int(np.min(time_snr_len)) + 1)],
# axis=1)
# self._time_snr = np.append(self._time_snr, time_snr, axis=0)
# # print(f"self._time.shape {self._time.shape}")
# elif self._time_snr.shape[1] < len(time_snr[0]):
# # print(f"4 time length : {len(time[0])}")
# time_snr = time_snr[:int(np.max(time_snr_len)) - (int(np.max(time_snr_len)) - int(np.min(time_snr_len)))]
# self._time_snr = np.append(self._time_snr, time_snr, axis=0)
# # print(f"self._time.shape {self._time.shape}")
# --- US Backscatter raw signal ---
BS_data = np.array([[]])
if config == 1:
BS_data = np.array([data_us_dicts[config][channel]['echo_avg_profile']['data'][0]])
print(f"cas 1 : BS_raw_data shape = {self._BS_raw_data.shape}")
print(f"cas 1 : BS_data shape = {BS_data.shape}")
for i in range(self._time.shape[1]):
BS_data = np.append(BS_data,
@ -218,27 +130,21 @@ class AcousticDataLoaderUBSediFlow:
axis=0)
self._BS_raw_data = np.array([BS_data[:self._time.shape[1], :].transpose()])
# print(f"a) BS_data shape = {BS_data.shape}")
# print(f"a) BS_raw_data shape = {BS_raw_data.shape}")
else:
BS_data = np.array([data_us_dicts[config][channel]['echo_avg_profile']['data'][0]])
print(f"{config}) BS_data shape = {BS_data.shape}")
for j in range(self._time.shape[1]):
BS_data = np.append(BS_data,
np.array(
[data_us_dicts[config][channel]['echo_avg_profile']['data'][j]]),
axis=0)
BS_data = np.array([BS_data.transpose()])
print(f"xxxx BS_data shape = {BS_data.shape}")
# print(f"b) BS_raw_data shape = {BS_raw_data.shape}")
# 1- time shape > BS data shape
# <=> data recorded with the frequency are longer than data recorded with the other lower frequencies
if (BS_data.shape[2] > self._BS_raw_data.shape[2]):
print(f"cas 2 : BS_raw_data shape = {self._BS_raw_data.shape}")
print(f"cas 2 : BS_data shape = {BS_data.shape}")
if (BS_data.shape[1] > self._BS_raw_data.shape[1]):
# print(f"BS_data shape[0] = {BS_data.shape[0]}")
@ -260,8 +166,6 @@ class AcousticDataLoaderUBSediFlow:
# 2- time shape < BS data shape
# <=> data recorded with the frequency are shorter than data recorded with the other lower frequencies
elif BS_data.shape[2] < self._BS_raw_data.shape[2]:
print(f"cas 3 : BS_raw_data shape = {self._BS_raw_data.shape}")
print(f"cas 3 : BS_data shape = {BS_data.shape}")
if (BS_data.shape[1] > self._BS_raw_data.shape[1]):
self._BS_raw_data = np.append(self._BS_raw_data[:, :, BS_data.shape[2]],
@ -277,16 +181,11 @@ class AcousticDataLoaderUBSediFlow:
self._BS_raw_data = np.append(self._BS_raw_data[:, :, BS_data.shape[0]],
BS_data,
axis=0)
# print(f"d) BS_data shape = {BS_data.shape}")
# print(f"d) BS_raw_data shape = {BS_raw_data.shape}")
# 3- time shape = BS data shape
# <=> data recorded with the frequency have the same duration than data recorded with the other lower frequency
else:
print(f"cas 4 : BS_raw_data shape = {self._BS_raw_data.shape}")
print(f"cas 4 : BS_data shape = {BS_data.shape}")
if (BS_data.shape[1] > self._BS_raw_data.shape[1]):
self._BS_raw_data = np.append(self._BS_raw_data,
@ -301,115 +200,6 @@ class AcousticDataLoaderUBSediFlow:
self._BS_raw_data = np.append(self._BS_raw_data,
BS_data, axis=0)
# print(f"e) BS_data shape = {BS_data.shape}")
print("Final BS_raw_data shape = ", self._BS_raw_data.shape)
print("********************************************")
# # --- US Backscatter raw signal + SNR data ---
# BS_data = np.array([[]])
#
# if config == 1:
# BS_data = np.array([data_us_dicts[config][channel]['echo_avg_profile']['data'][0]])
# # print("BS_data shape ", BS_data.shape)
# # print("******************************")
# # date_list = [np.abs(datetime.datetime(2021, 5, 19, 14, 10, 00).timestamp()
# # - date.timestamp()) for date in data_us_dicts[config][channel]['echo_avg_profile']['time']]
# # print(date_list)
# # print(np.where(date_list == np.min(date_list)))
# # print((data_us_dicts[config][channel]['echo_avg_profile']['time'][np.where(date_list == np.min(date_list))[0][0]] -
# # data_us_dicts[config][channel]['echo_avg_profile']['time'][0]).total_seconds())
# # # == datetime.datetime(2021, 5, 19, 14, 10, 2, 644000))
# # print("******************************")
#
# for i in range(self._time.shape[1]):
# BS_data = np.append(BS_data,
# np.array([data_us_dicts[config][channel]['echo_avg_profile']['data'][i]]),
# axis=0)
# print("0. BS_data shape ", BS_data.shape)
#
# self._BS_raw_data = np.array([BS_data[:self._time.shape[1], :].transpose()])
#
# print("0. BS_raw_data shape ", self._BS_raw_data.shape)
#
# # fig, ax = plt.subplots(nrows=1, ncols=1, layout="constrained")
# # pcm = ax.pcolormesh(list(range(self._BS_raw_data.shape[2])), list(range(self._BS_raw_data.shape[1])),
# # np.log(self._BS_raw_data[0, :, :]),
# # cmap='Blues')
# # fig.colorbar(pcm, ax=ax, shrink=1, location='right')
# # plt.show()
#
# else:
#
# BS_data = np.array([data_us_dicts[config][channel]['echo_avg_profile']['data'][0]])
# # print("BS_data shape ", BS_data.shape)
# for i in range(self._time.shape[1]):
# BS_data = np.append(BS_data,
# np.array(
# [data_us_dicts[config][channel]['echo_avg_profile']['data'][i]]),
# axis=0)
# print("1. BS_data shape ", BS_data.shape)
#
# #-----------------------------------------------------------------------------------------------------------------------
# # Ici il faut écrire les conditions sur les tailles selon r et selon time
# # donc sur BS_data.shape[0] (time) et BS_data.shape[1] (depth)
# #-----------------------------------------------------------------------------------------------------------------------
#
# # 1- time shape > BS data shape
# # <=> data recorded with the frequency are longer than data recorded with the other lower frequencies
# if (BS_data.shape[0] > self._BS_raw_data.shape[2]):
# self._BS_raw_data = np.append(self._BS_raw_data,
# np.array([BS_data[:self._BS_raw_data.shape[2], :].transpose()]),
# axis=0)
#
# # 2- time shape < BS data shape
# # <=> data recorded with the frequency are shorter than data recorded with the other lower frequencies
# elif BS_data.shape[0] < self._BS_raw_data.shape[2]:
# self._BS_raw_data = np.append(self._BS_raw_data[config-1, :, BS_data.shape[0]],
# np.array([BS_data.transpose()]),
# axis=0)
#
# # 3- time shape = BS data shape
# # <=> data recorded with the frequency have the same duration than data recorded with the other lower frequency
# else:
# self._BS_raw_data = np.append(self._BS_raw_data, np.array([BS_data.transpose()]), axis=0)
#
#
# print("1. BS_raw_data shape ", self._BS_raw_data.shape)
#
# # if f == 0:
# # print(np.array(data_us_dicts[config][channel]['echo_avg_profile']['data'][0]).shape)
# # self._BS_raw_data[f, :, :] = np.array([data_us_dicts[config][channel]['echo_avg_profile']['data'][0]])
# # # self._BS_raw_data = np.array([np.reshape(data_us_dicts[config][channel]['echo_avg_profile']['data'],
# # # (self._time.shape[1], self._r.shape[1])).transpose()])
# # print("self._BS_raw_data.shape ", self._BS_raw_data.shape)
# # self._SNR_data = np.array(
# # [np.reshape(np.abs(data_us_dicts[config][channel]['snr_doppler_avg_profile']['data']),
# # (self._time.shape[1], self._r.shape[1])).transpose()])
# # else:
# # # self._BS_raw_data = np.append(self._BS_raw_data,
# # # np.array(data_us_dicts[config][channel]['echo_avg_profile']['data']),
# # # (self._r.shape[1], self._time.shape[1]))]),
# # # axis=0)
# # # self._BS_raw_data = np.append(self._BS_raw_data,
# # # np.array([np.reshape(np.array(
# # # data_us_dicts[config][channel]['echo_avg_profile']['data']),
# # # (self._time.shape[1], self._r.shape[1])).transpose()]),
# # # axis=0)
# #
# # self._SNR_data = np.append(self._SNR_data,
# # np.array([np.reshape(np.array(
# # np.abs(data_us_dicts[config][channel]['snr_doppler_avg_profile']['data'])),
# # (self._time.shape[1], self._r.shape[1])).transpose()]),
# # axis=0)
# # # print(self._BS_raw_data.shape)
#
# # --- US Backscatter raw signal ---
#
#
# # print(len(self._BS_raw_data))
# # print(self._BS_raw_data)
if self._time.shape[1] > self._BS_raw_data.shape[2]:
self._time = self._time[:, :self._BS_raw_data.shape[2]]
@ -420,275 +210,39 @@ class AcousticDataLoaderUBSediFlow:
self._BS_raw_data = self._BS_raw_data
self._time = self._time[:, :self._BS_raw_data.shape[2]]
print("self._time.shape ", self._time.shape)
# print(f"time : {self._time}")
# print("****************************")
# for i in range(len(self._time[0, :])-1):
# print(self._time[0, i+1] - self._time[0, i])
# print("****************************")
print("self._r.shape ", self._r.shape)
self._freq_text = np.array([str(f) + " MHz" for f in [np.round(f*1e-6, 2) for f in self._freq]])
print("self._freq_text ", self._freq_text)
print("self._freq_text ", self._freq)
# self._BS_raw_data = np.array(np.reshape(self._BS_raw_data, (len(self._freq), self._r.shape[1], self._time.shape[1])))
print("self._BS_raw_data.shape ", self._BS_raw_data.shape)
# print("self._SNR_data.shape ", self._SNR_data.shape)
# print(self._SNR_data)
# print("device_name ", device_name, "\n")
# print("time_begin ", time_begin, "\n")
# print("time_end ", time_end, "\n")
# print(f"param_dicts keys {param_us_dicts.keys()} \n")
# print(param_us_dicts, "\n")
# for i in range(len(list(param_us_dicts.keys()))):
# print(f"param_us_dicts {i} : {list(param_us_dicts.items())[i]} \n")
# # print("settings_dict ", settings_dict, "\n")
# print(f"keys in data_us_dicts {data_us_dicts[1][1].keys()} \n")
# # les clés du dictionnaire data_us_dicts :
# # dict_keys(['echo_avg_profile', 'saturation_avg_profile', 'velocity_avg_profile', 'snr_doppler_avg_profile',
# # 'velocity_std_profile', 'a1_param', 'a0_param', 'noise_g_high', 'noise_g_low'])
# print(f"data_us_dicts keys in echo avg profile {data_us_dicts[1][1]['echo_avg_profile'].keys()} \n")
# print(f"number of profiles {len(data_us_dicts[1][1]['echo_avg_profile']['data'])} \n")
# print(f"number of cells {data_us_dicts[1][1]['echo_avg_profile']['data'][0].shape} \n")
# self._data_BS = RawAquascatData(self.path_BS_raw_data)
# self._nb_profiles = self._data_BS.NumProfiles
# self._nb_profiles_per_sec = self._data_BS.ProfileRate
# self._nb_cells = self._data_BS.NumCells
# self._cell_size = self._data_BS.cellSize
# self._pulse_length = self._data_BS.TxPulseLength
# self._nb_pings_per_sec = self._data_BS.PingRate
# self._nb_pings_averaged_per_profile = self._data_BS.Average
# self._kt = self._data_BS.Kt
# self._gain_rx = self._data_BS.RxGain
# self._gain_tx = self._data_BS.TxGain
# self._snr = np.array([])
# self._snr_reshape = np.array([])
# self._time_snr = np.array([])
# print(type(self._gain_tx))
# print(["BS - " + f for f in self._freq_text])
# print(self._time.shape[0]*self._r.shape[0]*4)
# print(self._time[np.where(np.floor(self._time) == 175)])
# print(np.where((self._time) == 155)[0][0])
# --- Plot Backscatter US data ---
# fig, ax = plt.subplots(nrows=1, ncols=1, layout="constrained")
# pcm = ax.pcolormesh(self._time[0, :], -self._r[0, :], np.log(self._BS_raw_data[0, :, :]),
# cmap='plasma')#, shading='gouraud')
# # pcm = ax.pcolormesh(list(range(self._BS_raw_data.shape[2])), list(range(self._BS_raw_data.shape[1])),
# # np.log(self._BS_raw_data[0, :, :]),
# # cmap='Blues') # , shading='gouraud')
# # norm=LogNorm(vmin=np.min(self._BS_raw_data[f, :, :]), vmax=np.max(self._BS_raw_data[f, :, :])), shading='gouraud')
# # ax.pcolormesh(range(self._BS_raw_data.shape[2]), range(self._BS_raw_data.shape[0]), self._BS_raw_data[:, 1, :], cmap='viridis',
# # norm=LogNorm(vmin=1e-5, vmax=np.max(self._BS_raw_data[:, 0, :]))) # , shading='gouraud')
# fig.colorbar(pcm, ax=ax, shrink=1, location='right')
# plt.show()
# fig, ax = plt.subplots(nrows=len(self._freq), ncols=1, layout="constrained")
# for f, freq in enumerate(self._freq):
# print(f"{f} : {freq} \n")
# # pcm = ax[f].imshow(np.log(self._BS_raw_data[f, :, :self._time.shape[1]]),
# # cmap='Blues')
# # pcm = ax[f].pcolormesh(list(range(self._BS_raw_data.shape[2])), list(range(self._BS_raw_data.shape[1])),
# # np.log(self._BS_raw_data[f, :, :]),
# # cmap='Blues', shading='gouraud')
# pcm = ax[f].pcolormesh(self._time[f, 50:247], -self._r[f, :], np.log(self._BS_raw_data[f, :, 50:247]),
# cmap='Blues')#, shading='gouraud')
# # norm=LogNorm(vmin=np.min(self._BS_raw_data[f, :, :]), vmax=np.max(self._BS_raw_data[f, :, :])), shading='gouraud')
# # ax.pcolormesh(range(self._BS_raw_data.shape[2]), range(self._BS_raw_data.shape[0]), self._BS_raw_data[:, 1, :], cmap='viridis',
# # norm=LogNorm(vmin=1e-5, vmax=np.max(self._BS_raw_data[:, 0, :]))) # , shading='gouraud')
# fig.colorbar(pcm, ax=ax[:], shrink=1, location='right')
# # plt.show()
#
# # --- Smooth value with savgol_filter ---
# BS_smooth = deepcopy(self._BS_raw_data[0, :, :])
# for k in range(self._time[0, :].shape[0]):
# BS_smooth[:, k] = savgol_filter(BS_smooth[:, k], 10, 2)
#
# fig1, ax1 = plt.subplots(nrows=1, ncols=1, layout="constrained")
# pcm1 = ax1.pcolormesh(self._time[0, :], -self._r[0, :], np.log(BS_smooth[:, :]), cmap='Blues')
# fig1.colorbar(pcm1, ax=ax1, shrink=1, location='right')
# print("find value in depth", np.where(np.abs(self._r - 3.3) == np.min(np.abs(self._r - 3.3))))
#
# fig, ax = plt.subplots(nrows=1, ncols=1, layout="constrained")
# ax.plot(self._r[0, :], self._BS_raw_data[0, :, 766])
# plt.show()
# # --- Plot vertical profile for bottom detection ---
# n = 60
# t0 = 200
# t1 = np.where(np.abs(self._time[0, :] - t0) == np.nanmin(np.abs(self._time[0, :] - t0)))[0][0]
# # print(np.abs(self._time[0, :] - 200))
# # print(f"x0 = {x0}")
# r1 = 98
# r2 = 150
# fig2, ax2 = plt.subplots(nrows=1, ncols=n, layout="constrained")
# for i in range(n):
# ax2[i].plot(self._BS_raw_data[0, r1:r2, t1+i], -self._r[0, r1:r2], 'b')
# ax2[i].plot(BS_smooth[r1:r2, t1+i], -self._r[0, r1:r2], 'r')
# ax2[i].set_xticks([])
# if i != 0:
# ax2[i].set_yticks([])
# plt.show()
# --- Plot SNR data ---
# fig_snr, ax_snr = plt.subplots(nrows=len(self._freq), ncols=1)
#
# x, y = np.meshgrid(self._time[0, :], self._r[0, :])
#
# for f, freq in enumerate(self._freq):
#
# val_min = np.nanmin(abs(self._SNR_data[f, :, :]))
# print(f"val_min = {val_min}")
# val_max = np.nanmax(self._SNR_data[f, :, :])
# print(f"val_max = {val_max}")
# if int(val_min) == 0:
# val_min = 1e-5
# if int(val_max) < 1000:
# levels = np.array([00.1, 1, 2, 10, 100, 1000, 1e6])
# bounds = [00.1, 1, 2, 10, 100, 1000, 1e6, 1e6 * 1.2]
# else:
# levels = np.array([00.1, 1, 2, 10, 100, val_max])
# bounds = [00.1, 1, 2, 10, 100, 1000, val_max, val_max * 1.2]
# norm = BoundaryNorm(boundaries=bounds, ncolors=300)
#
# print(f"levels = {levels}")
# print(f"norm = {norm.boundaries}")
#
# cf = ax_snr[f].contourf(x, y, self._SNR_data[f, :, :])#, levels, cmap='gist_rainbow', norm=norm)
#
# ax_snr[f].text(1, .70, self._freq_text[f],
# fontsize=14, fontweight='bold', fontname="Ubuntu", c="black", alpha=0.5,
# horizontalalignment='right', verticalalignment='bottom',
# transform=ax_snr[f].transAxes)
#
# fig_snr.supxlabel('Time (sec)', fontsize=10)
# fig_snr.supylabel('Depth (m)', fontsize=10)
# cbar = fig_snr.colorbar(cf, ax=ax_snr[:], shrink=1, location='right')
# cbar.set_label(label='Signal to Noise Ratio', rotation=270, labelpad=10)
# # cbar.set_ticklabels(['0', '1', '2', '10', '100', r'10$^3$', r'10$^6$'])
# plt.show()
# fig, ax = plt.subplots(nrows=1, ncols=1)
# ax.plot(list(range(self._time.shape[1])), self._time[0, :])
# # ax.set_ylim(2, 20)
# plt.show()
# print(self.reshape_BS_raw_cross_section())
# self.reshape_BS_raw_cross_section()
# self.reshape_r()
# self.reshape_t()
# self.compute_r_2D()
# Lecture du fichier excel
# path = ("/home/bmoudjed/Documents/3 SSC acoustic meas project/Graphical interface project/Data/APAVER_2021/"
# "transect_ubsediflow/01-raw_20210519_115128/Raw_data_csv/config_1/"
# "echo_avg_profile_1_1_20210519_115128.csv")
#
# df = pd.read_csv(path, sep="\t")
#
# arr = []
# for column in df.columns:
# arr.append(df[column].to_numpy())
# # arr = np.append(arr, np.array([df[column].to_numpy()]), axis=0)
# arr = arr[1:]
# print(len(arr))
#
# matrix = np.array([arr[0]])
# print(matrix.shape)
# for i in range(len(arr)-1):
# matrix = np.append(matrix, np.array([arr[i]]), axis=0)
# print(matrix.shape)
# fig, ax = plt.subplots(nrows=1, ncols=1, layout="constrained")
# pcm = ax.pcolormesh(list(range(matrix.shape[1])), list(range(matrix.shape[0])), np.log(matrix),
# cmap='Blues')#, shading='gouraud')
# # norm=LogNorm(vmin=np.min(self._BS_raw_data[f, :, :]), vmax=np.max(self._BS_raw_data[f, :, :])), shading='gouraud')
# # ax.pcolormesh(range(self._BS_raw_data.shape[2]), range(self._BS_raw_data.shape[0]), self._BS_raw_data[:, 1, :], cmap='viridis',
# # norm=LogNorm(vmin=1e-5, vmax=np.max(self._BS_raw_data[:, 0, :]))) # , shading='gouraud')
# fig.colorbar(pcm, ax=ax, shrink=1, location='right')
# plt.show()
def reshape_BS_raw_data(self):
BS_raw_cross_section = np.reshape(self._BS_raw_data,
(self._r.shape[1]*self._time.shape[1], len(self._freq)),
order="F")
# print(BS_raw_cross_section.shape)
return BS_raw_cross_section
# def reshape_SNR_data(self):
# SNR_data = np.reshape(self._SNR_data,
# (self._r.shape[1]*self._time.shape[1], len(self._freq)),
# order="F")
# # print(BS_raw_cross_section.shape)
# return SNR_data
def reshape_r(self):
r = np.zeros((self._r.shape[1]*self._time.shape[1], len(self._freq)))
for i, _ in enumerate(self._freq):
r[:, i] = np.repeat(self._r[i, :], self._time.shape[1])
# print(r.shape)
return r
def compute_r_2D(self):
r2D = np.zeros((self._freq.shape[0], self._r.shape[1], self._time.shape[1]))
for f, _ in enumerate(self._freq):
r2D[f, :, :] = np.repeat(np.transpose(self._r[0, :])[:, np.newaxis], self._time.shape[1], axis=1)
print("r2D.shape ", r2D.shape)
return r2D
# def compute_r_2D(self):
# r2D = np.repeat(self._r, self._time.size, axis=1)
# return r2D
def reshape_t(self):
t = np.zeros((self._r.shape[1]*self._time.shape[1], len(self._freq)))
for i, _ in enumerate(self._freq):
t[:, i] = np.repeat(self._time[i, :], self._r.shape[1])
# print(t.shape)
return t
def reshape_t_snr(self):
t = np.zeros((self._r.shape[1]*self._time_snr.shape[1], len(self._freq)))
for i, _ in enumerate(self._freq):
t[:, i] = np.repeat(self._time_snr[i, :], self._r.shape[1])
# print(t.shape)
return t
def detect_bottom(self):
rmin = 2.5
rmax = 3.5
# def concatenate_data(self):
# self.reshape_BS_raw_cross_section()
# # print(self.reshape_t().shape)
# # print(se.lf.reshape_BS_raw_cross_section().shape)
# df = pd.DataFrame(np.concatenate((self.reshape_t(), self.reshape_BS_raw_cross_section()), axis=1),
# columns=["time"] + self._freq_text)
# return df
# if __name__ == "__main__":
# AcousticDataLoaderUBSediFlow(path_BS_raw_data0 + filename0)

View File

@ -21,7 +21,6 @@
# -*- coding: utf-8 -*-
import matplotlib.pyplot as plt
import numpy as np
import settings as stg
from Model.GrainSizeTools import demodul_granulo, mix_gaussian_model
@ -58,17 +57,6 @@ class AcousticInversionMethodHighConcentration():
(np.log(10) / 20) * (freq * 1e-3) ** 2
return alpha
# ---------- Conmpute FBC ----------
# def compute_FCB(self):
# # print(self.BS_averaged_cross_section_corr.V.shape)
# # print(self.r_2D.shape)
# FCB = np.zeros((256, 4, 1912))
# for f in range(4):
# # print(self.alpha_w_function(self.Freq[f], self.temperature))
# FCB[:, f, :] = np.log(self.BS_averaged_cross_section_corr.V[:, f, :]) + np.log(self.r_3D[:, f, :]) + \
# np.log(2 * self.alpha_w_function(self.Freq[f], self.temperature) * self.r_3D[:, f, :])
# return FCB
# --- Gaussian mixture ---
def compute_particle_size_distribution_in_number_of_particles(self, num_sample, r_grain, frac_vol_cumul):
min_demodul = 1e-6
@ -82,15 +70,6 @@ class AcousticInversionMethodHighConcentration():
sample_demodul.demodul_data_list[2].sigma_list,
sample_demodul.demodul_data_list[2].w_list)
# N_modes = 3
# sample_demodul.print_mode_data(N_modes)
# sample_demodul.plot_interpolation()
# sample_demodul.plot_modes(N_modes)
# print(f"mu_list : {sample_demodul.demodul_data_list[3 - 1].mu_list}")
# print(f"sigma_list : {sample_demodul.demodul_data_list[3 - 1].sigma_list}")
# print(f"w_list : {sample_demodul.demodul_data_list[3 - 1].w_list}")
proba_vol_demodul = proba_vol_demodul / np.sum(proba_vol_demodul)
ss = np.sum(proba_vol_demodul / np.exp(resampled_log_array) ** 3)
proba_num = proba_vol_demodul / np.exp(resampled_log_array) ** 3 / ss
@ -106,23 +85,9 @@ class AcousticInversionMethodHighConcentration():
x = k * a
f = (x ** 2 * (1 - 0.25 * np.exp(-((x - 1.5) / 0.35) ** 2)) * (1 + 0.6 * np.exp(-((x - 2.9) / 1.15) ** 2))) / (
42 + 28 * x ** 2)
# print(f"form factor = {f}")
return f
# def ks(self, num_sample_sand, radius_grain_sand, frac_vol_sand_cumul, freq, C):
def ks(self, proba_num, freq, C):
# --- Calcul de la fonction de form ---
# form_factor = self.form_factor_function_MoateThorne2012(a, freq)
# print(f"form_factor shape = {form_factor}")
# print(f"form_factor = {form_factor}")
#--- Particle size distribution ---
# proba_num = (
# self.compute_particle_size_distribution_in_number_of_particles(
# num_sample=num_sample_sand, r_grain=radius_grain_sand, frac_vol_cumul=frac_vol_sand_cumul[num_sample_sand]))
# print(f"proba_num : {proba_num}")
# --- Compute k_s by dividing two integrals ---
resampled_log_array = np.log(np.logspace(-10, -2, 3000))
a2f2pdf = 0
@ -132,28 +97,17 @@ class AcousticInversionMethodHighConcentration():
a2f2pdf += a**2 * self.form_factor_function_MoateThorne2012(a, freq, C)**2 * proba_num[i]
a3pdf += a**3 * proba_num[i]
# print("form factor ", self.form_factor_function_MoateThorne2012(a, freq, C))
# print(f"a2f2pdf = {a2f2pdf}")
# print(f"a3pdf = {a3pdf}")
ks = np.sqrt(a2f2pdf / a3pdf)
# ks = np.array([0.04452077, 0.11415143, 0.35533713, 2.47960051])
# ks = ks0[ind]
return ks
# ------------- Computing sv ------------- #
def sv(self, ks, M_sand):
# print(f"ks = {ks}")
# print(f"M_sand = {M_sand}")
sv = (3 / (16 * np.pi)) * (ks ** 2) * M_sand
# sv = np.full((stg.r.shape[1], stg.t.shape[1]), sv0)
return sv
# ------------- Computing X ------------- #
def X_exponent(self, freq1, freq2, sv_freq1, sv_freq2):
# X0 = [3.450428714146802, 3.276478927777019, 3.6864638665972893, 0]
# X = X0[ind]
X = np.log(sv_freq1 / sv_freq2) / np.log(freq1 / freq2)
return X
@ -174,165 +128,43 @@ class AcousticInversionMethodHighConcentration():
gain = 10 ** ((RxGain + TxGain) / 20)
# Computing Kt
kt = kt_ref * gain * np.sqrt(tau * cel / (tau_ref * c_ref)) # 1D numpy array
# kt = np.reshape(kt0, (1, 2)) # convert to 2d numpy array to compute J_cross_section
# print(f"kt = {kt}")
# kt_2D = np.repeat(np.array([kt]), stg.r.shape[1], axis=0)
# print("kt 2D ", kt_2D)
# print("kt 2D shape ", kt_2D.shape)
# # kt_3D = np.zeros((kt_2D.shape[1], kt_2D.shape[0], stg.t.shape[1]))
# # for k in range(kt_2D.shape[1]):
# # kt_3D[k, :, :] = np.repeat(kt_2D, stg.t.shape[1], axis=1)[:, k * stg.t.shape[1]:(k + 1) * stg.t.shape[1]]
# kt_3D = np.repeat(kt_2D.transpose()[:, :, np.newaxis], stg.t.shape[1], axis=2)
# # print("kt 3D ", kt_3D)
# print("kt 3D shape ", kt_3D.shape)
return kt
# ------------- Computing J_cross_section ------------- #
def j_cross_section(self, BS, r2D, kt):
# J_cross_section = np.zeros((1, BS.shape[1], BS.shape[2])) # 2 because it's a pair of frequencies
# print("BS.shape", BS.shape)
# print("r2D.shape", r2D.shape)
# print("kt.shape", kt.shape)
# if stg.ABS_name == "Aquascat 1000R":
# print("--------------------------------")
# print("BS : ", BS)
# print("BS min : ", np.nanmin(BS))
# print("BS max : ", np.nanmax(BS))
# print("r2D : ", r2D)
# print("kt shape : ", kt.shape)
# print("kt : ", kt)
# print("--------------------------------")
# for k in range(1):
# J_cross_section[k, :, :] = (3 / (16 * np.pi)) * ((BS[k, :, :]**2 * r2D[k, :, :]**2) / kt[k, :, :]**2)
J_cross_section = (3 / (16 * np.pi)) * ((BS**2 * r2D**2) / kt**2)
# J_cross_section[J_cross_section == 0] = np.nan
# print("J_cross_section.shape", J_cross_section.shape)
# elif stg.ABS_name == "UB-SediFlow":
# for k in range(1):
# J_cross_section[k, :, :] = (3 / (16 * np.pi)) * ((BS[k, :, :]**2 * r2D[0, :, :]**2) / kt[k, :, :]**2)
# print("compute j_cross_section finished")
return J_cross_section
# ------------- Computing alpha_s ------------- #
def alpha_s(self, sv, j_cross_section, depth, alpha_w):
alpha_s = (np.log(sv / j_cross_section) / (4 * depth)) - alpha_w
print("----------------------------")
print(f"sv = {sv}")
print(f"j_cross_section = {j_cross_section}")
print(f"depth = {depth}")
print(f"alpha_w = {alpha_w}")
print(f"(np.log(sv / j_cross_section) / (4 * depth)) = {(np.log(sv / j_cross_section) / (4 * depth))}")
print(f"alpha_s {alpha_s}")
return alpha_s
# ------------- Computing interpolation of fine SSC data obtained from water sampling -------------
# ------------- collected at various depth in the vertical sample -------------
# def M_profile_SCC_fine_interpolated(self, sample_depth, M_profile, range_cells, r_bottom):
# res = np.zeros((len(range_cells),)) * np.nan
# for i in range(len(M_profile) - 1):
# # print(f"i = {i}")
# r_ini = sample_depth[i]
# # print(f"r_ini = {r_ini}")
# c_ini = M_profile[i]
# # print(f"c_ini = {c_ini}")
# r_end = sample_depth[i + 1]
# # print(f"r_end = {r_end}")
# c_end = M_profile[i + 1]
# # print(f"c_end = {c_end}")
#
# # Computing the linear equation
# a = (c_end - c_ini) / (r_end - r_ini)
# # print(f"a = {a}")
# b = c_ini - a * r_ini
# # print(f"b = {b}")
#
# # Finding the indices of r_ini and r_end in the interpolated array
# # print(f"range_cells = {range_cells}")
# loc = (range_cells >= r_ini) * (range_cells < r_end)
# # print(f"loc = {loc}")
# # print(f"loc shape = {len(loc)}")
#
# # Filling the array with interpolation values
# res[loc] = range_cells[loc] * a + b
# # print(res.shape)
# # print(f"res = {res}")
# # print(f"1. res.shape = {res.shape}")
#
# # Filling first and last values
# i = 0
# while np.isnan(res[i]):
# res[i] = M_profile[0]
# i += 1
#
# # Filling the last values
# i = -1
# while np.isnan(res[i]):
# res[i] = M_profile[-1]
# i += -1
# # print(f"res.shape = {res.shape}")
# # print(f"res = {res}")
# # print(f"r_bottom.shape = {r_bottom.shape}")
# # print(f" = {res}")
#
# if r_bottom.shape != (0,):
# res[np.where(range_cells > r_bottom)] = np.nan
#
# loc_point_lin_interp0 = range_cells[np.where((range_cells > sample_depth[0]) & (range_cells < sample_depth[-1]))]
# # print(f"range_cells : {range_cells}")
# # print(f"loc_point_lin_interp0 shape : {len(loc_point_lin_interp0)}")
# # print(f"loc_point_lin_interp0 : {loc_point_lin_interp0}")
# res0 = res[np.where((range_cells > sample_depth[0]) & (range_cells < sample_depth[-1]))]
#
# loc_point_lin_interp = loc_point_lin_interp0[np.where(loc_point_lin_interp0 > range_cells[0])]
# # print(f"loc_point_lin_interp shape : {len(loc_point_lin_interp)}")
# # print(f"loc_point_lin_interp : {loc_point_lin_interp}")
# res = res0[np.where(loc_point_lin_interp0 > range_cells[0])]
#
# # fig, ax = plt.subplots(nrows=1, ncols=1)
# # ax.plot(loc_point_lin_interp, res[:len(loc_point_lin_interp)], marker="*", mfc="blue")
# # ax.plot(sample_depth, M_profile, marker="o", mfc="k", mec="k")
# # plt.show()
#
# return (loc_point_lin_interp, res)
# ------------- Computing interpolation of fine SSC -------------
def M_profile_SCC_fine_interpolated(self, sample_depth, M_profile, range_cells, r_bottom):
'''Computing interpolation of fine SSC data obtained from water sampling
collected at various depth in the vertical sample'''
res = np.zeros((len(range_cells),)) * np.nan
print("range_cells ", range_cells.shape)
l0 = sample_depth
print("l0 = ", l0)
l1 = [l0.index(x) for x in sorted(l0)]
print("l1 = ", l1)
l2 = [l0[k] for k in l1]
print("l2 = ", l2)
c1 = [list(M_profile)[j] for j in l1]
print("c1 = ", c1)
for i in range(len(c1) - 1):
# print("i = ", i)
r_ini = l2[i]
c_ini = c1[i]
r_end = l2[i + 1]
c_end = c1[i + 1]
print("r_ini ", r_ini, "c_ini ", c_ini, "r_end ", r_end, "c_end ", c_end)
# Computing the linear equation
a = (c_end - c_ini) / (r_end - r_ini)
b = c_ini - a * r_ini
print("range_cells ", (range_cells))
# Finding the indices of r_ini and r_end in the interpolated array
loc = (range_cells >= r_ini) * (range_cells < r_end)
print("range_cells >= r_ini ", range_cells >= r_ini)
print("range_cells < r_end ", range_cells < r_end)
print("loc ", loc)
# Filling the array with interpolation values
res[loc] = range_cells[loc] * a + b
print("a = ", a, "b = ", b)
print("res ", res)
# Filling first and last values
i = 0
while np.isnan(res[i]):
@ -346,9 +178,6 @@ class AcousticInversionMethodHighConcentration():
i += -1
if r_bottom.size != 0:
print("res ", res.shape)
print("range_cells ", len(range_cells))
# print("r_bottom ", len(r_bottom))
res[np.where(range_cells > r_bottom)] = np.nan
loc_point_lin_interp0 = range_cells[np.where((range_cells > l2[0]) & (range_cells < l2[-1]))]
@ -357,13 +186,6 @@ class AcousticInversionMethodHighConcentration():
loc_point_lin_interp = loc_point_lin_interp0[np.where(loc_point_lin_interp0 > l2[0])]
res = res0[np.where(loc_point_lin_interp0 > l2[0])]
# fig, ax = plt.subplots(nrows=1, ncols=1)
# ax.plot(res[:len(loc_point_lin_interp)], -loc_point_lin_interp, marker="*", mfc="blue")
# ax.plot(c1, [-x for x in l2], marker="o", mfc="k", mec="k", ls="None")
# ax.set_xlabel("Concentration (g/L)")
# ax.set_ylabel("Depth (m)")
# plt.show()
return (loc_point_lin_interp, res)
# ------------- Computing zeta ------------- #
@ -372,39 +194,6 @@ class AcousticInversionMethodHighConcentration():
delta_r = r[1] - r[0]
zeta = alpha_s / (np.sum(np.array(M_profile_fine)*delta_r))
# print(f"np.sum(M_profile_fine*delta_r) : {np.sum(M_profile_fine*delta_r)}")
# zeta0 = np.array([0.021, 0.035, 0.057, 0.229])
# zeta = zeta0[ind]
# zeta0 = np.array([0.04341525, 0.04832906, 0.0847188, np.nan])
# zeta = zeta0[[ind1, ind2]]
# for k in range(3):
# for p in range(3):
# if np.isnan(ind_X_min_around_sample[p, k]):
# zeta_list_exp.append(np.nan)
# else:
# ind_X_min = int(ind_X_min_around_sample[p, k])
# ind_X_max = int(ind_X_max_around_sample[p, k])
# ind_r_min = int(ind_r_min_around_sample[p, k])
# ind_r_max = int(ind_r_max_around_sample[p, k])
#
# R_temp = R_cross_section[ind_r_min:ind_r_max, :, ind_X_min:ind_X_max]
# J_temp = J_cross_section[ind_r_min:ind_r_max, :, ind_X_min:ind_X_max]
# aw_temp = aw_cross_section[ind_r_min:ind_r_max, :, ind_X_min:ind_X_max]
# sv_temp_1 = np.repeat([sv_list_temp[3 * k + p]], np.shape(R_temp)[0], axis=0)
# sv_temp = np.swapaxes(np.swapaxes(np.repeat([sv_temp_1], np.shape(R_temp)[2], axis=0), 1, 0), 2, 1)
# ind_depth = np.where(R_cross_section[:, 0, 0] >= M_list_temp[k][0, p + 1])[0][0]
# # Using concentration profile
# zeta_temp = alpha_s / ((1 / M_list_temp[k][0, p + 1]) * (R_cross_section[0, 0, 0] * M_list_temp[k][1, 0] +
# delta_r * np.sum(M_interpolate_list[k][:ind_depth])))
# zeta_temp = (1 / (4 * R_temp) *
# np.log(sv_temp / J_temp) - aw_temp) / ((1 / M_list_temp[k][0, p + 1]) *
# (R_cross_section[0, 0, 0] * M_list_temp[k][
# 1, 0] +
# delta_r * np.sum(
# M_interpolate_list[k][:ind_depth])))
# zeta_list_exp.append(np.mean(np.mean(zeta_temp, axis=0), axis=1))
return zeta
# ------------- Computing VBI ------------- #
@ -415,21 +204,6 @@ class AcousticInversionMethodHighConcentration():
water_attenuation_freq1, water_attenuation_freq2,
X):
# print('self.zeta_exp[ind_j].shape', self.zeta_exp[ind_j])
# print('np.log(self.j_cross_section[:, ind_i, :]).shape', np.log(self.j_cross_section[:, ind_i, :]).shape)
# print('self.r_3D[:, ind_i, :]', self.r_3D[:, ind_i, :].shape)
# print('self.water_attenuation[ind_i]', self.water_attenuation[ind_i])
# print('self.x_exp[0.3-1 MHz]', self.x_exp['0.3-1 MHz'].values[0])
# print("start computing VBI")
# print("================================")
# print(f"zeta_freq2 : {zeta_freq2}")
# print(f"j_cross_section_freq1 : {j_cross_section_freq1.shape}")
# print(f"r2D : {r2D.shape}")
# print(f"water_attenuation_freq1 : {water_attenuation_freq1}")
# print(f"freq1 : {freq1}")
# print(f"X : {X}")
# print("================================")
logVBI = ((zeta_freq2 *
np.log(j_cross_section_freq1 * np.exp(4 * r2D * water_attenuation_freq1) /
(freq1 ** X)) -
@ -438,31 +212,16 @@ class AcousticInversionMethodHighConcentration():
(freq2 ** X))) /
(zeta_freq2 - zeta_freq1))
# logVBI = (freq2**2 * np.log(j_cross_section_freq1 / freq1**X) -
# freq1**2 * np.log(j_cross_section_freq2 / freq2**X)) / (freq2**2 - freq1**2)
# logVBI = (( np.full((stg.r.shape[1], stg.t.shape[1]), zeta_freq2) *
# np.log(j_cross_section_freq1 * np.exp(4 * r2D * np.full((stg.r.shape[1], stg.t.shape[1]), water_attenuation_freq1)) /
# (freq1 ** X)) -
# np.full((stg.r.shape[1], stg.t.shape[1]), zeta_freq1) *
# np.log(j_cross_section_freq2 * np.exp(4 * r2D * np.full((stg.r.shape[1], stg.t.shape[1]), water_attenuation_freq2)) /
# (freq2 ** X))) /
# (zeta_freq2 - zeta_freq1))
print("compute VBI finished")
return np.exp(logVBI)
# ------------- Computing SSC fine ------------- #
def SSC_fine(self, zeta, r2D, VBI, freq, X, j_cross_section, alpha_w):
SSC_fine = (1/zeta) * ( 1/(4 * r2D) * np.log((VBI * freq**X) / j_cross_section) - alpha_w)
print("compute SSC fine finished")
return SSC_fine
# ------------- Computing SSC sand ------------- #
def SSC_sand(self, VBI, freq, X, ks):
SSC_sand = (16 * np.pi * VBI * freq ** X) / (3 * ks**2)
print("compute SSC sand finished")
return SSC_sand

View File

@ -25,7 +25,6 @@ from PyQt5.QtWidgets import (QWidget, QVBoxLayout, QDialog, QTabWidget, QGridLay
QFileDialog, QMessageBox, QLabel)
from PyQt5.QtCore import Qt
class CalibrationConstantKt(QDialog):
def __init__(self, parent=None):
@ -113,11 +112,3 @@ class CalibrationConstantKt(QDialog):
eval("self.gridLayout_tab_" + str(t_index) + ".addWidget(self.label_kt_" + str(x) + "_ABS_" + str(t_index) +
", " + str(x+1) + ", 1, 1, 1, Qt.AlignCenter)")
# if __name__ == "__main__":
# app = QApplication(sys.argv)
# cal = CalibrationConstantKt()
# cal.show()
# # sys.exit(app.exec_())
# app.exec()

View File

@ -20,168 +20,199 @@
# -*- coding: utf-8 -*-
import os
import time
import sqlite3
import logging
import numpy as np
from PyQt5.QtWidgets import QFileDialog, QApplication, QMessageBox
import sqlite3
import settings as stg
from os import chdir
import time
from PyQt5.QtWidgets import QFileDialog, QApplication, QMessageBox
import settings as stg
from settings import ABS_name
logger = logging.getLogger("acoused")
class CreateTableForSaveAs:
def __init__(self):
self.create_AcousticFile = """
CREATE TABLE AcousticFile(
ID INTEGER PRIMARY KEY AUTOINCREMENT,
acoustic_data INTEGER,
acoustic_file STRING,
ABS_name STRING,
path_BS_noise_data STRING,
filename_BS_noise_data STRING,
noise_method FLOAT,
noise_value FLOAT,
data_preprocessed STRING
)
"""
self.create_AcousticFile = """CREATE TABLE AcousticFile(
ID INTEGER PRIMARY KEY AUTOINCREMENT,
acoustic_data INTEGER,
acoustic_file STRING,
ABS_name STRING,
path_BS_noise_data STRING,
filename_BS_noise_data STRING,
noise_method FLOAT,
noise_value FLOAT,
data_preprocessed STRING
)
"""
self.create_Measure = """
CREATE TABLE Measure(
ID INTEGER PRIMARY KEY AUTOINCREMENT,
acoustic_data INTEGER,
Date DATE,
Hour TIME,
frequency FLOAT,
sound_attenuation FLOAT,
kt_read FLOAT,
kt_corrected FLOAT,
NbProfiles FLOAT,
NbProfilesPerSeconds FLOAT,
NbCells FLOAT,
CellSize FLOAT,
PulseLength FLOAT,
NbPingsPerSeconds FLOAT,
NbPingsAveragedPerProfile FLOAT,
GainRx FLOAT,
GainTx FLOAT
)
"""
self.create_Measure = """ CREATE TABLE Measure(
ID INTEGER PRIMARY KEY AUTOINCREMENT,
acoustic_data INTEGER,
Date DATE,
Hour TIME,
frequency FLOAT,
sound_attenuation FLOAT,
kt_read FLOAT,
kt_corrected FLOAT,
NbProfiles FLOAT,
NbProfilesPerSeconds FLOAT,
NbCells FLOAT,
CellSize FLOAT,
PulseLength FLOAT,
NbPingsPerSeconds FLOAT,
NbPingsAveragedPerProfile FLOAT,
GainRx FLOAT,
GainTx FLOAT
)
"""
self.create_BSRawData = """
CREATE TABLE BSRawData(
ID INTEGER PRIMARY KEY AUTOINCREMENT,
acoustic_data INTEGER,
time BLOB, depth BLOB, BS_raw_data BLOB,
time_reshape BLOB, depth_reshape BLOB, BS_raw_data_reshape BLOB,
time_cross_section BLOB, depth_cross_section BLOB,
BS_cross_section BLOB, BS_stream_bed BLO B,
depth_bottom, val_bottom, ind_bottom,
time_noise BLOB, depth_noise BLOB, BS_noise_raw_data BLOB,
SNR_raw_data BLOB, SNR_cross_section BLOB, SNR_stream_bed BLOB,
BS_raw_data_pre_process_SNR BLOB,
BS_raw_data_pre_process_average BLOB,
BS_cross_section_pre_process_SNR BLOB,
BS_cross_section_pre_process_average BLOB,
BS_stream_bed_pre_process_SNR BLOB,
BS_stream_bed_pre_process_average BLOB,
BS_mean BLOB
)
"""
self.create_BSRawData = '''CREATE TABLE BSRawData(
ID INTEGER PRIMARY KEY AUTOINCREMENT,
acoustic_data INTEGER,
time BLOB, depth BLOB, BS_raw_data BLOB,
time_reshape BLOB, depth_reshape BLOB, BS_raw_data_reshape BLOB,
time_cross_section BLOB, depth_cross_section BLOB, BS_cross_section BLOB, BS_stream_bed BLOB,
depth_bottom, val_bottom, ind_bottom,
time_noise BLOB, depth_noise BLOB, BS_noise_raw_data BLOB,
SNR_raw_data BLOB, SNR_cross_section BLOB, SNR_stream_bed BLOB,
BS_raw_data_pre_process_SNR BLOB, BS_raw_data_pre_process_average BLOB,
BS_cross_section_pre_process_SNR BLOB, BS_cross_section_pre_process_average BLOB,
BS_stream_bed_pre_process_SNR BLOB, BS_stream_bed_pre_process_average BLOB,
BS_mean BLOB
)'''
self.create_Settings = """
CREATE TABLE Settings(
ID INTEGER PRIMARY KEY AUTOINCREMENT,
acoustic_data INTEGER,
temperature FLOAT,
tmin_index FLOAT, tmin_value FLOAT,
tmax_index FLOAT, tmax_value FLOAT,
rmin_index FLOAT, rmin_value FLOAT,
rmax_index FLOAT, rmax_value FLOAT,
freq_bottom_detection_index FLOAT,
freq_bottom_detection_value STRING,
SNR_filter_value FLOAT, Nb_cells_to_average_BS_signal FLOAT
)
"""
self.create_Settings = '''CREATE TABLE Settings(
ID INTEGER PRIMARY KEY AUTOINCREMENT,
acoustic_data INTEGER,
temperature FLOAT,
tmin_index FLOAT, tmin_value FLOAT, tmax_index FLOAT, tmax_value FLOAT,
rmin_index FLOAT, rmin_value FLOAT, rmax_index FLOAT, rmax_value FLOAT,
freq_bottom_detection_index FLOAT, freq_bottom_detection_value STRING,
SNR_filter_value FLOAT, Nb_cells_to_average_BS_signal FLOAT
)'''
self.create_SedimentsFile = """
CREATE TABLE SedimentsFile(
ID INTEGER PRIMARY KEY AUTOINCREMENT,
path_fine STRING,
filename_fine STRING,
radius_grain_fine BLOB,
path_sand STRING,
filename_sand STRING,
radius_grain_sand BLOB,
time_column_label STRING,
distance_from_bank_column_label STRING,
depth_column_label STRING,
Ctot_fine_column_label STRING,
D50_fine_column_label STRING,
Ctot_sand_column_label STRING,
D50_sand_column_label STRING
)
"""
self.create_SedimentsFile = """CREATE TABLE SedimentsFile(
ID INTEGER PRIMARY KEY AUTOINCREMENT,
path_fine STRING,
filename_fine STRING,
radius_grain_fine BLOB,
path_sand STRING,
filename_sand STRING,
radius_grain_sand BLOB,
time_column_label STRING,
distance_from_bank_column_label STRING,
depth_column_label STRING,
Ctot_fine_column_label STRING,
D50_fine_column_label STRING,
Ctot_sand_column_label STRING,
D50_sand_column_label STRING
)
"""
self.create_SedimentsData = """
CREATE TABLE SedimentsData(
ID INTEGER PRIMARY KEY AUTOINCREMENT,
sample_fine_name STRING,
sample_fine_index INTEGER,
distance_from_bank_fine FLOAT,
depth_fine FLOAT,
time_fine FLOAT,
Ctot_fine FLOAT,
Ctot_fine_per_cent FLOAT,
D50_fine FLOAT,
frac_vol_fine BLOB,
frac_vol_fine_cumul BLOB,
sample_sand_name STRING,
sample_sand_index INTEGER,
distance_from_bank_sand FLOAT,
depth_sand FLOAT,
time_sand FLOAT,
Ctot_sand FLOAT,
Ctot_sand_per_cent FLOAT,
D50_sand FLOAT,
frac_vol_sand BLOB,
frac_vol_sand_cumul BLOB
)
"""
self.create_SedimentsData = """CREATE TABLE SedimentsData(
ID INTEGER PRIMARY KEY AUTOINCREMENT,
sample_fine_name STRING,
sample_fine_index INTEGER,
distance_from_bank_fine FLOAT,
depth_fine FLOAT,
time_fine FLOAT,
Ctot_fine FLOAT,
Ctot_fine_per_cent FLOAT,
D50_fine FLOAT,
frac_vol_fine BLOB,
frac_vol_fine_cumul BLOB,
sample_sand_name STRING,
sample_sand_index INTEGER,
distance_from_bank_sand FLOAT,
depth_sand FLOAT,
time_sand FLOAT,
Ctot_sand FLOAT,
Ctot_sand_per_cent FLOAT,
D50_sand FLOAT,
frac_vol_sand BLOB,
frac_vol_sand_cumul BLOB
)
"""
self.create_Calibration = """
CREATE TABLE Calibration(
ID INTEGER PRIMARY KEY AUTOINCREMENT,
path_calibration_file STRING,
filename_calibration_file STRING,
range_lin_interp BLOB,
M_profile_fine BLOB,
ks BLOB,
sv BLOB,
X_exponent BLOB,
alpha_s BLOB,
zeta BLOB,
FCB BLOB,
depth_real BLOB,
lin_reg BLOB
)
"""
self.create_Calibration = """CREATE TABLE Calibration(
ID INTEGER PRIMARY KEY AUTOINCREMENT,
path_calibration_file STRING,
filename_calibration_file STRING,
range_lin_interp BLOB,
M_profile_fine BLOB,
ks BLOB,
sv BLOB,
X_exponent BLOB,
alpha_s BLOB,
zeta BLOB,
FCB BLOB,
depth_real BLOB,
lin_reg BLOB
)"""
self.create_Inversion = """CREATE TABLE Inversion(
ID INTEGER PRIMARY KEY AUTOINCREMENT,
J_cross_section_freq1 BLOB,
J_cross_section_freq2 BLOB,
VBI_cross_section BLOB,
SSC_fine BLOB,
SSC_sand BLOB
)"""
self.create_Inversion = """
CREATE TABLE Inversion(
ID INTEGER PRIMARY KEY AUTOINCREMENT,
J_cross_section_freq1 BLOB,
J_cross_section_freq2 BLOB,
VBI_cross_section BLOB,
SSC_fine BLOB,
SSC_sand BLOB
)
"""
self.open_file_dialog()
def open_file_dialog(self):
options = QFileDialog.Options()
name = QFileDialog.getSaveFileName(
caption="Save As", directory="", filter="AcouSed Files (*.acd)", options=QFileDialog.DontUseNativeDialog)
name, _ = QFileDialog.getSaveFileName(
caption="Save As",
directory="",
filter="AcouSed Files (*.acd)",
options=QFileDialog.DontUseNativeDialog
)
if name[0]:
if name != "":
filename = os.path.basename(name)
if os.path.splitext(filename)[1] != ".acd":
filename += ".acd"
stg.dirname_save_as = "/".join(name[0].split("/")[:-1]) + "/"
stg.filename_save_as = name[0].split("/")[-1]
logger.debug(f"selected save file: '{filename}'")
chdir(stg.dirname_save_as)
stg.dirname_save_as = os.path.dirname(name)
stg.filename_save_as = filename
try:
os.chdir(stg.dirname_save_as)
except OSError as e:
logger.warning(f"chdir: {str(e)}")
start = time.time()
self.create_table()
print(f"end : {time.time() - start} sec")
else:
msgBox = QMessageBox()
msgBox.setWindowTitle("Save Error")
msgBox.setIcon(QMessageBox.Warning)
@ -190,18 +221,24 @@ class CreateTableForSaveAs:
msgBox.exec()
def create_table(self):
# Create a new database and open a database connection to allow sqlite3 to work with it.
cnx = sqlite3.connect(stg.filename_save_as + '.acd')
# Create database cursor to execute SQL statements and fetch results from SQL queries.
cnx = sqlite3.connect(stg.filename_save_as)
cur = cnx.cursor()
# --------------------------------------------------------------------------------------------------------------
# +++++++++++++++++++++++++++
# --- Table Acoustic File ---
# +++++++++++++++++++++++++++
self.create_table_acoustic_file(cnx, cur)
self.create_table_measure(cnx, cur)
self.create_table_BSRawData(cnx, cur)
self.create_table_settings(cnx, cur)
self.create_table_sediments_file(cnx, cur)
self.create_table_sediments_data(cnx, cur)
self.create_table_calibration(cnx, cur)
self.create_table_inversion(cnx, cur)
cnx.commit()
cur.close()
cnx.close()
def create_table_acoustic_file(self, cnx, cur):
start_table_File = time.time()
cur.execute("DROP TABLE if exists AcousticFile")
@ -209,28 +246,42 @@ class CreateTableForSaveAs:
cur.execute(self.create_AcousticFile)
for i in stg.acoustic_data:
print("stg.acoustic_data ", stg.acoustic_data[i])
print("stg.filename_BS_raw_data ", stg.filename_BS_raw_data[i])
print('stg.ABS_name', stg.ABS_name)
print("stg.path_BS_raw_data ", stg.path_BS_raw_data[i])
logger.debug(f"stg.acoustic_data: {stg.acoustic_data[i]}")
logger.debug("stg.filename_BS_raw_data: "
+ f"{stg.filename_BS_raw_data[i]}")
logger.debug(f"stg.ABS_name: {stg.ABS_name}")
logger.debug(f"stg.path_BS_raw_data: {stg.path_BS_raw_data[i]}")
cur.execute(''' INSERT into AcousticFile(acoustic_data, acoustic_file, ABS_name, path_BS_noise_data,
filename_BS_noise_data, noise_method, noise_value, data_preprocessed)
VALUES(?, ?, ?, ?, ?, ?, ?, ?)''',
(stg.acoustic_data[i], stg.filename_BS_raw_data[i].split('.')[0], stg.ABS_name[i],
stg.path_BS_noise_data[i], stg.filename_BS_noise_data[i], stg.noise_method[i],
stg.noise_value[i], stg.data_preprocessed[i])
)
cur.execute(
"""
INSERT into AcousticFile(
acoustic_data,
acoustic_file,
ABS_name,
path_BS_noise_data,
filename_BS_noise_data,
noise_method,
noise_value,
data_preprocessed)
VALUES(?, ?, ?, ?, ?, ?, ?, ?)
""",
(
stg.acoustic_data[i],
stg.filename_BS_raw_data[i].split('.')[0],
stg.ABS_name[i],
stg.path_BS_noise_data[i],
stg.filename_BS_noise_data[i],
stg.noise_method[i],
stg.noise_value[i],
stg.data_preprocessed[i]
)
)
cnx.commit()
print(f"table File : {time.time() - start_table_File} sec")
# --------------------------------------------------------------------------------------------------------------
# +++++++++++++++++++++
# --- Table Measure ---
# +++++++++++++++++++++
logger.info(f"table File : {time.time() - start_table_File} sec")
def create_table_measure(self, cnx, cur):
start_table_Measure = time.time()
# Drop Table if exists
@ -238,35 +289,52 @@ class CreateTableForSaveAs:
# Execute the CREATE TABLE statement
cur.execute(self.create_Measure)
print("stg.date ", stg.date, "stg.hour ", stg.hour)
# Fill the table Measure
logger.debug(f"stg.date: {stg.date}, stg.hour: {stg.hour}")
for i in stg.acoustic_data:
for j in range(stg.freq[i].shape[0]):
cur.execute(''' INSERT into Measure(acoustic_data, Date, Hour, frequency, sound_attenuation, kt_read, kt_corrected,
NbProfiles, NbProfilesPerSeconds, NbCells, CellSize, PulseLength,
NbPingsPerSeconds, NbPingsAveragedPerProfile, GainRx, GainTx
)
VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)''',
(stg.acoustic_data[i], #stg.date[i], stg.hour[i],
str(stg.date[i].year) + str('-') + str(stg.date[i].month) + str('-') + str(stg.date[i].day),
str(stg.hour[i].hour) + str(':') + str(stg.hour[i].minute),
stg.freq[i][j], stg.water_attenuation[i][j], stg.kt_read[j], stg.kt_corrected[j],
stg.nb_profiles[i][j], stg.nb_profiles_per_sec[i][j], stg.nb_cells[i][j],
stg.cell_size[i][j], stg.pulse_length[i][j], stg.nb_pings_per_sec[i][j],
stg.nb_pings_averaged_per_profile[i][j], stg.gain_rx[i][j], stg.gain_tx[i][j]))
cur.execute(
"""
INSERT into Measure(
acoustic_data,
Date, Hour,
frequency,
sound_attenuation,
kt_read, kt_corrected,
NbProfiles, NbProfilesPerSeconds,
NbCells, CellSize,
PulseLength,
NbPingsPerSeconds,
NbPingsAveragedPerProfile,
GainRx, GainTx
)
VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
(
stg.acoustic_data[i], #stg.date[i], stg.hour[i],
str(stg.date[i].year) + str('-')
+ str(stg.date[i].month) + str('-')
+ str(stg.date[i].day),
str(stg.hour[i].hour) + str(':') + str(stg.hour[i].minute),
stg.freq[i][j],
stg.water_attenuation[i][j],
stg.kt_read[j], stg.kt_corrected[j],
stg.nb_profiles[i][j], stg.nb_profiles_per_sec[i][j],
stg.nb_cells[i][j], stg.cell_size[i][j],
stg.pulse_length[i][j],
stg.nb_pings_per_sec[i][j],
stg.nb_pings_averaged_per_profile[i][j],
stg.gain_rx[i][j], stg.gain_tx[i][j]
)
)
# Commit the transaction after executing INSERT.
cnx.commit()
print(f"table Measure : {time.time() - start_table_Measure} sec")
# --------------------------------------------------------------------------------------------------------------
# +++++++++++++++++++++++++
# --- Table BSRawData_i ---
# +++++++++++++++++++++++++
logger.info(f"table Measure : {time.time() - start_table_Measure} sec")
def create_table_BSRawData(self, cnx, cur):
start_table_BSRawData = time.time()
cur.execute('DROP TABLE if exists BSRawData')
@ -275,105 +343,136 @@ class CreateTableForSaveAs:
cur.execute(self.create_BSRawData)
for i in stg.acoustic_data:
cur.execute(
"""
INSERT into BSRawData(
acoustic_data,
time, depth,
BS_raw_data,
time_reshape,
depth_reshape,
BS_raw_data_reshape,
time_cross_section, depth_cross_section,
BS_cross_section, BS_stream_bed,
depth_bottom, val_bottom, ind_bottom,
time_noise, depth_noise, BS_noise_raw_data,
SNR_raw_data, SNR_cross_section, SNR_stream_bed,
BS_raw_data_pre_process_SNR, BS_raw_data_pre_process_average,
BS_cross_section_pre_process_SNR, BS_cross_section_pre_process_average,
BS_stream_bed_pre_process_SNR, BS_stream_bed_pre_process_average,
BS_mean
)
VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
?, ?, ?, ?, ?, ?)
""",
(
stg.acoustic_data[i], stg.time[i].tobytes(),
stg.depth[i].tobytes(), stg.BS_raw_data[i].tobytes(),
stg.time_reshape[i].tobytes(), stg.depth_reshape[i].tobytes(),
stg.BS_raw_data_reshape[i].tobytes(),
stg.time_cross_section[i].tobytes(),
stg.depth_cross_section[i].tobytes(),
stg.BS_cross_section[i].tobytes(), stg.BS_stream_bed[i].tobytes(),
stg.depth_bottom[i].tobytes(), np.array(stg.val_bottom[i]).tobytes(),
np.array(stg.ind_bottom[i]).tobytes(),
stg.time_noise[i].tobytes(), stg.depth_noise[i].tobytes(),
stg.BS_noise_raw_data[i].tobytes(),
stg.SNR_raw_data[i].tobytes(), stg.SNR_cross_section[i].tobytes(),
stg.SNR_stream_bed[i].tobytes(),
stg.BS_raw_data_pre_process_SNR[i].tobytes(),
stg.BS_raw_data_pre_process_average[i].tobytes(),
stg.BS_cross_section_pre_process_SNR[i].tobytes(),
stg.BS_cross_section_pre_process_average[i].tobytes(),
stg.BS_stream_bed_pre_process_SNR[i].tobytes(),
stg.BS_stream_bed_pre_process_average[i].tobytes(),
stg.BS_mean[i].tobytes()
)
)
cur.execute(''' INSERT into BSRawData(acoustic_data, time, depth, BS_raw_data,
time_reshape, depth_reshape, BS_raw_data_reshape,
time_cross_section, depth_cross_section,
BS_cross_section, BS_stream_bed,
depth_bottom, val_bottom, ind_bottom,
time_noise, depth_noise, BS_noise_raw_data,
SNR_raw_data, SNR_cross_section, SNR_stream_bed,
BS_raw_data_pre_process_SNR, BS_raw_data_pre_process_average,
BS_cross_section_pre_process_SNR, BS_cross_section_pre_process_average,
BS_stream_bed_pre_process_SNR, BS_stream_bed_pre_process_average,
BS_mean)
VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)''',
(stg.acoustic_data[i], stg.time[i].tobytes(),
stg.depth[i].tobytes(), stg.BS_raw_data[i].tobytes(),
stg.time_reshape[i].tobytes(), stg.depth_reshape[i].tobytes(), stg.BS_raw_data_reshape[i].tobytes(),
stg.time_cross_section[i].tobytes(), stg.depth_cross_section[i].tobytes(),
stg.BS_cross_section[i].tobytes(), stg.BS_stream_bed[i].tobytes(),
stg.depth_bottom[i].tobytes(), np.array(stg.val_bottom[i]).tobytes(), np.array(stg.ind_bottom[i]).tobytes(),
stg.time_noise[i].tobytes(), stg.depth_noise[i].tobytes(), stg.BS_noise_raw_data[i].tobytes(),
stg.SNR_raw_data[i].tobytes(), stg.SNR_cross_section[i].tobytes(), stg.SNR_stream_bed[i].tobytes(),
stg.BS_raw_data_pre_process_SNR[i].tobytes(), stg.BS_raw_data_pre_process_average[i].tobytes(),
stg.BS_cross_section_pre_process_SNR[i].tobytes(), stg.BS_cross_section_pre_process_average[i].tobytes(),
stg.BS_stream_bed_pre_process_SNR[i].tobytes(), stg.BS_stream_bed_pre_process_average[i].tobytes(),
stg.BS_mean[i].tobytes()
)
)
print("stg.ind_bottom ", stg.ind_bottom[i])
print(np.array([stg.ind_bottom[i]]), np.array(stg.ind_bottom[i]).shape)
# Commit the transaction after executing INSERT.
# Commit the transaction after executing INSERT.
cnx.commit()
print(f"table BSRawData : {time.time() - start_table_BSRawData} sec")
# --------------------------------------------------------------------------------------------------------------
# ++++++++++++++++++++++
# --- Table Settings ---
# ++++++++++++++++++++++
logger.info(f"table BSRawData : {time.time() - start_table_BSRawData} sec")
def create_table_settings(self, cnx, cur):
start_table_Settings = time.time()
cur.execute("DROP TABLE if exists Settings")
cur.execute(self.create_Settings)
print(stg.acoustic_data, stg.temperature, stg.rmin, stg.rmax, stg.tmin, stg.tmax)
logger.debug(f"acoustic_data: {stg.acoustic_data}")
logger.debug(f"temperature: {stg.temperature}")
logger.debug(f"rmin: {stg.rmin}, rmax: {stg.rmax}")
logger.debug(f"tmin: {stg.tmin}, tmax: {stg.tmax}")
for i in stg.acoustic_data:
cur.execute('''INSERT into Settings(acoustic_data, temperature,
tmin_index, tmin_value, tmax_index, tmax_value,
rmin_index, rmin_value, rmax_index, rmax_value,
freq_bottom_detection_index, freq_bottom_detection_value,
SNR_filter_value, Nb_cells_to_average_BS_signal
)
VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)''',
(stg.acoustic_data[i], stg.temperature,
stg.tmin[i][0], stg.tmin[i][1], stg.tmax[i][0], stg.tmax[i][1],
stg.rmin[i][0], stg.rmin[i][1], stg.rmax[i][0], stg.rmax[i][1],
stg.freq_bottom_detection[i][0], stg.freq_bottom_detection[i][1],
stg.SNR_filter_value[i], stg.Nb_cells_to_average_BS_signal[i]
)
)
cur.execute(
"""
INSERT into Settings(
acoustic_data, temperature,
tmin_index, tmin_value, tmax_index, tmax_value,
rmin_index, rmin_value, rmax_index, rmax_value,
freq_bottom_detection_index, freq_bottom_detection_value,
SNR_filter_value, Nb_cells_to_average_BS_signal
)
VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
(
stg.acoustic_data[i], stg.temperature,
stg.tmin[i][0], stg.tmin[i][1],
stg.tmax[i][0], stg.tmax[i][1],
stg.rmin[i][0], stg.rmin[i][1],
stg.rmax[i][0], stg.rmax[i][1],
stg.freq_bottom_detection[i][0],
stg.freq_bottom_detection[i][1],
stg.SNR_filter_value[i],
stg.Nb_cells_to_average_BS_signal[i]
)
)
cnx.commit()
print(f"table Settings : {time.time() - start_table_Settings} sec")
# --------------------------------------------------------------------------------------------------------------
# ++++++++++++++++++++++++++++
# --- Table Sediments File ---
# ++++++++++++++++++++++++++++
logger.info(f"table Settings : {time.time() - start_table_Settings} sec")
def create_table_sediments_file(self, cnx, cur):
start_table_SedimentsFile = time.time()
cur.execute("DROP TABLE if exists SedimentsFile")
cur.execute(self.create_SedimentsFile)
cur.execute('''INSERT into SedimentsFile(path_fine, filename_fine, radius_grain_fine,
path_sand, filename_sand, radius_grain_sand,
time_column_label, distance_from_bank_column_label,
depth_column_label, Ctot_fine_column_label, D50_fine_column_label,
Ctot_sand_column_label, D50_sand_column_label)
VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)''',
(stg.path_fine, stg.filename_fine, stg.radius_grain_fine.tobytes(),
stg.path_sand, stg.filename_sand, stg.radius_grain_sand.tobytes(),
stg.columns_fine[0], stg.columns_fine[1], stg.columns_fine[2],
stg.columns_fine[3], stg.columns_fine[4], stg.columns_sand[3], stg.columns_sand[4]))
if stg.path_fine != "" and stg.path_sand != "":
cur.execute(
"""
INSERT into SedimentsFile(
path_fine, filename_fine, radius_grain_fine,
path_sand, filename_sand, radius_grain_sand,
time_column_label, distance_from_bank_column_label,
depth_column_label, Ctot_fine_column_label,
D50_fine_column_label,
Ctot_sand_column_label, D50_sand_column_label
)
VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
(
stg.path_fine, stg.filename_fine,
stg.radius_grain_fine.tobytes(),
stg.path_sand, stg.filename_sand,
stg.radius_grain_sand.tobytes(),
stg.columns_fine[0], stg.columns_fine[1],
stg.columns_fine[2], stg.columns_fine[3],
stg.columns_fine[4],
stg.columns_sand[3], stg.columns_sand[4]
)
)
cnx.commit()
print(f"table SedimentsFile : {time.time() - start_table_SedimentsFile} sec")
logger.info(f"table SedimentsFile : {time.time() - start_table_SedimentsFile} sec")
# --------------------------------------------------------------------------------------------------------------
# ++++++++++++++++++++++++++++
# --- Table Sediments Data ---
# ++++++++++++++++++++++++++++
def create_table_sediments_data(self, cnx, cur):
start_table_SedimentsData = time.time()
cur.execute("DROP TABLE if exists SedimentsData")
@ -381,59 +480,79 @@ class CreateTableForSaveAs:
cur.execute(self.create_SedimentsData)
for f in range(len(stg.sample_fine)):
cur.execute('''INSERT into SedimentsData(sample_fine_name, sample_fine_index, distance_from_bank_fine,
depth_fine, time_fine, Ctot_fine, Ctot_fine_per_cent, D50_fine,
frac_vol_fine, frac_vol_fine_cumul,
sample_sand_name, sample_sand_index, distance_from_bank_sand,
depth_sand, time_sand, Ctot_sand, Ctot_sand_per_cent, D50_sand,
frac_vol_sand, frac_vol_sand_cumul
)
VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)''',
(stg.sample_fine[f][0] , stg.sample_fine[f][1],
stg.distance_from_bank_fine[f], stg.depth_fine[f], stg.time_fine[f], stg.Ctot_fine[f],
stg.Ctot_fine_per_cent[f], stg.D50_fine[f],
stg.frac_vol_fine[f].tobytes(), stg.frac_vol_fine_cumul[f].tobytes(),
stg.sample_sand[f][0], stg.sample_sand[f][1],
stg.distance_from_bank_sand[f], stg.depth_sand[f], stg.time_sand[f], stg.Ctot_sand[f],
stg.Ctot_sand_per_cent[f], stg.D50_sand[f],
stg.frac_vol_sand[f].tobytes(), stg.frac_vol_sand_cumul[f].tobytes()))
cur.execute(
"""
INSERT into SedimentsData(
sample_fine_name, sample_fine_index,
distance_from_bank_fine,
depth_fine, time_fine, Ctot_fine,
Ctot_fine_per_cent, D50_fine,
frac_vol_fine, frac_vol_fine_cumul,
sample_sand_name, sample_sand_index,
distance_from_bank_sand,
depth_sand, time_sand, Ctot_sand,
Ctot_sand_per_cent, D50_sand,
frac_vol_sand, frac_vol_sand_cumul
)
VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
(
stg.sample_fine[f][0] , stg.sample_fine[f][1],
stg.distance_from_bank_fine[f], stg.depth_fine[f],
stg.time_fine[f], stg.Ctot_fine[f],
stg.Ctot_fine_per_cent[f], stg.D50_fine[f],
stg.frac_vol_fine[f].tobytes(),
stg.frac_vol_fine_cumul[f].tobytes(),
stg.sample_sand[f][0], stg.sample_sand[f][1],
stg.distance_from_bank_sand[f], stg.depth_sand[f],
stg.time_sand[f], stg.Ctot_sand[f],
stg.Ctot_sand_per_cent[f], stg.D50_sand[f],
stg.frac_vol_sand[f].tobytes(),
stg.frac_vol_sand_cumul[f].tobytes()
)
)
cnx.commit()
print(f"table SedimentsData : {time.time() - start_table_SedimentsData} sec")
# --------------------------------------------------------------------------------------------------------------
# ++++++++++++++++++++++++++++++
# --- Table Calibration ---
# ++++++++++++++++++++++++++++++
logger.info(f"table SedimentsData : {time.time() - start_table_SedimentsData} sec")
def create_table_calibration(self, cnx, cur):
start_table_Calibration = time.time()
cur.execute("DROP TABLE if exists Calibration")
cur.execute(self.create_Calibration)
cur.execute('''INSERT into Calibration(path_calibration_file, filename_calibration_file,
range_lin_interp, M_profile_fine,
ks, sv, X_exponent, alpha_s, zeta,
FCB, depth_real, lin_reg)
VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)''',
(stg.path_calibration_file, stg.filename_calibration_file,
stg.range_lin_interp.tobytes(), stg.M_profile_fine.tobytes(),
np.array(stg.ks).tobytes(), np.array(stg.sv).tobytes(), np.array(stg.X_exponent).tobytes(),
np.array(stg.alpha_s).tobytes(), np.array(stg.zeta).tobytes(),
stg.FCB.tobytes(), stg.depth_real.tobytes(), np.array(stg.lin_reg).tobytes())
)
if len(stg.range_lin_interp) != 0:
cur.execute(
"""
INSERT into Calibration(
path_calibration_file, filename_calibration_file,
range_lin_interp, M_profile_fine,
ks, sv, X_exponent, alpha_s, zeta,
FCB, depth_real, lin_reg
)
VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
(
stg.path_calibration_file, stg.filename_calibration_file,
stg.range_lin_interp.tobytes(),
stg.M_profile_fine.tobytes(),
np.array(stg.ks).tobytes(), np.array(stg.sv).tobytes(),
np.array(stg.X_exponent).tobytes(),
np.array(stg.alpha_s).tobytes(),
np.array(stg.zeta).tobytes(),
stg.FCB.tobytes(), stg.depth_real.tobytes(),
np.array(stg.lin_reg).tobytes()
)
)
cnx.commit()
print(f"table Calibration : {time.time() - start_table_Calibration} sec")
# --------------------------------------------------------------------------------------------------------------
# ++++++++++++++++++++++++++++++
# --- Table Inversion ---
# ++++++++++++++++++++++++++++++
logger.info(f"table Calibration : {time.time() - start_table_Calibration} sec")
def create_table_inversion(self, cnx, cur):
start_table_Inversion = time.time()
cur.execute("DROP TABLE if exists Inversion")
@ -441,24 +560,23 @@ class CreateTableForSaveAs:
cur.execute(self.create_Inversion)
for i in range(len(stg.SSC_fine)):
cur.execute('''INSERT into Inversion(J_cross_section_freq1, J_cross_section_freq2,
VBI_cross_section, SSC_fine, SSC_sand)
VALUES(?, ?, ?, ?, ?)''',
(stg.J_cross_section[i][0].tobytes(), stg.J_cross_section[i][1].tobytes(),
stg.VBI_cross_section[i].tobytes(), stg.SSC_fine[i].tobytes(), stg.SSC_sand[i].tobytes())
)
cur.execute(
"""
INSERT into Inversion(
J_cross_section_freq1, J_cross_section_freq2,
VBI_cross_section, SSC_fine, SSC_sand
)
VALUES(?, ?, ?, ?, ?)
""",
(
stg.J_cross_section[i][0].tobytes(),
stg.J_cross_section[i][1].tobytes(),
stg.VBI_cross_section[i].tobytes(),
stg.SSC_fine[i].tobytes(),
stg.SSC_sand[i].tobytes()
)
)
cnx.commit()
print(f"table Inversion : {time.time() - start_table_Inversion} sec")
# --------------------------------------------------------------------------------------------------------------
# Close database cursor
cur.close()
# Close database connection
cnx.close()
logger.info(f"table Inversion : {time.time() - start_table_Inversion} sec")

View File

@ -1,7 +1,5 @@
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from Model.GrainSizeTools import demodul_granulo, mix_gaussian_model
class GranuloLoader:
@ -16,8 +14,7 @@ class GranuloLoader:
self._y = self._data.iloc[:, 1].tolist() # distance from left bank (m)
self._z = self._data.iloc[:, 2].tolist() # depth (m)
self._r_grain = 1e-6 * np.array(self._data.columns.values)[5:].astype(float) / 2
# self._r_grain = 1e-6 * np.array(self._data.columns.values)[5:].astype(float) / 2 # grain radius (m)
self._r_grain = 1e-6 * np.array(self._data.columns.values)[5:].astype(float) / 2 # grain radius (m)
self._Ctot = self._data.iloc[:, 3].tolist() # Total concentration (g/L)
self._D50 = self._data.iloc[:, 4].tolist() # median diameter (um)
@ -25,147 +22,3 @@ class GranuloLoader:
self._frac_vol_cumul = np.cumsum(self._frac_vol, axis=1) # Cumulated volume fraction (%)
# print(type(self._frac_vol_cumul), self._frac_vol_cumul.shape, self._frac_vol_cumul)
# # --- Load sand sediments data file ---
# self.path_sand = path_sand
# self._data_sand = pd.read_excel(self.path_sand, engine="odf", header=0)
#
# self._Ctot_sand = np.array(self._data_sand.iloc[:, 2]) # Total concentration (g/L)
# self._D50_sand = np.array(self._data_sand.iloc[:, 3]) # median diameter (um)
# self._frac_vol_sand = np.array(self._data_sand.iloc[:, 4:]) # Volume fraction (%)
#
# self._frac_vol_sand_cumul = np.cumsum(self._frac_vol_sand, axis=1) # Cumulated volume fraction (%)
#
# # --- Compute % of fine and % of sand sediment in total concentration ---
#
# self._Ctot_fine_per_cent = 100 * self._Ctot_fine / (self._Ctot_fine + self._Ctot_sand)
# self._Ctot_sand_per_cent = 100 * self._Ctot_sand / (self._Ctot_fine + self._Ctot_sand)
# ==============================================================================================================
# ==============================================================================================================
# N_sample = 0
# #
# fig, ax = plt.subplots(1, 2)
# ax[0].plot(self._r_grain, self._frac_vol[N_sample, :], color="k", marker='.')
# ax[0].set_xscale('log')
# ax[0].set_xlabel('Radius ($\mu m$)')
# ax[0].set_ylabel('Class size volume fraction')
#
# ax[1].plot([self._r_grain[i+1]-self._r_grain[i] for i in range(self._r_grain.shape[0]-1)], list(range(self._r_grain.shape[0]-1)), color="k", marker="x")
# ax[1].set_xlabel('Ecart inter-class')
# ax[1].set_ylabel('n° échantillon')
#
# plt.show()
#
# print(f"self._r_grain.shape : {self._r_grain.shape}")
# print(f"self._r_grain : {self._r_grain}")
# print(f"self._frac_vol_cumul.shape : {self._frac_vol_cumul[N_sample, :].shape}")
# print(f"self._frac_vol_cumul[N_sample, :] : {self._frac_vol_cumul[N_sample, :]}")
# print(np.where(self._frac_vol_cumul[N_sample, :] > 0))
# #
# min_demodul = 1e-6
# max_demodul = 500e-6
# sample_demodul = demodul_granulo(self._r_grain[:],
# self._frac_vol_cumul[N_sample, :],
# min_demodul, max_demodul)
#
# print(f"sample_demodul : {sample_demodul.demodul_data_list}")
# N_modes = 3
# sample_demodul.print_mode_data(N_modes)
# sample_demodul.plot_interpolation()
# sample_demodul.plot_modes(N_modes)
#
# print(f"mu_list : {sample_demodul.demodul_data_list[3 - 1].mu_list}")
# print(f"sigma_list : {sample_demodul.demodul_data_list[3 - 1].sigma_list}")
# print(f"w_list : {sample_demodul.demodul_data_list[3 - 1].w_list}")
#
# resampled_log_array = np.log(np.logspace(-10, -2, 3000))
# proba_vol_demodul = mix_gaussian_model(resampled_log_array,
# sample_demodul.demodul_data_list[2].mu_list,
# sample_demodul.demodul_data_list[2].sigma_list,
# sample_demodul.demodul_data_list[2].w_list)
#
#
#
# proba_vol_demodul = proba_vol_demodul / np.sum(proba_vol_demodul)
# ss = np.sum(proba_vol_demodul / np.exp(resampled_log_array) ** 3)
# proba_num = proba_vol_demodul / np.exp(resampled_log_array) ** 3 / ss
#
# print(f"proba_num : {proba_num}")
# freq = 5e6
#
# a2f2pdf = 0
# a3pdf = 0
# for i in range(len(resampled_log_array)):
# a = np.exp(resampled_log_array)[i]
# a2f2pdf += a**2 * form_factor_function_MoateThorne2012(a, freq)**2 * proba_num[i]
# a3pdf += a**3 * proba_num[i]
#
# print(f"a2f2pdf = {a2f2pdf}")
# print(f"a3pdf = {a3pdf}")
#
# ks = (a2f2pdf / a3pdf)
#
# print(f"ks = {ks}")
#
#
# def form_factor_function_MoateThorne2012(a_s, freq, C=1500):
# """This function computes the form factor based on the equation of
# Moate and Thorne (2012)"""
# # computing the wave number
# k = 2 * np.pi * freq / C
# x = k * a_s
# f = (x ** 2 * (1 - 0.25 * np.exp(-((x - 1.5) / 0.35) ** 2)) * (
# 1 + 0.6 * np.exp(-((x - 2.9) / 1.15) ** 2))) / (
# 42 + 28 * x ** 2)
# return f
# if __name__ == "__main__":
# GranuloLoader("/home/bmoudjed/Documents/2 Data/Confluence_Rhône_Isere_2018/Granulo_data/fine_sample_file.ods")
# GranuloLoader("/home/bmoudjed/Documents/3 SSC acoustic meas project/Graphical interface project/Data/Granulo_data/"
# "sand_sample_file.ods")
# GranuloLoader("/home/bmoudjed/Documents/3 SSC acoustic meas project/Graphical interface project/Data/"
# "pt_acoused_cnr_7nov2023/echantil/fine_new_file_acoused_101RDS 3.ods")
# GranuloLoader("/home/bmoudjed/Documents/3 SSC acoustic meas project/Graphical interface project/Data/"
# "pt_acoused_cnr_7nov2023/echantil/sand_new_file_acoused_101RDS 3.ods")
# # form_factor = form_factor_function_MoateThorne2012(GranuloLoader.)
#
# def ks(a_s, rho_s, freq, pdf):
# # --- Calcul de la fonction de form ---
# form_factor = form_factor_function_MoateThorne2012(a_s, freq)
# print(f"form_factor shape = {len(form_factor)}")
# # print(f"form_factor = {form_factor}")
#
# # --- Gaussian mixture ---
# # sample_demodul = demodul_granulo(a_s.astype(float), pdf, 0.17e-6, 200e-6)
# # sample_demodul.plot_interpolation()
#
# # ss = np.sum(pdf / a_s ** 3)
# # proba_num = (pdf / a_s ** 3) / ss
#
# # --- Compute k_s by dividing two integrals ---
# a2f2pdf = 0
# a3pdf = 0
# for i in range(len(pdf)):
# a2f2pdf += a_s[i] ** 2 * form_factor[i] * proba_num[i]
# a3pdf += a_s[i] ** 3 * proba_num[i]
#
# ks = np.sqrt(a2f2pdf / (rho_s * a3pdf))
#
# # ks = np.array([0.04452077, 0.11415143, 0.35533713, 2.47960051])
# # ks = ks0[ind]
# return ks

View File

@ -20,37 +20,50 @@
# -*- coding: utf-8 -*-
import os
import sys
import numpy as np
from PyQt5.QtWidgets import QFileDialog, QApplication, QWidget, QTabWidget
import sqlite3
from os import path, chdir
import logging
import numpy as np
from PyQt5.QtWidgets import QFileDialog, QApplication, QWidget, QTabWidget
import settings as stg
from settings import BS_raw_data, acoustic_data
from View.acoustic_data_tab import AcousticDataTab
logger = logging.getLogger("acoused")
class ReadTableForOpen:
def __init__(self):
self.opened = False
pass
self.open_file_dialog()
def open_file_dialog(self):
name, _ = QFileDialog.getOpenFileName(
caption="Open Acoused file",
directory="",
filter="Acoused file (*.acd)",
options=QFileDialog.DontUseNativeDialog
)
name = QFileDialog.getOpenFileName(caption="Open Acoused file", directory="", filter="Acoused file (*.acd)",
options=QFileDialog.DontUseNativeDialog)
if name != "":
stg.dirname_open = os.path.dirname(name)
stg.filename_open = os.path.basename(name)
if name:
try:
os.chdir(stg.dirname_open)
except OSError as e:
logger.warning(f"chdir: {str(e)}")
stg.dirname_open = path.dirname(name[0])
stg.filename_open = path.basename(name[0])
chdir(stg.dirname_open)
self.sql_file_to_open = open(stg.filename_open)
self.read_table()
self.opened = True
def read_table(self):
@ -76,7 +89,7 @@ class ReadTableForOpen:
for k in range(len(stg.acoustic_data)):
print("hello")
query = f'''SELECT acoustic_data, acoustic_file, ABS_name, path_BS_noise_data, filename_BS_noise_data,
query = f'''SELECT acoustic_data, acoustic_file, ABS_name, path_BS_noise_data, filename_BS_noise_data,
noise_method, noise_value, data_preprocessed FROM AcousticFile WHERE (acoustic_data = {k})'''
data = cur.execute(query).fetchall()
print("data acoustic file", data)
@ -101,8 +114,8 @@ class ReadTableForOpen:
stg.hour = [0]*len(stg.acoustic_data)
for i in range(len(stg.acoustic_data)):
print("i = ", i)
query1 = f'''SELECT acoustic_data, Date, Hour, frequency, sound_attenuation, kt_read, kt_corrected, NbProfiles,
NbProfilesPerSeconds, NbCells, CellSize, PulseLength, NbPingsPerSeconds, NbPingsAveragedPerProfile,
query1 = f'''SELECT acoustic_data, Date, Hour, frequency, sound_attenuation, kt_read, kt_corrected, NbProfiles,
NbProfilesPerSeconds, NbCells, CellSize, PulseLength, NbPingsPerSeconds, NbPingsAveragedPerProfile,
GainRx, GainTx FROM Measure WHERE (acoustic_data = {i})'''
data1 = cur.execute(query1).fetchall()
@ -140,16 +153,16 @@ class ReadTableForOpen:
print("len stg.acoustic_data ", len(stg.acoustic_data))
for j in range(len(stg.acoustic_data)):
print(f"j = {j}")
query2 = f'''SELECT acoustic_data, time, depth, BS_raw_data,
query2 = f'''SELECT acoustic_data, time, depth, BS_raw_data,
time_reshape, depth_reshape, BS_raw_data_reshape,
time_cross_section, depth_cross_section,
BS_cross_section, BS_stream_bed,
time_cross_section, depth_cross_section,
BS_cross_section, BS_stream_bed,
depth_bottom, val_bottom, ind_bottom,
time_noise, depth_noise, BS_noise_raw_data,
time_noise, depth_noise, BS_noise_raw_data,
SNR_raw_data, SNR_cross_section, SNR_stream_bed,
BS_raw_data_pre_process_SNR, BS_raw_data_pre_process_average,
BS_cross_section_pre_process_SNR, BS_cross_section_pre_process_average,
BS_stream_bed_pre_process_SNR, BS_stream_bed_pre_process_average, BS_mean
BS_raw_data_pre_process_SNR, BS_raw_data_pre_process_average,
BS_cross_section_pre_process_SNR, BS_cross_section_pre_process_average,
BS_stream_bed_pre_process_SNR, BS_stream_bed_pre_process_average, BS_mean
FROM BSRawData WHERE (acoustic_data = {j})'''
data2 = cur.execute(query2).fetchall()
@ -270,10 +283,10 @@ class ReadTableForOpen:
# +++++++++++++++++++++++
for s in range(len(stg.acoustic_data)):
query3 = f'''SELECT acoustic_data, temperature,
query3 = f'''SELECT acoustic_data, temperature,
tmin_index, tmin_value, tmax_index, tmax_value,
rmin_index, rmin_value, rmax_index, rmax_value,
freq_bottom_detection_index, freq_bottom_detection_value,
freq_bottom_detection_index, freq_bottom_detection_value,
SNR_filter_value, Nb_cells_to_average_BS_signal
FROM Settings WHERE (acoustic_data = {s})'''
@ -293,10 +306,10 @@ class ReadTableForOpen:
# --- Table Sediment File ---
# +++++++++++++++++++++++++++
query4 = f'''SELECT path_fine, filename_fine, radius_grain_fine, path_sand, filename_sand, radius_grain_sand,
time_column_label, distance_from_bank_column_label, depth_column_label,
Ctot_fine_column_label, D50_fine_column_label,
Ctot_sand_column_label, D50_sand_column_label
query4 = f'''SELECT path_fine, filename_fine, radius_grain_fine, path_sand, filename_sand, radius_grain_sand,
time_column_label, distance_from_bank_column_label, depth_column_label,
Ctot_fine_column_label, D50_fine_column_label,
Ctot_sand_column_label, D50_sand_column_label
from SedimentsFile'''
data4 = cur.execute(query4).fetchall()
@ -321,10 +334,10 @@ class ReadTableForOpen:
# --- Table Sediment Data ---
# +++++++++++++++++++++++++++
query5 = f'''SELECT sample_fine_name, sample_fine_index, distance_from_bank_fine, depth_fine, time_fine,
query5 = f'''SELECT sample_fine_name, sample_fine_index, distance_from_bank_fine, depth_fine, time_fine,
Ctot_fine, Ctot_fine_per_cent, D50_fine, frac_vol_fine, frac_vol_fine_cumul,
sample_sand_name, sample_sand_index, distance_from_bank_sand, depth_sand, time_sand,
Ctot_sand, Ctot_sand_per_cent, D50_sand, frac_vol_sand, frac_vol_sand_cumul
sample_sand_name, sample_sand_index, distance_from_bank_sand, depth_sand, time_sand,
Ctot_sand, Ctot_sand_per_cent, D50_sand, frac_vol_sand, frac_vol_sand_cumul
from SedimentsData'''
data5 = cur.execute(query5).fetchall()
@ -428,4 +441,3 @@ class ReadTableForOpen:
stg.BS_raw_data.append(np.reshape(stg.BS_raw_data_reshape[i],
(len(stg.freq[i]), stg.depth[i].shape[1], stg.time[i].shape[1])))

View File

@ -43,7 +43,7 @@ def raw_extract(_raw_file):
while 1:
flag, size, data = fileraw.read_chunk()
# Pour raw UDT005 (ie. UB-Lab P, UB-SediFlow, UB-Lab 3C) on peut
# Pour raw UDT005 (ie. UB-Lab P, UB-SediFlow, UB-Lab 3C) on peut
# rencontrer 4 flags: const, settings json, configs (HW), profils
if flag == CONST_TAG:
try:
@ -55,7 +55,6 @@ def raw_extract(_raw_file):
.replace("True", "true")
.replace("False", "false")
)
print("const: %s" % const_dict)
ubt_data = ubt_raw_data( const_dict )
@ -70,12 +69,9 @@ def raw_extract(_raw_file):
.replace("True", "true")
.replace("False", "false")
)
print("settings: %s" % settings_dict)
ubt_data.set_config(settings_dict)
print("ubt_data.set_config(settings_dict) : ", ubt_data.set_config(settings_dict))
if flag == CONFIG_TAG:
# what is needed from here and which is not in param_us_dict is only blind_ca0 and blind_ca1
# note: this is not useful on APF06, but could be used for double check
@ -100,7 +96,7 @@ def raw_extract(_raw_file):
#print("%d profiles read" % profile_id)
# last timestamp of udt file for time_end definition of run:
# based on the last profile processed
# based on the last profile processed
time_end = timestamp
return (
@ -111,4 +107,4 @@ def raw_extract(_raw_file):
ubt_data.data_us_dicts,
ubt_data.data_dicts,
settings_dict,
)
)

View File

@ -51,7 +51,7 @@ class UpdateTableForSave:
def update_table(self):
# Create a new database and open a database connection to allow sqlite3 to work with it.
cnx = sqlite3.connect(stg.filename_save_as + '.acd')
cnx = sqlite3.connect(stg.filename_save_as)
# Create database cursor to execute SQL statements and fetch results from SQL queries.
cur = cnx.cursor()
@ -67,12 +67,12 @@ class UpdateTableForSave:
cur.execute("""CREATE TABLE AcousticFile(ID INTEGER PRIMARY KEY AUTOINCREMENT,
acoustic_data INTEGER,
acoustic_file STRING,
acoustic_file STRING,
ABS_name STRING,
path_BS_noise_data STRING,
filename_BS_noise_data STRING,
noise_method FLOAT,
noise_value FLOAT,
noise_value FLOAT,
data_preprocessed STRING
)"""
)
@ -83,8 +83,8 @@ class UpdateTableForSave:
print('stg.ABS_name', stg.ABS_name)
print("stg.path_BS_raw_data ", stg.path_BS_raw_data[i])
cur.execute(''' INSERT into AcousticFile(acoustic_data, acoustic_file, ABS_name, path_BS_noise_data,
filename_BS_noise_data, noise_method, noise_value, data_preprocessed)
cur.execute(''' INSERT into AcousticFile(acoustic_data, acoustic_file, ABS_name, path_BS_noise_data,
filename_BS_noise_data, noise_method, noise_value, data_preprocessed)
VALUES(?, ?, ?, ?, ?, ?, ?, ?)''',
(stg.acoustic_data[i], stg.filename_BS_raw_data[i].split('.')[0], stg.ABS_name[i],
stg.path_BS_noise_data[i], stg.filename_BS_noise_data[i], stg.noise_method[i],
@ -104,42 +104,72 @@ class UpdateTableForSave:
# Drop Table if exists
cur.execute("DROP TABLE if exists Measure")
cur.execute("""CREATE TABLE Measure(ID INTEGER PRIMARY KEY AUTOINCREMENT,
acoustic_data INTEGER,
Date STRING,
Hour STRING,
frequency FLOAT,
sound_attenuation FLOAT,
kt_read FLOAT,
kt_corrected FLOAT,
NbProfiles FLOAT,
NbProfilesPerSeconds FLOAT,
NbCells FLOAT,
CellSize FLOAT,
PulseLength FLOAT,
NbPingsPerSeconds FLOAT,
NbPingsAveragedPerProfile FLOAT,
GainRx FLOAT,
GainTx FLOAT
)
""")
cur.execute(
"""
CREATE TABLE Measure(
ID INTEGER PRIMARY KEY AUTOINCREMENT,
acoustic_data INTEGER,
Date STRING,
Hour STRING,
frequency FLOAT,
sound_attenuation FLOAT,
kt_read FLOAT,
kt_corrected FLOAT,
NbProfiles FLOAT,
NbProfilesPerSeconds FLOAT,
NbCells FLOAT,
CellSize FLOAT,
PulseLength FLOAT,
NbPingsPerSeconds FLOAT,
NbPingsAveragedPerProfile FLOAT,
GainRx FLOAT,
GainTx FLOAT
)"""
)
# Fill the table Measure
for i in stg.acoustic_data:
for j in range(stg.freq[i].shape[0]):
cur.execute(''' INSERT into Measure(acoustic_data, Date, Hour, frequency, sound_attenuation, kt_read, kt_corrected, NbProfiles,
NbProfilesPerSeconds, NbCells, CellSize, PulseLength,
NbPingsPerSeconds, NbPingsAveragedPerProfile, GainRx, GainTx,
)
VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)''',
(stg.acoustic_data[i], stg.freq[i][j], stg.water_attenuation[i][j], stg.kt_read[j], stg.kt_corrected[j],
stg.nb_profiles[i][j], stg.nb_profiles_per_sec[i][j], stg.nb_cells[i][j],
stg.cell_size[i][j], stg.pulse_length[i][j], stg.nb_pings_per_sec[i][j],
stg.nb_pings_averaged_per_profile[i][j], stg.gain_rx[i][j], stg.gain_tx[i][j],
str(stg.date[i].year) + str('-') + str(stg.date[i].month) + str('-') + str(stg.date[i].day),
str(stg.hour[i].hour) + str(':') + str(stg.hour[i].minute)
))
cur.execute(
'''
INSERT into Measure(
acoustic_data,
Date, Hour,
frequency,
sound_attenuation,
kt_read, kt_corrected,
NbProfiles, NbProfilesPerSeconds,
NbCells, CellSize,
PulseLength,
NbPingsPerSeconds,
NbPingsAveragedPerProfile,
GainRx, GainTx
) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)''',
(
stg.acoustic_data[i],
(
str(stg.date[i].year) + str('-') +
str(stg.date[i].month) + str('-') +
str(stg.date[i].day)
),
(
str(stg.hour[i].hour) + str(':') +
str(stg.hour[i].minute)
),
stg.freq[i][j],
stg.water_attenuation[i][j],
stg.kt_read[j],
stg.kt_corrected[j],
stg.nb_profiles[i][j],
stg.nb_profiles_per_sec[i][j],
stg.nb_cells[i][j],
stg.cell_size[i][j],
stg.pulse_length[i][j],
stg.nb_pings_per_sec[i][j],
stg.nb_pings_averaged_per_profile[i][j],
stg.gain_rx[i][j], stg.gain_tx[i][j]
)
)
# Commit the transaction after executing INSERT.
cnx.commit()
@ -150,7 +180,7 @@ class UpdateTableForSave:
# +++++++++++++++++++++++++++
# --- Table Acoustic Data ---
# +++++++++++++++++++++++++++
start_table_BSRawData = time.time()
cur.execute(''' DROP TABLE BSRawData ''')
@ -158,29 +188,29 @@ class UpdateTableForSave:
cur.execute('''CREATE TABLE BSRawData(ID INTEGER PRIMARY KEY AUTOINCREMENT,
acoustic_data INTEGER,
time BLOB, depth BLOB, BS_raw_data BLOB,
time_reshape BLOB, depth_reshape BLOB, BS_raw_data_reshape BLOB,
time_reshape BLOB, depth_reshape BLOB, BS_raw_data_reshape BLOB,
time_cross_section BLOB, depth_cross_section BLOB, BS_cross_section BLOB, BS_stream_bed BLOB,
depth_bottom, val_bottom, ind_bottom,
time_noise BLOB, depth_noise BLOB, BS_noise_raw_data BLOB,
depth_bottom, val_bottom, ind_bottom,
time_noise BLOB, depth_noise BLOB, BS_noise_raw_data BLOB,
SNR_raw_data BLOB, SNR_cross_section BLOB, SNR_stream_bed BLOB,
BS_raw_data_pre_process_SNR BLOB, BS_raw_data_pre_process_average BLOB,
BS_cross_section_pre_process_SNR BLOB, BS_cross_section_pre_process_average BLOB,
BS_stream_bed_pre_process_SNR BLOB, BS_stream_bed_pre_process_average BLOB,
BS_raw_data_pre_process_SNR BLOB, BS_raw_data_pre_process_average BLOB,
BS_cross_section_pre_process_SNR BLOB, BS_cross_section_pre_process_average BLOB,
BS_stream_bed_pre_process_SNR BLOB, BS_stream_bed_pre_process_average BLOB,
BS_mean BLOB
)''')
for i in stg.acoustic_data:
cur.execute(''' INSERT into BSRawData(acoustic_data, time, depth, BS_raw_data,
cur.execute(''' INSERT into BSRawData(acoustic_data, time, depth, BS_raw_data,
time_reshape, depth_reshape, BS_raw_data_reshape,
time_cross_section, depth_cross_section,
BS_cross_section, BS_stream_bed,
depth_bottom, val_bottom, ind_bottom,
time_noise, depth_noise, BS_noise_raw_data,
time_cross_section, depth_cross_section,
BS_cross_section, BS_stream_bed,
depth_bottom, val_bottom, ind_bottom,
time_noise, depth_noise, BS_noise_raw_data,
SNR_raw_data, SNR_cross_section, SNR_stream_bed,
BS_raw_data_pre_process_SNR, BS_raw_data_pre_process_average,
BS_cross_section_pre_process_SNR, BS_cross_section_pre_process_average,
BS_stream_bed_pre_process_SNR, BS_stream_bed_pre_process_average,
BS_raw_data_pre_process_SNR, BS_raw_data_pre_process_average,
BS_cross_section_pre_process_SNR, BS_cross_section_pre_process_average,
BS_stream_bed_pre_process_SNR, BS_stream_bed_pre_process_average,
BS_mean)
VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)''',
(stg.acoustic_data[i], stg.time[i].tobytes(), stg.depth[i].tobytes(), stg.BS_raw_data[i].tobytes(),
@ -219,8 +249,8 @@ class UpdateTableForSave:
acoustic_data INTEGER,
temperature FLOAT,
tmin_index FLOAT, tmin_value FLOAT, tmax_index FLOAT, tmax_value FLOAT,
rmin_index FLOAT, rmin_value FLOAT, rmax_index FLOAT, rmax_value FLOAT,
freq_bottom_detection_index FLOAT, freq_bottom_detection_value STRING,
rmin_index FLOAT, rmin_value FLOAT, rmax_index FLOAT, rmax_value FLOAT,
freq_bottom_detection_index FLOAT, freq_bottom_detection_value STRING,
SNR_filter_value FLOAT, Nb_cells_to_average_BS_signal FLOAT
)'''
)
@ -229,8 +259,8 @@ class UpdateTableForSave:
cur.execute('''INSERT into Settings(acoustic_data, temperature,
tmin_index, tmin_value, tmax_index, tmax_value,
rmin_index, rmin_value, rmax_index, rmax_value,
freq_bottom_detection_index, freq_bottom_detection_value,
SNR_filter_value, Nb_cells_to_average_BS_signal)
freq_bottom_detection_index, freq_bottom_detection_value,
SNR_filter_value, Nb_cells_to_average_BS_signal)
VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)''',
(stg.acoustic_data[i], stg.temperature,
stg.tmin[i][0], stg.tmin[i][1], stg.tmax[i][0], stg.tmax[i][1],
@ -252,29 +282,29 @@ class UpdateTableForSave:
cur.execute("DROP TABLE if exists SedimentsFile")
cur.execute("""CREATE TABLE SedimentsFile(ID INTEGER PRIMARY KEY AUTOINCREMENT,
path_fine STRING,
filename_fine STRING,
radius_grain_fine BLOB,
path_sand STRING,
cur.execute("""CREATE TABLE SedimentsFile(ID INTEGER PRIMARY KEY AUTOINCREMENT,
path_fine STRING,
filename_fine STRING,
radius_grain_fine BLOB,
path_sand STRING,
filename_sand STRING,
radius_grain_sand BLOB,
time_column_label STRING,
distance_from_bank_column_label STRING,
depth_column_label STRING,
Ctot_fine_column_label STRING,
radius_grain_sand BLOB,
time_column_label STRING,
distance_from_bank_column_label STRING,
depth_column_label STRING,
Ctot_fine_column_label STRING,
D50_fine_column_label STRING,
Ctot_sand_column_label STRING,
Ctot_sand_column_label STRING,
D50_sand_column_label STRING
)"""
)
cur.execute('''INSERT into SedimentsFile(path_fine, filename_fine, radius_grain_fine,
path_sand, filename_sand, radius_grain_sand,
time_column_label, distance_from_bank_column_label,
depth_column_label, Ctot_fine_column_label,
D50_fine_column_label, Ctot_sand_column_label,
D50_sand_column_label)
cur.execute('''INSERT into SedimentsFile(path_fine, filename_fine, radius_grain_fine,
path_sand, filename_sand, radius_grain_sand,
time_column_label, distance_from_bank_column_label,
depth_column_label, Ctot_fine_column_label,
D50_fine_column_label, Ctot_sand_column_label,
D50_sand_column_label)
VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)''',
(stg.path_fine, stg.filename_fine, stg.radius_grain_fine.tobytes(),
stg.path_sand, stg.filename_sand, stg.radius_grain_sand.tobytes(),
@ -294,26 +324,26 @@ class UpdateTableForSave:
cur.execute("DROP TABLE if exists SedimentsData")
cur.execute("""CREATE TABLE SedimentsData(ID INTEGER PRIMARY KEY AUTOINCREMENT,
sample_fine_name STRING,
cur.execute("""CREATE TABLE SedimentsData(ID INTEGER PRIMARY KEY AUTOINCREMENT,
sample_fine_name STRING,
sample_fine_index INTEGER,
distance_from_bank_fine FLOAT,
depth_fine FLOAT,
time_fine FLOAT,
distance_from_bank_fine FLOAT,
depth_fine FLOAT,
time_fine FLOAT,
Ctot_fine FLOAT,
Ctot_fine_per_cent FLOAT,
D50_fine FLOAT,
frac_vol_fine BLOB,
frac_vol_fine_cumul BLOB,
sample_sand_name STRING,
Ctot_fine_per_cent FLOAT,
D50_fine FLOAT,
frac_vol_fine BLOB,
frac_vol_fine_cumul BLOB,
sample_sand_name STRING,
sample_sand_index INTEGER,
distance_from_bank_sand FLOAT,
depth_sand FLOAT,
time_sand FLOAT,
distance_from_bank_sand FLOAT,
depth_sand FLOAT,
time_sand FLOAT,
Ctot_sand FLOAT,
Ctot_sand_per_cent FLOAT,
D50_sand FLOAT,
frac_vol_sand BLOB,
Ctot_sand_per_cent FLOAT,
D50_sand FLOAT,
frac_vol_sand BLOB,
frac_vol_sand_cumul BLOB
)"""
)
@ -321,7 +351,7 @@ class UpdateTableForSave:
for f in range(len(stg.sample_fine)):
cur.execute('''INSERT into SedimentsData(sample_fine_name, sample_fine_index, distance_from_bank_fine,
depth_fine, time_fine, Ctot_fine, Ctot_fine_per_cent, D50_fine, frac_vol_fine,
frac_vol_fine_cumul,
frac_vol_fine_cumul,
sample_sand_name, sample_sand_index, distance_from_bank_sand,
depth_sand, time_sand, Ctot_sand, Ctot_sand_per_cent, D50_sand, frac_vol_sand,
frac_vol_sand_cumul)
@ -349,26 +379,26 @@ class UpdateTableForSave:
cur.execute("DROP TABLE if exists Calibration")
cur.execute("""CREATE TABLE Calibration(ID INTEGER PRIMARY KEY AUTOINCREMENT,
path_calibration_file STRING,
filename_calibration_file STRING,
range_lin_interp BLOB,
M_profile_fine BLOB,
cur.execute("""CREATE TABLE Calibration(ID INTEGER PRIMARY KEY AUTOINCREMENT,
path_calibration_file STRING,
filename_calibration_file STRING,
range_lin_interp BLOB,
M_profile_fine BLOB,
ks BLOB,
sv BLOB,
X_exponent BLOB,
alpha_s BLOB,
zeta BLOB,
FCB BLOB,
depth_real BLOB,
sv BLOB,
X_exponent BLOB,
alpha_s BLOB,
zeta BLOB,
FCB BLOB,
depth_real BLOB,
lin_reg BLOB
)"""
)
cur.execute('''INSERT into Calibration(path_calibration_file, filename_calibration_file,
cur.execute('''INSERT into Calibration(path_calibration_file, filename_calibration_file,
range_lin_interp, M_profile_fine,
ks, sv, X_exponent, alpha_s, zeta,
FCB, depth_real, lin_reg)
FCB, depth_real, lin_reg)
VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)''',
(stg.path_calibration_file, stg.filename_calibration_file,
stg.range_lin_interp.tobytes(), stg.M_profile_fine.tobytes(),
@ -390,17 +420,17 @@ class UpdateTableForSave:
cur.execute("DROP TABLE if exists Inversion")
cur.execute("""CREATE TABLE Inversion(ID INTEGER PRIMARY KEY AUTOINCREMENT,
cur.execute("""CREATE TABLE Inversion(ID INTEGER PRIMARY KEY AUTOINCREMENT,
J_cross_section_freq1 BLOB,
J_cross_section_freq2 BLOB,
VBI_cross_section BLOB,
SSC_fine BLOB,
VBI_cross_section BLOB,
SSC_fine BLOB,
SSC_sand BLOB
)""")
for i in range(len(stg.SSC_fine)):
cur.execute('''INSERT into Inversion(J_cross_section_freq1, J_cross_section_freq2,
VBI_cross_section, SSC_fine, SSC_sand)
cur.execute('''INSERT into Inversion(J_cross_section_freq1, J_cross_section_freq2,
VBI_cross_section, SSC_fine, SSC_sand)
VALUES(?, ?, ?, ?, ?)''',
(stg.J_cross_section[i][0].tobytes(), stg.J_cross_section[i][1].tobytes(),
stg.VBI_cross_section[i].tobytes(), stg.SSC_fine[i].tobytes(), stg.SSC_sand[i].tobytes())
@ -417,6 +447,3 @@ class UpdateTableForSave:
# Close database connection
cnx.close()

194
README.md
View File

@ -1,92 +1,134 @@
# AcouSed
AcouSed for **Acou**stic Backscattering for Concentration of Suspended **Sed**iments in Rivers is a software developped by INRAE, in collaboation with CNR.
<p>
<img src="logos/AcouSed.png" align="center" width=20% height=20% >
</p>
## Getting started
To make it easy for you to get started with GitLab, here's a list of recommended next steps.
Already a pro? Just edit this README.md and make it your own. Want to make it easy? [Use the template at the bottom](#editing-this-readme)!
## Add your files
- [ ] [Create](https://docs.gitlab.com/ee/user/project/repository/web_editor.html#create-a-file) or [upload](https://docs.gitlab.com/ee/user/project/repository/web_editor.html#upload-a-file) files
- [ ] [Add files using the command line](https://docs.gitlab.com/ee/gitlab-basics/add-file.html#add-a-file-using-the-command-line) or push an existing Git repository with the following command:
```
cd existing_repo
git remote add origin https://gitlab.irstea.fr/brahim/acoused.git
git branch -M main
git push -uf origin main
```
## Integrate with your tools
- [ ] [Set up project integrations](https://gitlab.irstea.fr/brahim/acoused/-/settings/integrations)
## Collaborate with your team
- [ ] [Invite team members and collaborators](https://docs.gitlab.com/ee/user/project/members/)
- [ ] [Create a new merge request](https://docs.gitlab.com/ee/user/project/merge_requests/creating_merge_requests.html)
- [ ] [Automatically close issues from merge requests](https://docs.gitlab.com/ee/user/project/issues/managing_issues.html#closing-issues-automatically)
- [ ] [Enable merge request approvals](https://docs.gitlab.com/ee/user/project/merge_requests/approvals/)
- [ ] [Automatically merge when pipeline succeeds](https://docs.gitlab.com/ee/user/project/merge_requests/merge_when_pipeline_succeeds.html)
## Test and Deploy
Use the built-in continuous integration in GitLab.
- [ ] [Get started with GitLab CI/CD](https://docs.gitlab.com/ee/ci/quick_start/index.html)
- [ ] [Analyze your code for known vulnerabilities with Static Application Security Testing(SAST)](https://docs.gitlab.com/ee/user/application_security/sast/)
- [ ] [Deploy to Kubernetes, Amazon EC2, or Amazon ECS using Auto Deploy](https://docs.gitlab.com/ee/topics/autodevops/requirements.html)
- [ ] [Use pull-based deployments for improved Kubernetes management](https://docs.gitlab.com/ee/user/clusters/agent/)
- [ ] [Set up protected environments](https://docs.gitlab.com/ee/ci/environments/protected_environments.html)
***
# Editing this README
When you're ready to make this README your own, just edit this file and use the handy template below (or feel free to structure it however you want - this is just a starting point!). Thank you to [makeareadme.com](https://www.makeareadme.com/) for this template.
## Suggestions for a good README
Every project is different, so consider which of these sections apply to yours. The sections used in the template are suggestions for most open source projects. Also keep in mind that while a README can be too long and detailed, too long is better than too short. If you think your README is too long, consider utilizing another form of documentation rather than cutting out information.
## Name
Choose a self-explaining name for your project.
## Description
Let people know what your project can do specifically. Provide context and add a link to any reference visitors might be unfamiliar with. A list of Features or a Background subsection can also be added here. If there are alternatives to your project, this is a good place to list differentiating factors.
## Badges
On some READMEs, you may see small images that convey metadata, such as whether or not all the tests are passing for the project. You can use Shields to add some to your README. Many services also have instructions for adding a badge.
## Visuals
Depending on what you are making, it can be a good idea to include screenshots or even a video (you'll frequently see GIFs rather than actual videos). Tools like ttygif can help, but check out Asciinema for a more sophisticated method.
It is divided in six tabs:
- Acoustic data : acoustic raw data are downloaded and visualised
- Signal preprocessing : acoustic raw signal is preprocessed with filters
- Sample data : fine and sand sediments samples data are downloaded and visualised
- Calibration : calibration parameter are computed
- Inversion : inversion method is calculated to provide fine and sand sediments fields
## Installation
Within a particular ecosystem, there may be a common way of installing things, such as using Yarn, NuGet, or Homebrew. However, consider the possibility that whoever is reading your README is a novice and would like more guidance. Listing specific steps helps remove ambiguity and gets people to using your project as quickly as possible. If it only runs in a specific context like a particular programming language version or operating system or has dependencies that have to be installed manually, also add a Requirements subsection.
## Usage
Use examples liberally, and show the expected output if you can. It's helpful to have inline the smallest example of usage that you can demonstrate, while providing links to more sophisticated examples if they are too long to reasonably include in the README.
### Standalone software
## Support
Tell people where they can go to for help. It can be any combination of an issue tracker, a chat room, an email address, etc.
AcouSed can be launched with python installation. An executable is available on [River Hydraulics](https://riverhydraulics.riverly.inrae.fr/outils/logiciels-pour-la-mesure/acoused) teams website.
The user needs to download the folder "acoused-packaging" including :
- icons and logos folder
- _internal folder (python packages)
- executable file
- calibration constant file
- documentation
## Roadmap
If you have ideas for releases in the future, it is a good idea to list them in the README.
Acoused.exe file must be launched from this folder.
Test data can be dowloaded from the [INRAE nextcloud](https://nextcloud.inrae.fr/s/3zZdieztrx7nwYa)
## Contributing
State if you are open to contributions and what your requirements are for accepting them.
### Python environment
For people who want to make changes to your project, it's helpful to have some documentation on how to get started. Perhaps there is a script that they should run or some environment variables that they need to set. Make these steps explicit. These instructions could also be useful to your future self.
Acoused is developped for Linux and Windows on Python version 3.8 or
greater. By default, Acoused is developped with Pypi package
dependencies, but is also possible to use Guix package manager to run
Acoused.
You can also document commands to lint the code or run tests. These steps help to ensure high code quality and reduce the likelihood that the changes inadvertently break something. Having instructions for running tests is especially helpful if it requires external setup, such as starting a Selenium server for testing in a browser.
#### Windows
You can use Pypi to get correct software environment and run the
program.
```shell
python -m venv env
env\Scripts\activate.bat
python -m pip install -U -r ..\virtualenv\requirements.txt
python main.py
```
#### Linux
You can use Pypi to get correct software environment and run the
program.
```shell
python3 -m venv venv
source ./venv/bin/activate
python3 -m pip install -r requirement.txt
python3 main.py
```
#### Linux with Guix
To run Acoused within a [GNU Guix](https://guix.gnu.org/) software
environment, you need Guix installed on your computer and run the
script `guix.sh` to run the program.
```shell
./guix.sh
# If you need sqlitebrowser, use this command
guix shell sqlitebrowser -- ./guix.sh
```
## Support files & References
- [ ] [Acoustic inversion method diagram](https://forgemia.inra.fr/theophile.terraz/acoused/-/blob/main/Acoustic_Inversion_theory.pdf?ref_type=heads)
- [ ] [Tutorial AQUAscat software : AQUAtalk](https://forgemia.inra.fr/theophile.terraz/acoused/-/blob/main/Tutorial_AQUAscat_software.pdf?ref_type=heads)
- [ ] [Adrien Vergne thesis (2018)](https://theses.fr/2018GREAU046)
- [ ] [Vergne A., Le Coz J., Berni C., & Pierrefeu G. (2020), Water Resources Research, 56(2)](https://doi.org/10.1029/2019WR024877)
- [ ] [Vergne A., Berni C., Le Coz J., & Tencé F., (2021), Water Resources Research, 57(9)](https://doi.org/10.1029/2021WR029589)
## Authors & Contacts
- Brahim MOUDJED 2022-2025 ([INRAE](https://www.inrae.fr/))
- Pierre-Antoine ROUBY 2025 ([TECC](https://parouby.fr))
If you have any questions or suggestions, please contact us to celine.berni@inrae.fr and/or jerome.lecoz@inrae.fr.
## Acknowledgment
This study was conducted within the [Rhône Sediment Observatory](https://observatoire-sediments-rhone.fr/) (OSR), a multi-partner research program funded through the Plan Rhône by the European Regional Development Fund (ERDF), Agence de lEau RMC, CNR, EDF and three regional councils (Auvergne-Rhône-Alpes, PACA and Occitanie).
<p>
<img src="logos/OSR.png" align="center" width=10% height=10% >
</p>
## Industrial partners
[CNR](https://www.cnr.tm.fr/)
<p>
<img src="logos/CNR.png" align="center" width=10% height=10% >
</p>
[UBERTONE](https://ubertone.com/)
<p>
<img src="logos/Ubertone.jpeg" align="center" width=10% height=10% >
</p>
[EDF](https://www.edf.fr/hydraulique-isere-drome)
<p>
<img src="logos/EDF.png" align="center" width=10% height=10% >
</p>
## Authors and acknowledgment
Show your appreciation to those who have contributed to the project.
## License
For open source projects, say how it is licensed.
## Project status
If you have run out of energy or time for your project, put a note at the top of the README saying that development has slowed down or stopped completely. Someone may choose to fork your project or volunteer to step in as a maintainer or owner, allowing your project to keep going. You can also make an explicit request for maintainers.
AcouSed
Copyright (C) 2024-2025 - INRAE
<p>
<img src="logos/BlocMarque-INRAE-Inter.jpg" align="center" width=10% height=10% >
</p>
This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program. If not, see <https://www.gnu.org/licenses/>.

View File

@ -31,13 +31,13 @@ class AboutWindow(QDialog):
super().__init__()
self.logo_path = "./Logo"
self.logo_AcouSed = QPixmap(self.logo_path + "/" + "Logo_AcouSed_AboutAcouSedWindow.png")
self.logo_path = "./logos"
self.logo_AcouSed = QPixmap(self.logo_path + "/" + "AcouSed.png")
self.logo_AcouSed.scaled(16, 16, Qt.KeepAspectRatio, Qt.SmoothTransformation)
self.logo_INRAE = QPixmap(self.logo_path + "/" + "BlocMarque-INRAE-Inter.jpg")
self.setGeometry(400, 200, 300, 200)
self.setGeometry(400, 200, 350, 200)
self.setWindowTitle("About AcouSed")
@ -51,7 +51,7 @@ class AboutWindow(QDialog):
self.label_logo_AcouSed = QLabel()
self.label_logo_AcouSed.setPixmap(self.logo_AcouSed.scaledToHeight(128, Qt.SmoothTransformation))
self.gridLayout.addWidget(self.label_logo_AcouSed, 0, 0, 3, 1, Qt.AlignCenter)
self.gridLayout.addWidget(self.label_logo_AcouSed, 0, 0, 5, 1, Qt.AlignCenter)
self.label_acoused = QLabel()
self.label_acoused.setText("Acoused 2.0")
@ -70,7 +70,13 @@ class AboutWindow(QDialog):
self.label_contact = QLabel()
self.label_contact.setText("Contact : celine.berni@inrae.fr \n"
" jerome.lecoz@inrae.fr")
self.gridLayout.addWidget(self.label_contact, 3, 1, 1, 1, Qt.AlignLeft)
self.gridLayout.addWidget(self.label_contact, 3, 1, 1, 1, Qt.AlignCenter)
self.label_link = QLabel()
self.label_link.setText("< a href = https://forgemia.inra.fr/theophile.terraz/acoused > "
"https://forgemia.inra.fr/theophile.terraz/acoused </a>")
self.label_link.setOpenExternalLinks(True)
self.gridLayout.addWidget(self.label_link, 4, 1, 1, 1, Qt.AlignCenter)
# ----------------------------------------------------------
@ -179,7 +185,7 @@ class Support(QDialog):
super().__init__()
self.logo_path = "./Logo"
self.logo_path = "./logos"
self.logo_OSR = QPixmap(self.logo_path + '/' + "OSR.png")
self.logo_CNR = QPixmap(self.logo_path + '/' + "CNR.png")

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@ -39,12 +39,11 @@ from View.about_window import AboutWindow
import settings as stg
import numpy as np
from subprocess import check_call, run
import pandas as pd
from subprocess import Popen
import time
from View.acoustic_data_tab import AcousticDataTab
class Ui_MainWindow(object):
def setupUi(self, MainWindow):
@ -55,10 +54,13 @@ class Ui_MainWindow(object):
self.mainwindow.setDocumentMode(False)
self.mainwindow.setDockNestingEnabled(False)
self.mainwindow.setUnifiedTitleAndToolBarOnMac(False)
self.centralwidget = QtWidgets.QWidget(self.mainwindow)
self.centralwidget.setObjectName("centralwidget")
self.verticalLayout = QtWidgets.QVBoxLayout(self.centralwidget)
self.verticalLayout.setObjectName("verticalLayout")
self.tabWidget = QtWidgets.QTabWidget(self.centralwidget)
self.tabWidget.setAutoFillBackground(False)
self.tabWidget.setLocale(QtCore.QLocale(QtCore.QLocale.French, QtCore.QLocale.France))
@ -66,115 +68,127 @@ class Ui_MainWindow(object):
self.tabWidget.setTabsClosable(False)
self.tabWidget.setTabBarAutoHide(False)
self.tabWidget.setObjectName("tabWidget")
self.tab1 = QtWidgets.QWidget()
self.tab1.setContextMenuPolicy(QtCore.Qt.ActionsContextMenu)
self.tab1.setObjectName("tab1")
self.tabWidget.addTab(self.tab1, "")
self.tab2 = QtWidgets.QWidget()
self.tab2.setObjectName("tab2")
self.tabWidget.addTab(self.tab2, "")
self.tab3 = QtWidgets.QWidget()
self.tab3.setObjectName("tab3")
self.tabWidget.addTab(self.tab3, "")
self.tab4 = QtWidgets.QWidget()
self.tab4.setObjectName("tab4")
self.tabWidget.addTab(self.tab4, "")
self.tab5 = QtWidgets.QWidget()
self.tab5.setObjectName("tab5")
self.tabWidget.addTab(self.tab5, "")
self.tab6 = QtWidgets.QWidget()
self.tab6.setObjectName("tab6")
self.tabWidget.addTab(self.tab6, "")
# self.tab7 = QtWidgets.QWidget()
# self.tab7.setObjectName("tab7")
# self.tabWidget.addTab(self.tab7, "")
self.verticalLayout.addWidget(self.tabWidget)
self.mainwindow.setCentralWidget(self.centralwidget)
self.menubar = QtWidgets.QMenuBar(self.mainwindow)
self.menubar.setGeometry(QtCore.QRect(0, 0, 898, 22))
self.menubar.setObjectName("menubar")
self.menuFile = QtWidgets.QMenu(self.menubar)
self.menuFile.setLocale(QtCore.QLocale(QtCore.QLocale.French, QtCore.QLocale.France))
self.menuFile.setObjectName("menuFile")
self.menuSettings = QtWidgets.QMenu(self.menuFile)
self.menuSettings.setObjectName("menuSettings")
self.menuLanguage = QtWidgets.QMenu(self.menuSettings)
self.menuLanguage.setObjectName("menuLanguage")
self.menuExport = QtWidgets.QMenu(self.menuFile)
self.menuExport.setObjectName("menuExport")
self.menuTools = QtWidgets.QMenu(self.menubar)
self.menuTools.setLocale(QtCore.QLocale(QtCore.QLocale.French, QtCore.QLocale.France))
self.menuTools.setObjectName("menuTools")
self.menuHelp = QtWidgets.QMenu(self.menubar)
self.menuHelp.setObjectName("menuHelp")
self.mainwindow.setMenuBar(self.menubar)
self.statusbar = QtWidgets.QStatusBar(self.mainwindow)
self.statusbar.setObjectName("statusbar")
self.mainwindow.setStatusBar(self.statusbar)
self.toolBar = QtWidgets.QToolBar(self.mainwindow)
self.toolBar.setObjectName("toolBar")
self.mainwindow.addToolBar(QtCore.Qt.TopToolBarArea, self.toolBar)
# self.actionNew = QtWidgets.QAction(self.mainwindow)
# icon = QtGui.QIcon()
# icon.addPixmap(QtGui.QPixmap("icons/new.png"), QtGui.QIcon.Normal, QtGui.QIcon.Off)
# self.actionNew.setIcon(icon)
# self.actionNew.setObjectName("actionNew")
self.actionOpen = QtWidgets.QAction(self.mainwindow)
icon1 = QtGui.QIcon()
icon1.addPixmap(QtGui.QPixmap("icons/icon_folder.png"), QtGui.QIcon.Normal, QtGui.QIcon.Off)
self.actionOpen.setIcon(icon1)
self.actionOpen.setObjectName("actionOpen")
self.actionSave = QtWidgets.QAction(self.mainwindow)
icon2 = QtGui.QIcon()
icon2.addPixmap(QtGui.QPixmap("icons/save.png"), QtGui.QIcon.Normal, QtGui.QIcon.Off)
self.actionSave.setIcon(icon2)
self.actionSave.setObjectName("actionSave")
# self.actionCopy = QtWidgets.QAction(self.mainwindow)
# icon3 = QtGui.QIcon()
# icon3.addPixmap(QtGui.QPixmap("icons/copy.png"), QtGui.QIcon.Normal, QtGui.QIcon.Off)
# self.actionCopy.setIcon(icon3)
# self.actionCopy.setObjectName("actionCopy")
# self.actionCut = QtWidgets.QAction(self.mainwindow)
# icon4 = QtGui.QIcon()
# icon4.addPixmap(QtGui.QPixmap("icons/cut.png"), QtGui.QIcon.Normal, QtGui.QIcon.Off)
# self.actionCut.setIcon(icon4)
# self.actionCut.setObjectName("actionCut")
# self.actionPaste = QtWidgets.QAction(self.mainwindow)
# icon5 = QtGui.QIcon()
# icon5.addPixmap(QtGui.QPixmap("icons/paste.png"), QtGui.QIcon.Normal, QtGui.QIcon.Off)
# self.actionPaste.setIcon(icon5)
# self.actionPaste.setObjectName("actionPaste")
self.actionEnglish = QtWidgets.QAction(self.mainwindow)
icon6 = QtGui.QIcon()
icon6.addPixmap(QtGui.QPixmap("icons/en.png"), QtGui.QIcon.Normal, QtGui.QIcon.Off)
self.actionEnglish.setIcon(icon6)
self.actionEnglish.setObjectName("actionEnglish")
self.actionEnglish.setEnabled(False)
self.actionFrench = QtWidgets.QAction(self.mainwindow)
icon7 = QtGui.QIcon()
icon7.addPixmap(QtGui.QPixmap("icons/fr.png"), QtGui.QIcon.Normal, QtGui.QIcon.Off)
self.actionFrench.setIcon(icon7)
self.actionFrench.setObjectName("actionFrench")
self.actionFrench.setEnabled(False)
self.action_ABSCalibrationConstant = QtWidgets.QAction(self.mainwindow)
self.action_ABSCalibrationConstant.setText("ABS constant calibration kt")
self.actionTable_of_Backscatter_values = QtWidgets.QAction(self.mainwindow)
self.actionTable_of_Backscatter_values.setObjectName("actionTable_of_Backscatter_values")
self.actionSave_As = QtWidgets.QAction(self.mainwindow)
self.actionSave_As.setObjectName("actionSave_As")
self.actionAbout = QtWidgets.QAction(self.mainwindow)
self.actionAbout.setObjectName("actionAbout")
self.actionUserManual = QtWidgets.QAction(self.mainwindow)
self.actionUserManual.setText("User Manual")
self.action_AcousticInversionTheory = QtWidgets.QAction(self.mainwindow)
self.action_AcousticInversionTheory.setText("Acoustic inversion theory")
self.action_AQUAscatUserManual = QtWidgets.QAction(self.mainwindow)
self.action_AQUAscatUserManual.setText("Tutorial AQUAscat software")
self.actionDB_Browser_for_SQLite = QtWidgets.QAction(self.mainwindow)
self.actionDB_Browser_for_SQLite.setObjectName("actionDB_Browser_for_SQLite")
self.menuLanguage.addAction(self.actionEnglish)
self.menuLanguage.addAction(self.actionFrench)
self.menuSettings.addAction(self.menuLanguage.menuAction())
self.menuSettings.addAction(self.action_ABSCalibrationConstant)
self.menuExport.addAction(self.actionTable_of_Backscatter_values)
self.menuFile.addAction(self.actionOpen)
self.menuFile.addAction(self.actionSave)
self.menuFile.addAction(self.actionSave_As)
@ -182,21 +196,21 @@ class Ui_MainWindow(object):
self.menuFile.addAction(self.menuSettings.menuAction())
self.menuFile.addSeparator()
self.menuFile.addAction(self.menuExport.menuAction())
self.menuTools.addAction(self.actionDB_Browser_for_SQLite)
self.menuHelp.addAction(self.actionAbout)
self.menuHelp.addAction(self.actionUserManual)
self.menuHelp.addAction(self.action_AcousticInversionTheory)
self.menuHelp.addAction(self.action_AQUAscatUserManual)
self.menubar.addAction(self.menuFile.menuAction())
self.menubar.addAction(self.menuTools.menuAction())
self.menubar.addAction(self.menuHelp.menuAction())
# self.toolBar.addAction(self.actionNew)
self.toolBar.addAction(self.actionOpen)
self.toolBar.addAction(self.actionSave)
self.toolBar.addSeparator()
# self.toolBar.addAction(self.actionCopy)
# self.toolBar.addAction(self.actionCut)
# self.toolBar.addAction(self.actionPaste)
self.toolBar.addSeparator()
self.toolBar.addAction(self.actionEnglish)
self.toolBar.addAction(self.actionFrench)
@ -251,61 +265,82 @@ class Ui_MainWindow(object):
def save_as(self):
CreateTableForSaveAs()
self.mainwindow.setWindowTitle("AcouSed - " + stg.filename_save_as + ".acd")
self.mainwindow.setWindowTitle(
"AcouSed - " +
stg.filename_save_as
)
def save(self):
UpdateTableForSave()
if stg.dirname_save_as:
UpdateTableForSave()
else:
self.save_as()
def open(self):
pass
# ReadTableForOpen()
# acoustic_data_tab = AcousticDataTab()
#
# acoustic_data_tab.fileListWidget.addItems(stg.acoustic_data)
reader = ReadTableForOpen()
if reader.opened:
self.mainwindow.open_study_update_tabs()
def load_calibration_constant_values(self):
cc_kt = CalibrationConstantKt()
cc_kt.exec()
def db_browser_for_sqlite(self):
check_call("/usr/bin/sqlitebrowser")
try:
Popen("sqlitebrowser")
except OSError as e:
msg_box = QtWidgets.QMessageBox()
msg_box.setWindowTitle("DB Browser for SQLite Error")
msg_box.setIcon(QtWidgets.QMessageBox.Critical)
msg_box.setText(f"DB Browser for SQLite Error:\n {str(e)}")
msg_box.setStandardButtons(QtWidgets.QMessageBox.Ok)
msg_box.exec()
def about_window(self):
print("about")
aw = AboutWindow()
aw.exec()
def current_file_path(self, filename):
return os.path.abspath(
os.path.join(
os.path.dirname(__file__),
"..", filename
)
)
def open_doc_file(self, filename):
QtGui.QDesktopServices.openUrl(
QtCore.QUrl(
f"file://{self.current_file_path(filename)}"
)
)
def user_manual(self):
open('AcouSed_UserManual.pdf')
run(["open", 'AcouSed_UserManual.pdf'])
self.open_doc_file('AcouSed_UserManual.pdf')
def inversion_acoustic_theory(self):
open('Acoustic_Inversion_theory.pdf')
run(["open", 'Acoustic_Inversion_theory.pdf'])
self.open_doc_file('Acoustic_Inversion_theory.pdf')
def tutorial_AQUAscat_software(self):
open('Tutorial_AQUAscat_software.pdf')
run(["open", 'Tutorial_AQUAscat_software.pdf'])
self.open_doc_file('Tutorial_AQUAscat_software.pdf')
def export_table_of_acoustic_BS_values_to_excel_or_libreOfficeCalc_file(self):
if len(stg.BS_raw_data_reshape) != 0:
# --- Open file dialog to select the directory ---
name = QtWidgets.QFileDialog.getExistingDirectory(caption="Select Directory - Acoustic BS raw data Table")
name = QtWidgets.QFileDialog.getExistingDirectory(
caption="Select Directory - Acoustic BS raw data Table"
)
print("name table to save ", name)
# --- Save the raw acoustic backscatter data from a Dataframe to csv file ---
# --- Save the raw acoustic backscatter data from a
# --- Dataframe to csv file ---
t0 = time.time()
print("len(stg.BS_raw_data_reshape) ", len(stg.BS_raw_data_reshape))
print("len(stg.BS_raw_data_reshape) ",
len(stg.BS_raw_data_reshape))
if name:
for i in range(len(stg.BS_raw_data_reshape)):
header_list = []
header_list.clear()
table_data = np.array([[]])
@ -315,34 +350,42 @@ class Ui_MainWindow(object):
header_list.append("BS - " + freq_value)
if freq_ind == 0:
table_data = np.vstack((np.vstack((stg.time_reshape[i][:, freq_ind],
stg.depth_reshape[i][:, freq_ind])),
stg.BS_raw_data_reshape[i][:, freq_ind]))
table_data = np.vstack(
(
np.vstack(
(stg.time_reshape[i][:, freq_ind],
stg.depth_reshape[i][:, freq_ind])),
stg.BS_raw_data_reshape[i][:, freq_ind]
)
)
else:
table_data = np.vstack((table_data,
np.vstack((np.vstack(
(stg.time_reshape[i][:, freq_ind],
stg.depth_reshape[i][:, freq_ind])),
stg.BS_raw_data_reshape[i][:, freq_ind]))
))
table_data = np.vstack(
(
table_data,
np.vstack((
np.vstack((
stg.time_reshape[i][:, freq_ind],
stg.depth_reshape[i][:, freq_ind]
)),
stg.BS_raw_data_reshape[i][:, freq_ind]
))
)
)
exec("DataFrame_acoustic_" + str(i) + "= pd.DataFrame(None)")
exec("DataFrame_acoustic_" + str(i) + "= pd.DataFrame(data=table_data.transpose(), columns=header_list)")
DataFrame_acoustic = pd.DataFrame(None)
DataFrame_acoustic = pd.DataFrame(
data=table_data.transpose(), columns=header_list
)
# exec("DataFrame_acoustic_" + str(i) + ".to_csv(" +
# "excel_writer=" +
# '/home/bmoudjed/Documents/3 SSC acoustic meas project/Graphical interface project/BS_raw_data_table.xlsx' + "," +
# "sheet_name=stg.filename_BS_raw_data[i]," +
# "header=DataFrame_acoustic.columns," +
# "engine=" + "xlsxwriter" + ")")
exec("DataFrame_acoustic_" + str(i) + ".to_csv(" +
"path_or_buf=" +
"'" + name + "/" + "Table_" +
str(stg.filename_BS_raw_data[i][:-4]) + ".csv'" + ", " +
"sep=" + "',' " + ", " +
"header=DataFrame_acoustic_" + str(i) + ".columns" + ")")
DataFrame_acoustic.to_csv(
path_or_buf=os.path.join(
name,
f"Table_{str(stg.filename_BS_raw_data[i][:-4])}.csv"
),
header=DataFrame_acoustic.columns,
sep=',',
)
t1 = time.time() - t0
print("time duration export BS ", t1)
@ -357,7 +400,6 @@ class Ui_MainWindow(object):
self.tabWidget.setTabText(self.tabWidget.indexOf(self.tab4), _translate("MainWindow", "Sediment Calibration"))
self.tabWidget.setTabText(self.tabWidget.indexOf(self.tab5), _translate("MainWindow", "Acoustic inversion"))
self.tabWidget.setTabText(self.tabWidget.indexOf(self.tab6), _translate("MainWindow", "Note"))
# self.tabWidget.setTabText(self.tabWidget.indexOf(self.tab7), _translate("MainWindow", "User manual"))
self.menuFile.setTitle(_translate("MainWindow", "File"))
self.menuSettings.setTitle(_translate("MainWindow", "Settings"))
self.menuLanguage.setTitle(_translate("MainWindow", "Language"))
@ -365,12 +407,8 @@ class Ui_MainWindow(object):
self.menuTools.setTitle(_translate("MainWindow", "Tools"))
self.menuHelp.setTitle(_translate("MainWindow", "Help"))
self.toolBar.setWindowTitle(_translate("MainWindow", "toolBar"))
# self.actionNew.setText(_translate("MainWindow", "New"))
self.actionOpen.setText(_translate("MainWindow", "Open ..."))
self.actionSave.setText(_translate("MainWindow", "Save"))
# self.actionCopy.setText(_translate("MainWindow", "Copy"))
# self.actionCut.setText(_translate("MainWindow", "Cut"))
# self.actionPaste.setText(_translate("MainWindow", "Paste"))
self.actionEnglish.setText(_translate("MainWindow", "English"))
self.actionFrench.setText(_translate("MainWindow", "French"))
self.actionTable_of_Backscatter_values.setText(_translate("MainWindow", "Table of Backscatter values"))

View File

@ -1,7 +1,8 @@
import sys
from PyQt5.QtWidgets import QApplication, QWidget, QVBoxLayout, QHBoxLayout, QTextEdit, QPushButton, QSpacerItem, \
QSpinBox, QSizePolicy, QFontComboBox, QColorDialog
from PyQt5.QtWidgets import (
QApplication, QWidget, QVBoxLayout, QHBoxLayout,
QTextEdit, QPushButton, QSpacerItem, QSpinBox,
QSizePolicy, QFontComboBox, QColorDialog
)
from PyQt5.QtGui import QPixmap, QIcon, QFont
from PyQt5.QtCore import Qt
@ -189,27 +190,21 @@ class NoteTab(QWidget):
self.textEdit.setAlignment(Qt.AlignJustify)
def print_settings(self):
self.textEdit.setText(
"Acoustic data: \n\n"
f" ABS raw data file: {stg.path_BS_raw_data}/{stg.filename_BS_raw_data} \n"
f" ABS noise data file: {stg.path_BS_noise_data}/{stg.filename_BS_noise_data} \n"
"\n\n"
"------------------------------------------------------------------------- \n\n\n"
self.textEdit.setText("Acoustic data: \n\n"
f" ABS raw data file: {stg.path_BS_raw_data}/{stg.filename_BS_raw_data} \n"
f" ABS noise data file: {stg.path_BS_noise_data}/{stg.filename_BS_noise_data} \n"
"\n\n"
"------------------------------------------------------------------------- \n\n\n"
"Particle size data: \n"
f" Fine sediments data file: {stg.path_fine}/{stg.filename_fine} \n"
f" Sand sediments data file: {stg.path_sand}/{stg.filename_sand} \n"
"\n\n"
"------------------------------------------------------------------------- \n\n\n"
)
"Particle size data: \n"
f" Fine sediments data file: {stg.fine_sediment_path}/{stg.fine_sediment_filename} \n"
f" Sand sediments data file: {stg.sand_sediment_path}/{stg.sand_sediment_filename} \n"
"\n\n"
"------------------------------------------------------------------------- \n\n\n")
# "Acoustic Inversion parameters: \n"
# f" frequencies to compute VBI: {stg.freq_text[int(stg.frequencies_to_compute_VBI[0, 0])]}, "
# f"{stg.freq_text[int(stg.frequencies_to_compute_VBI[1, 0])]} \n"
# f" frequency to compute SSC: {stg.freq_text[int(stg.frequency_to_compute_SSC[0])]}")
# if __name__ == "__main__":
# app = QApplication(sys.argv)
# window = NoteTab()
# window.show()
# sys.exit(app.exec_())
# "Acoustic Inversion parameters: \n"
# f" frequencies to compute VBI: {stg.freq_text[int(stg.frequencies_to_compute_VBI[0, 0])]}, "
# f"{stg.freq_text[int(stg.frequencies_to_compute_VBI[1, 0])]} \n"
# f" frequency to compute SSC: {stg.freq_text[int(stg.frequency_to_compute_SSC[0])]}")

View File

@ -1,9 +1,7 @@
import sys
from PyQt5.QtGui import QIcon, QPixmap
from PyQt5.QtWidgets import (QWidget, QLabel, QHBoxLayout, QVBoxLayout, QApplication, QMainWindow, QGridLayout,
QDialog, QDialogButtonBox, QPushButton, QTextEdit, QFrame, QTabWidget, QScrollArea)
from PyQt5.QtCore import Qt
QDialog, QFrame, QTabWidget, QScrollArea)
import numpy as np
@ -12,13 +10,8 @@ from matplotlib.backends.backend_qt5agg import FigureCanvasQTAgg as FigureCanvas
from matplotlib.backends.backend_qt5agg import NavigationToolbar2QT as NavigationToolBar
from matplotlib.colors import LogNorm, BoundaryNorm
import datetime
import settings as stg
from Translation.constant_string import HORIZONTAL
from settings import depth_cross_section
class PlotNoiseWindow(QDialog):
@ -56,12 +49,10 @@ class PlotNoiseWindow(QDialog):
val_min = np.nanmin(stg.BS_noise_raw_data[i][freq_ind, :, :])
val_max = np.nanmax(stg.BS_noise_raw_data[i][freq_ind, :, :])
print("val_min = ", val_min, "val_max = ", val_max)
if val_min == val_max:
exec("pcm = self.ax" + str(i) + "[" + str(freq_ind) + "]" + ".pcolormesh(" +
"stg.time[" + str(i) + "][" + str(freq_ind) + ", :]," +
"-stg.depth[" + str(i) + "][" + str(freq_ind) + ", :]," +
"stg.time_noise[" + str(i) + "][" + str(freq_ind) + ", :]," +
"-stg.depth_noise[" + str(i) + "][" + str(freq_ind) + ", :]," +
"stg.BS_noise_raw_data[" + str(i) + "][" + str(freq_ind) + ", :, :]," +
"cmap='hsv')")
else:
@ -72,74 +63,6 @@ class PlotNoiseWindow(QDialog):
"-stg.depth_noise[" + str(i) + "][" + str(freq_ind) + ", :]," +
"stg.BS_noise_raw_data[" + str(i) + "][" + str(freq_ind) + ", :, :]," +
"cmap='hsv')")
# , norm = LogNorm(vmin=val_min, vmax=val_max)
# if stg.time_cross_section[i].shape != (0,):
#
# if depth_cross_section[i].shape != (0,):
# if val_min == val_max:
# exec("pcm = self.ax" + str(i) + "[" + str(freq_ind) + "]" + ".pcolormesh(" +
# "stg.time_cross_section[" + str(i) + "][" + str(freq_ind) + ", :]," +
# "-stg.depth_cross_section[" + str(i) + "][" + str(freq_ind) + ", :]," +
# "stg.BS_noise_raw_data[" + str(i) + "][" + str(freq_ind) + ", :, :]," +
# "cmap='viridis')" )
# else:
# val_min = 0
# val_max = 1e-5
# exec("pcm = self.ax" + str(i) + "[" + str(freq_ind) + "]" + ".pcolormesh(" +
# "stg.time_cross_section[" + str(i) + "][" + str(freq_ind) + ", :]," +
# "-stg.depth_cross_section[" + str(i) + "][" + str(freq_ind) + ", :]," +
# "stg.BS_noise_raw_data[" + str(i) + "][" + str(freq_ind) + ", :, :]," +
# "cmap='viridis', norm=LogNorm(vmin=val_min, vmax=val_max))")
# else:
# if val_min == val_max:
# exec("pcm = self.ax" + str(i) + "[" + str(freq_ind) + "]" + ".pcolormesh(" +
# "stg.time_cross_section[" + str(i) + "][" + str(freq_ind) + ", :]," +
# "-stg.depth[" + str(i) + "][" + str(freq_ind) + ", :]," +
# "stg.BS_noise_raw_data[" + str(i) + "][" + str(freq_ind) + ", :, :]," +
# "cmap='viridis')" )
# else:
# val_min = 0
# val_max = 1e-5
# exec("pcm = self.ax" + str(i) + "[" + str(freq_ind) + "]" + ".pcolormesh(" +
# "stg.time_cross_section[" + str(i) + "][" + str(freq_ind) + ", :]," +
# "-stg.depth[" + str(i) + "][" + str(freq_ind) + ", :]," +
# "stg.BS_noise_averaged_data[" + str(i) + "][" + str(freq_ind) + ", :, :]," +
# "cmap='viridis', norm=LogNorm(vmin=val_min, vmax=val_max))")
#
# else:
#
# if depth_cross_section[i].shape != (0,):
# if val_min == val_max:
# exec("pcm = self.ax" + str(i) + "[" + str(freq_ind) + "]" + ".pcolormesh(" +
# "stg.time[" + str(i) + "][" + str(freq_ind) + ", :]," +
# "-stg.depth_cross_section[" + str(i) + "][" + str(freq_ind) + ", :]," +
# "stg.BS_noise_averaged_data[" + str(i) + "][" + str(freq_ind) + ", :, :]," +
# "cmap='viridis')" )
# else:
# val_min = 0
# val_max = 1e-5
# exec("pcm = self.ax" + str(i) + "[" + str(freq_ind) + "]" + ".pcolormesh(" +
# "stg.time[" + str(i) + "][" + str(freq_ind) + ", :]," +
# "-stg.depth_cross_section[" + str(i) + "][" + str(freq_ind) + ", :]," +
# "stg.BS_noise_averaged_data[" + str(i) + "][" + str(freq_ind) + ", :, :]," +
# "cmap='viridis', norm=LogNorm(vmin=val_min, vmax=val_max))")
# else:
# if val_min == val_max:
# exec("pcm = self.ax" + str(i) + "[" + str(freq_ind) + "]" + ".pcolormesh(" +
# "stg.time[" + str(i) + "][" + str(freq_ind) + ", :]," +
# "-stg.depth[" + str(i) + "][" + str(freq_ind) + ", :]," +
# "stg.BS_noise_averaged_data[" + str(i) + "][" + str(freq_ind) + ", :, :]," +
# "cmap='viridis')" )
# else:
# val_min = 0
# val_max = 1e-5
# exec("pcm = self.ax" + str(i) + "[" + str(freq_ind) + "]" + ".pcolormesh(" +
# "stg.time[" + str(i) + "][" + str(freq_ind) + ", :]," +
# "-stg.depth[" + str(i) + "][" + str(freq_ind) + ", :]," +
# "stg.BS_noise_averaged_data[" + str(i) + "][" + str(freq_ind) + ", :, :]," +
# "cmap='viridis')")
# # , norm = LogNorm(vmin=val_min, vmax=val_max)
eval("self.ax" + str(i) + "[" + str(freq_ind) + "]" + ".text(1, .70, stg.freq_text[" + str(i) +
"][" + str(freq_ind) + "]," +
@ -160,8 +83,6 @@ class PlotNoiseWindow(QDialog):
pass
# self.axis_noise.tick_params(axis='both', which='minor', labelsize=10)
exec("self.canvas" + str(i) + "= FigureCanvas(self.fig" + str(i) + ")")
exec("self.toolbar" + str(i) + "= NavigationToolBar(self.canvas" + str(i) + ", self)")
@ -171,10 +92,3 @@ class PlotNoiseWindow(QDialog):
exec("self.verticalLayout_tab" + str(i) + ".addWidget(self.toolbar" + str(i) + ")")
exec("self.verticalLayout_tab" + str(i) + ".addWidget(self.scroll" + str(i) + ")")
# self.tab1 = QWidget()
# self.tab.addTab(self.tab1, "Tab 1")
# ----------------------------------------------------------

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

BIN
logos/AcouSed.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 43 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 54 KiB

BIN
logos/CNR.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 35 KiB

BIN
logos/EDF.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 22 KiB

BIN
logos/OSR.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.9 KiB

BIN
logos/Ubertone.jpeg Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.4 KiB

29
main.py
View File

@ -1,4 +1,5 @@
import sys
import logging
import traceback
from PyQt5.QtWidgets import QApplication, QMainWindow
@ -26,6 +27,14 @@ import matplotlib.pyplot as plt
PERCENT_SCREEN_SIZE = 0.85
_translate = QCoreApplication.translate
logging.basicConfig(
level=logging.INFO,
format=('[AcouSed][%(levelname)s] %(message)s')
)
logger = logging.getLogger("acoused")
logger.setLevel(logging.DEBUG)
#logger.setLevel(logging.INFO)
class MainApplication(QMainWindow):
@ -41,15 +50,17 @@ class MainApplication(QMainWindow):
height = size.height()
self.resize(int(PERCENT_SCREEN_SIZE*width), int(PERCENT_SCREEN_SIZE*height))
try:
self.read_table_open = ReadTableForOpen()
# **************************************************
# -------------- Acoustic data tab ---------------
self.acoustic_data_tab = AcousticDataTab(self.ui_mainwindow.tab1)
print("0 AcousticDataTab ", id(AcousticDataTab))
self.acoustic_data_tab.combobox_ABS_system_choice.editTextChanged.connect(
self.acoustic_data_tab.ABS_system_choice)
self.acoustic_data_tab\
.combobox_ABS_system_choice\
.editTextChanged\
.connect(
self.acoustic_data_tab.ABS_system_choice
)
# **************************************************
# --------- Signal pre-processing data tab ----------
@ -84,20 +95,19 @@ class MainApplication(QMainWindow):
# self.user_manual_tab = UserManualTab(self.ui_mainwindow.tab7)
self.ui_mainwindow.actionOpen.triggered.connect(self.trig_open)
# **************************************************
# ---------------- Text File Error -----------------
except Exception as e:
logger.error(str(e))
logger.error(traceback.format_exc())
with open("Error_file.txt", "w", encoding="utf-8") as sortie:
sortie.write(str(e))
sortie.write(traceback.format_exc())
# traceback.TracebackException.from_exception(e).print(file=sortie)
def trig_open(self):
self.read_table_open.open_file_dialog()
def open_study_update_tabs(self):
self.acoustic_data_tab.combobox_ABS_system_choice.setCurrentText(stg.ABS_name[0])
self.acoustic_data_tab.fileListWidget.addFilenames(stg.filename_BS_raw_data)
@ -108,7 +118,6 @@ class MainApplication(QMainWindow):
self.sample_data_tab.lineEdit_fine_sediment.setToolTip(stg.path_fine)
# self.sample_data_tab.fill_table_fine()
if __name__ == '__main__':
# print("sys.argv:", [arg for arg in sys.argv])
# app = MainApplication(sys.argv)

View File

@ -210,4 +210,4 @@ user-defined extensions).")
"python-scipy" "python-scikit-learn"
"python-pyqt@5" "python-pyqt5-sip"
"python-numpy@1" "python-pandas@1.5"
"python-matplotlib"))))
"python-matplotlib" "python-odfpy"))))

5
packages/debug.bat Normal file
View File

@ -0,0 +1,5 @@
@ECHO OFF
start cmd /c test3\Acoused.exe

35
packages/windows.bat Normal file
View File

@ -0,0 +1,35 @@
@ECHO OFF
rem Python environment (-U = update python packages / -r = texte file)
python -m pip install -U -r ..\virtualenv\requirements.txt
rem Build windows version
mkdir acoused_packaging
pyinstaller --name "acoused" ..\main.py -y
rem Icons
mkdir acoused_packaging\icons
copy /y ..\icons\*.png acoused_packaging\icons
rem Logos
mkdir acoused_packaging\logos
copy /y ..\logos\* acoused_packaging\logos
rem Doc
copy /y ..\ABS_calibration_constant_kt.xlsx acoused_packaging
copy /y ..\AcouSed_UserManual.pdf acoused_packaging
copy /y ..\Acoustic_Inversion_theory.pdf acoused_packaging
copy /y ..\Tutorial_AQUAscat_software.pdf acoused_packaging
rem move exe
move /y dist\AcouSed\acoused.exe acoused_packaging
move /y dist\acoused\_internal acoused_packaging
copy debug.bat acoused_packaging
rmdir /s /q build
rmdir /s /q dist
del /q AcouSed.spec
set PATH=%PATH%;C:\Program Files (x86)/7-Zip
7z a -tzip acoused_packaging.zip acoused_packaging

45
tools.py Normal file
View File

@ -0,0 +1,45 @@
import os
import time
import logging
import traceback
from datetime import datetime, timedelta
from pathlib import Path
from functools import wraps
###########
# LOGGING #
###########
logger = logging.getLogger("acoused")
#########
# WRAPS #
#########
def trace(func):
@wraps(func)
def wrapper(*args, **kwargs):
t = time.time()
head = f"[TRACE]"
logger.debug(
f"{head} Call {func.__module__}." +
f"{func.__qualname__}({args}, {kwargs})"
)
value = func(*args, **kwargs)
t1 = time.time()
logger.debug(
f"{head} Return {func.__module__}." +
f"{func.__qualname__}: {value}"
)
logger.debug(
f"{head}[TIME] {func.__module__}." +
f"{func.__qualname__}: {t1-t} sec"
)
return value
return wrapper

View File

@ -1,3 +1,5 @@
astropy==6.1.7
astropy-iers-data==0.2025.3.3.0.34.45
contourpy==1.0.7
cycler==0.11.0
defusedxml==0.7.1
@ -16,15 +18,21 @@ packaging==23.0
pandas==1.5.3
Pillow==9.4.0
profilehooks==1.12.0
pyerfa==2.0.1.5
pyparsing==3.0.9
pyqt-checkbox-table-widget==0.0.14
pyqt-file-list-widget==0.0.1
pyqt-files-already-exists-dialog==0.0.1
pyqt-tooltip-list-widget==0.0.1
PyQt5==5.15.9
PyQt5-Qt5==5.15.2
PyQt5-sip==12.11.0
python-dateutil==2.8.2
pytz==2022.7.1
PyYAML==6.0.2
scikit-learn==1.2.1
scipy==1.10.0
simplePyQt5==0.0.1
six==1.16.0
threadpoolctl==3.1.0
utm==0.7.0