CN114924706A - Display device, audio collection and transmission method thereof, and related equipment - Google Patents

Abstract

The present disclosure provides a display device, an audio acquisition and transmission method thereof, and related equipment, wherein the display device includes a plurality of first acquisition units disposed on a first side of the display device, and the method includes: acquiring a plurality of first audio data acquired by the plurality of first acquisition units; determining whether the position of a sound source is in a preset position range or not according to the plurality of first audio data; and in response to determining that the sound source position is within a preset position range, determining target audio data according to the plurality of first audio data and transmitting the target audio data to the target device.

Description

Display device, audio collection and transmission method thereof, and related equipment

technical field

The present disclosure relates to the field of display technology, and in particular, to a display device, a method for collecting and transmitting audio, and related equipment.

Background technique

In the related art, the display device and the sound pickup device are usually separated. In addition, the pickup equipment is prone to whistling due to audio interference.

SUMMARY OF THE INVENTION

The present disclosure proposes a display device, an audio collection and transmission method, and related equipment to solve or partially solve the above problems.

A first aspect of the present disclosure provides an audio collection and transmission method based on a display device, wherein the display device includes a plurality of first collection units disposed on a first side of the display device, and the method includes:

acquiring multiple first audio data collected by the multiple first collecting units;

determining whether the sound source position is within a preset position range according to the plurality of first audio data; and

In response to determining that the sound source position is within a preset position range, target audio data is determined according to the plurality of first audio data and sent to the target device.

In a second aspect of the present disclosure, there is provided a display device, comprising one or more processors, a memory; and one or more programs, wherein the one or more programs are stored in the memory and used by the Executed by one or more processors, the program includes instructions for performing the method according to the first aspect.

In a third aspect of the present disclosure, a display system is provided, including:

a plurality of display devices according to the second aspect; and

The central control system is configured to: register the display device and collect target audio data from the display device.

In a fourth aspect of the present disclosure, there is provided a non-volatile computer-readable storage medium containing a computer program, when the computer program is executed by one or more processors, the processor causes the processor to execute the first aspect Methods.

A fifth aspect of the present disclosure provides a computer program product, comprising computer program instructions, which, when the computer program instructions are run on a computer, cause the computer to execute the method of the first aspect.

The display device, the audio collection and transmission method, and the related equipment provided by the present disclosure utilize the audio data collected by a plurality of first collection units disposed on the first side of the display device to determine whether the sound source position is within a preset position range, If so, upload the audio data, thereby ensuring the quality of the audio data and reducing noise.

Description of drawings

In order to illustrate the technical solutions in the present disclosure or related technologies more clearly, the following briefly introduces the accompanying drawings used in the description of the embodiments or related technologies. Obviously, the drawings in the following description are only for the present disclosure. In the embodiments, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without any creative effort.

FIG. 1 shows a schematic diagram of an exemplary system provided by an embodiment of the present disclosure.

FIG. 2A shows a schematic diagram of an exemplary workflow in a terminal registration mode according to an embodiment of the present disclosure.

FIG. 2B shows a schematic diagram of an exemplary workflow in a data transmission mode according to an embodiment of the present disclosure.

FIG. 3A shows a schematic diagram of an exemplary display device provided by an embodiment of the present disclosure.

FIG. 3B shows a schematic diagram of an exemplary display device provided by an embodiment of the present disclosure viewed from another angle.

FIG. 3C shows a schematic diagram of the positional relationship between the first acquisition unit and the sound source according to an embodiment of the present disclosure.

FIG. 4A shows a schematic diagram of the principle of effectively identifying the sound field range according to an embodiment of the present disclosure.

4B shows a schematic diagram of an internal sound system of an exemplary display device according to an embodiment of the present disclosure.

FIG. 5 shows a schematic flowchart of an exemplary method provided by an embodiment of the present disclosure.

FIG. 6 shows a schematic diagram of the hardware structure of an exemplary computer device provided by an embodiment of the present disclosure.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present disclosure clearer, the present disclosure will be further described in detail below with reference to the specific embodiments and the accompanying drawings.

It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should have the usual meanings understood by those with ordinary skill in the art to which the present disclosure belongs. "First", "second" and similar words used in the embodiments of the present disclosure do not denote any order, quantity or importance, but are only used to distinguish different components. "Comprises" or "comprising" and similar words mean that the elements or things appearing before the word encompass the elements or things recited after the word and their equivalents, but do not exclude other elements or things. Words like "connected" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "Down", "Left", "Right", etc. are only used to represent the relative positional relationship, and when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

FIG. 1 shows a schematic diagram of an exemplary system 100 provided by an embodiment of the present disclosure.

As shown in FIG. 1 , the system 100 may include a plurality of terminal devices, for example, terminal devices 300A to 300C. In some embodiments, the system 100 may be a display system, and the terminal device may be a display device. As an optional embodiment, the display system 100 can be applied to a conference scene, and the display device can be an electronic table card, which is used to provide corresponding participant information for each seat in the conference room. In some embodiments, the display device may be provided with a collection unit (eg, a microphone) for collecting audio data, so that the display device can collect the speeches of the participants, and there is no need to separately set up a sound pickup device, thereby reducing the number of conference equipment .

In some embodiments, as shown in FIG. 1 , the system 100 may further include a central control system 200 for registering each display device and collecting audio data from each display device. The central control system 200 can communicate with each display device in a wired or wireless manner. In order to facilitate the transmission of data and the arrangement and installation of equipment, the central control system 200 and each display device may use wireless communication to realize data transmission.

In the related art, since the sound transmission requires strong real-time performance, when the audio data is wirelessly transmitted, there is generally a one-to-one correspondence between the receiving end and the transmitting end. However, in a conference scene, many sound pickup devices generally need to be set up, and the central control system 200 needs to set up many transmitting modules to correspond to them, which makes the whole system extremely complicated. In addition, the addition and debugging of equipment require special personnel to be responsible, which is inconvenient to use.

In view of this, the embodiments of the present disclosure provide a system 100 that adopts an ad hoc network mode, and adopts a time division power control mode within an acceptable delay range to complete a mode in which one central control system 200 pairs multiple display devices (that is, a pair of display devices). multiple topologies), so that the central control system only needs to set up one transmitter and one receiver.

The system 100 in the ad hoc network mode is mainly divided into two working modes: first, a terminal registration mode; and second, a data transmission mode.

FIG. 2A shows a schematic diagram of an exemplary workflow 202 in a terminal registration mode according to an embodiment of the present disclosure.

When preparing equipment before the meeting, all display devices (eg, electronic table cards 102A-102C) in the system 100 can be turned on, and the display devices automatically enter the registration mode after being powered on to receive broadcast messages from the central control system 200 .

As shown in FIG. 2A , the central control system 200 may broadcast a registration message to these display devices, and the registration message may include the ID of the central control system 200 and the number of display devices that have been registered successfully. Optionally, the central control system 200 may broadcast the registration message at a preset time interval (eg, 1-2 s), so that the registration of the newly added display device can be completed at any time.

For the sake of clarity of expression, the following description takes the display device 300A among the plurality of display devices as the target display device.

After receiving the registration message, the target display device 300A may determine whether the ID of the central control system 200 is stored locally (eg, the register of the display device 300A) based on the ID of the central control system 200 in the registration message. . If the ID of the central control system 200 is locally stored in the target display device 300A, it means that the registration has been completed. If the ID of the central control system 200 is not stored locally in the target display device 300A, the target display device 300A may send a registration request to the central control system 200, and the registration request may include the ID of the target display device 300A.

After the central control system 200 receives the registration request of the target display device 300A, it can assign an identification code to the target display device 300A, the identification code is in one-to-one correspondence with the display device, and can be associated with the ID of the display device, Therefore, the corresponding display device can be determined according to the identification code.

In some embodiments, the identification code may be Arabic numerals and may be sorted according to the number of registered devices. For example, if the current number of registered devices is 2, when a new identification code is obtained, the corresponding identification code may be 3,

Next, the central control system 200 may broadcast the identification code of the target display device 300A together with the ID of the target display device 300A. After receiving the identification code and the ID, the target display device 300A confirms the identification by identifying the ID. If the code is an identification code assigned by the central control system 200, the identification code can be saved locally and a registration success message can be sent to the central control system 200 to confirm that the device registration is completed. It can be understood that the central control system 200 sends the message by broadcasting, which omits the handshake process between the central control system 200 and each display device, which can simplify the data transmission process and reduce the delay. Moreover, since the broadcast message includes the identification code and the corresponding device ID, when other display devices receive the broadcast message, they can know whether the corresponding identification code is the identification code assigned by the central control system 200 by identifying the ID, thereby It can be determined whether the identification code needs to be saved.

Further, after receiving the registration success message of the target display device 300A, the central control system 200 confirms that an identification code has been successfully allocated to the target display device 300A, and can locally store the associated information of the ID of the target display device 300A and its identification code. , and then perform the registration process of the next display device until the registration of all display devices is completed.

It can be understood that since the registration process is automatically entered after the display apparatus is powered on, there may be conflicts caused when multiple devices are registered at the same time. Therefore, in some embodiments, if the uplink channel between the display device and the central control system 200 is occupied, the display device may randomly generate a delayed registration time, and then initiate a registration request to the central control system 200 again when the time expires, until All registrations are completed. In this way, each display device in the ad hoc network can use this mechanism to automatically realize channel allocation, and automatically complete registration without human intervention, thereby improving system work efficiency.

In some embodiments, after each display device has been registered and entered the working state, in order to ensure the joining of the temporary device, the central control system 200 may broadcast the registration message at predetermined time intervals, so that the temporary device can broadcast the registration message based on the to complete the registration. In this way, even if the system 100 has entered the data collection stage, the temporary device can still complete the registration when the central control system 200 broadcasts the registration message next time. Optionally, the length of the predetermined time interval may be greater than one data collection period (that is, the time required to complete the data collection of all display devices), or greater than an integer multiple of the data collection period, so as to ensure that the completed data collection period is not affected. The registered display device works normally. In some embodiments, the data collection period is variable and can be adjusted according to the number of registered display devices, thereby improving channel utilization and reducing latency.

In some embodiments, after sending the registration success message, the target display device 300A can close the transmission channel (eg, wireless transmission channel) and enter the monitoring state, thereby reducing the occupation of the uplink channel, improving the channel utilization rate, and reducing the delay.

In this way, all registered display devices can share the same downlink channel and the same uplink channel, and the central control system 200 only needs one transmitter and one receiver to complete data transmission.

After all display devices are registered, the system 100 may enter data transfer mode.

FIG. 2B shows a schematic diagram of an exemplary workflow 204 in a data transfer mode according to an embodiment of the present disclosure.

As shown in FIG. 2B , after entering the data transmission mode, the central control system 200 can broadcast data collection messages in sequence according to the sequence of the identification codes, and the data collection messages include the identification codes in the current sequence. Assuming that the central control system 200 adds an identification code from the beginning to the data collection message, for example, the first sequence bit of the identification code is 1, which corresponds to the first registered device in the system 100, then the identification code included in the data collection message is "1".

Since the data collection message is broadcast, each display device in the system 100 can receive the message, and can judge whether it is its turn to upload data according to the identification code therein.

For example, if the target display device 300A receives the data collection message and determines whether the identification code of the current sequence bit in the data collection message is the identification code of the target display device 300A.

Assuming that the identification code of the target display device 300A is "1", the target display device 300A can determine that the identification code of the current sequence bit is its identification code, and can send the audio data collected by the target display device 300A to the central control system 200. In some embodiments, after the target display device 300A determines that the identification code of the current sequence bit is its identification code, the target display device 300A can open its transmission channel, increase the transmission power, and then send the audio data to the central control system 200.

After receiving the audio data sent by the target display device 300A, the central control system 200 generates and broadcasts a data collection message by using the identification code of the next sequential bit (for example, "2") to collect the identification code of the next sequential bit The audio data of the corresponding display device.

Since the data collection message contains the identification code instead of the ID of each display device, the data bits in the message can be saved, so that the communication efficiency can be improved and the delay can be reduced. In addition, each display device can upload data after recognizing its own identification code, and does not need to perform steps such as identification and authentication with the central control system 200. On the premise that the system 100 forms an ad hoc network, the security is guaranteed. At the same time, the communication efficiency is improved and the delay is reduced.

In some embodiments, since there are multiple display devices in the system 100, after receiving the audio data, the central control system 200 can use the audio data as a channel and transmit the audio data to the subsequent stage for subsequent synthesis of multi-channel audio Data processing, at the same time broadcast the next data collection message in the sequence of identification codes, until all identification codes are traversed. Then, in some embodiments, the registration message may be broadcast again to add new devices to the system 100 .

In some embodiments, the central control system 200 may send a data collection message based on a clock signal. When the uploaded audio data is not received within one clock cycle after the data collection message is sent, the central control system 200 may send the data collection message based on the next identification code. Continue sending data collection messages. At this time, the display device corresponding to the unreceived audio data is skipped, and the audio data can be re-uploaded after the next data collection cycle. In this way, even if there are damaged devices in the system 100 that cannot respond to this broadcast, the central control system can normally enter the next identification code broadcast, which does not affect the normal operation of the system 100 .

As an optional embodiment, when the central control system 200 does not receive corresponding audio data in a corresponding clock cycle, it may output prompt information or send prompt information to a designated device, where the prompt information may include the ID of the corresponding display device and the prompt content (For example, "the device is in an abnormal state"), thereby prompting the relevant personnel that a certain display device may be faulty and needs to be dealt with in time.

In some embodiments, in addition to the display device corresponding to the identification code of the data collection message, when other display devices receive the data collection message, the other display devices can also determine, according to the identification code in the data collection message, how far away it is from its own upload time point. For a long time, you can prepare in advance.

For example, assuming that the identification code of the current sequence bit is "1", and the display device with the identification code "3" receives the identification code "1", it can determine that its upload time point is 2 clock cycles later, and can After two clock cycles, audio data starts to be uploaded without monitoring the corresponding data collection message or identifying the identification code therein, thereby further improving communication efficiency and reducing delay. Moreover, since the monitoring function is turned off, the power consumption of the display device can also be saved.

As an optional embodiment, when the display device determines, according to the identification code in the received data collection message, that the time difference between the current time and its upload time point is less than a preset time threshold (for example, 1s), the monitor function can be turned off and wait for After the time difference elapses, the audio data starts to be uploaded; if it is determined that the time difference between the current time and the upload time point is greater than the preset time threshold, the monitoring function can be turned off after the preset time threshold has passed, and the monitoring function can be turned on again, and receive new data collection message to determine whether it is your turn to upload data. In this way, system errors and data upload errors or conflicts caused by the display device turning off the monitoring function for a long time are avoided.

According to the above embodiment, it can be seen that, due to sufficient preparation time before the meeting, the central control system 200 can mainly work in the terminal registration mode, the channel bandwidth is used to complete the registration of all devices, and the main work is in the data transmission mode after the meeting. In this way, the channel bandwidth is reasonably utilized, the utilization rate of the bandwidth audio data is improved, the throughput is increased, and the delay time can be reduced. The upstream data of the display device is mainly audio data and the downstream data is mainly the identification code, which reduces the number of responses and the overhead of non-audio data bits, and also increases the utilization rate of bandwidth audio data.

In the related art, when a microphone is used to speak in a conference, howling is likely to be formed due to the incoming sound from the speaker. To avoid this situation, the speaker generally needs to turn on the microphone switch before speaking and turn off the switch after speaking. When multiple people are speaking, other people's voices will be mixed in, causing interference and making the other end of the video conference unable to hear clearly.

In view of this, the embodiments of the present disclosure also provide a display device with a collection unit for collecting audio data, and design a corresponding audio collection and transmission method, which can improve the quality of audio data and avoid audio interference or even howling. question called.

FIG. 3A shows a schematic diagram of an exemplary display device 300A provided by an embodiment of the present disclosure.

Taking the display device 300A of FIG. 1 as an example, as shown in FIG. 3A , as an optional embodiment, the display device 300A may be an electronic table card, and may include a front case 302 and a rear case 304 . A first display screen 306 can be set in the front shell 302, and the first display screen 306 can be used to display participant information, for example, the relevant information of the participant sitting at the corresponding position of the electronic table card (for example, name, company , position, etc.). Optionally, four first collection units 308A to 308D (eg, microphones) may be arranged around the first display screen 306 for collecting sound to form the first audio data. The side provided with the first display screen 306 can face the participants in actual use, so that the four first acquisition units 308A to 308D formed as an array can better collect the voices of the participants to form corresponding voices. The audio data is uploaded to the central control system 200 .

In some embodiments, the rear case 304 may also be provided with a collection unit and a display screen for collecting audio data.

FIG. 3B shows a schematic diagram of an exemplary display device 300A provided by an embodiment of the present disclosure viewed from another angle.

As shown in FIG. 3B , a second display screen 310 may be provided in the rear case 304 , and the second display screen 310 may also be used to display participant information. The side provided with the second display screen 310 may face the opposite side of the participant in actual use, so that others can determine the identity of the participant by observing the information displayed on the second display screen 310 . As an optional embodiment, the included angle between the display surface of the first display screen 306 and the display surface of the second display screen 310 may be greater than or equal to 90° and less than 180°, so that the display device 300A sets the first display screen One side of 306 may preferably face the participants, and the side of the second display screen 310 may preferably face the opposite side of the participants, so as to better realize the role of electronic table cards.

In some embodiments, a second collection unit 312 (eg, a microphone) may be disposed above the rear case 304 for collecting sound to form second audio data. The second audio data may be ambient sound, so that the second audio data may be used to denoise the first audio data.

During the meeting, the electronic table card 300A completes the registration with the central control system 200 according to the foregoing embodiment, and can start to use the first collection units 308A to 308D to collect sounds. The front shell 302 of the electronic table card 300A faces the participants, so that the first acquisition units 308A-308D also face the participants, so that the clearer voices of the participants can be collected, which are then used to generate the first audio data.

As shown in FIG. 3A , the four first collection units 308A to 308D are respectively arranged around the first display screen 306 , so that the four first collection units 308A to 308D have a certain relative distance relationship. Using the relative distance relationship, combined with The speed of sound propagation can be determined according to the collected first audio data whether the sound source position is within the preset position range. If the sound source position is within the preset position range, it means that the first audio data is from the participant, and Target audio data is obtained according to the collected first audio data, and sent to the target device (for example, the central control system 200 ). If the position of the sound source is outside the preset position range, it means that the first audio data is from someone other than the participants, and the collected first audio data can be discarded.

FIG. 3C shows a schematic diagram of the positional relationship between the first acquisition unit and the sound source according to an embodiment of the present disclosure. As shown in FIG. 3C , the first collection units 308B and 308D are arranged along a first direction (eg, lateral direction) with an interval of L ₂ . Assuming that the distance between the sound source 314 and the first acquisition unit 308B is l ₁ , and the distance between the sound source 314 and the first acquisition unit 308D is l ₂ , in this way, when the position of the sound source 314 is not on the vertical center axis, l ₁ ≠l _2. The time sequence in which the sound emitted by the sound source 314 propagates to the two first collection units 308B and 308D is different. For example, by comparing the first audio data collected by the two first collecting units 308B and 308D, the time difference Δt ₁ between the two can be determined. Multiplying the time difference Δt ₁ by the speed of sound υ (approximately equal to the speed of sound in a vacuum of 340 m/s), the distance difference Δt ₁ ·υ between l ₁ and l ₂ can be obtained.

According to the positional relationship between the sound source 314 and the first acquisition units 308B and 308D, when the sound source 314 is located on the central axis, l ₁ and l ₂ are equal. Therefore, when the distance difference between l ₁ and l ₂ is larger, the sound The farther the source 314 is from the central axis, the angle α gradually decreases, and Δt ₁ ·υ approaches L ₂ , and when the distance difference Δt ₁ ·υ is greater than a certain first distance threshold (or, Δt ₁ ·υ and L When the ratio of ₂ is greater than a certain first preset ratio), it can be considered that the position of the sound source exceeds the preset position range, and the collected first audio data can be discarded.

Continuing to refer to FIG. 3C , the first collection units 308A and 308C are arranged in a second direction (eg, longitudinal direction) with a spacing L ₁ . Assuming that the distance between the sound source 314 and the first acquisition unit 308A is l ₃ , and the distance between the sound source 314 and the first acquisition unit 308C is l ₄ , in this way, when the position of the sound source 314 is not on the horizontal central axis, l ₃ ≠l _4. The time sequence in which the sound emitted by the sound source 314 propagates to the two first collecting units 308A and 308C is different. For example, by comparing the first audio data collected by the two first collecting units 308A and 308C, the time difference Δt ₂ between the two can be determined. Multiplying the time difference Δt ₂ by the speed of sound υ (approximately equal to the speed of sound in a vacuum of 340 m/s), the distance difference Δt ₂ ·υ between l ₃ and l ₄ can be obtained.

According to the positional relationship between the sound source 314 and the first acquisition units 308A and 308C, when the sound source 314 is located _on the vertical central axis, the distance difference Δt _{2 ·} υ between l3 and _l4 is equal to L1, therefore, when the sound source 314 The more the deviation from the vertical central axis, the sound source 314 is about far from the central axis, the angle β gradually increases, and Δt ₂ ·υ will become smaller and smaller, and then when the distance difference Δt ₂ ·υ is smaller than a certain second distance threshold ( Alternatively, when the ratio of Δt ₂ ·υ to L ₁ is smaller than a certain second preset ratio), it can be considered that the sound source position exceeds the preset position range, and the collected first audio data can be discarded.

In addition, according to the geometric formula, the following relationship can be obtained:

|l ₁ -l ₂ |=Δt ₁ ·υ

|l ₃ -l ₄ |=Δt ₂ ·υ

In the above formula, Δt ₁ , Δt ₂ , υ, L ₁ , and L ₂ are all known numbers, so l ₁ , l ₂ , l ₃ , l ₄ , and D can be solved, and then it can be calculated from the perspective of the actual distance. It is determined whether to discard the first audio data.

As an optional embodiment, the display device 300A may retain the first audio data when the distance difference Δt ₁ ·υ and the distance difference Δt ₂ ·υ both meet the threshold requirements, thereby ensuring the collected voice data of the participants the quality of.

FIG. 4A shows a schematic diagram of the principle of effectively identifying the sound field range according to an embodiment of the present disclosure.

As shown in FIG. 4A , in the conference scenario, the participants are sitting in front of the conference table 402, the electronic table card 300A is placed on the conference table 402, and the side of the electronic table card 300A on which the first collection unit is set faces the participant, forming a A preset position range 404 of the sound source position is set. This range 404 is the effective sound field range for identification. According to the usage habits, this range just surrounds the participants. The sound within this range will be recorded effectively, and the sound outside this range will be recorded. It is greatly attenuated or shielded, so as to ensure that the voice of the participants is picked up, recorded and transmitted to the central control system 200 by the system.

The side of the electronic table card 300A on which the second collection unit is disposed faces the opposite side of the participant, and the noise or background sound 406 in front of it can be collected.

In some embodiments, the display apparatus 300A may acquire the second audio data collected by the second collection unit; and then determine the target audio data according to the plurality of first audio data and the second audio data and send it to the target device (for example, the central control system 200).

As shown in Figure 4A, when the volume of the sound in the direction of the rear shell is too large, the sound pickup effect of the MIC array of the front shell will be weakened. Therefore, after the second audio data enters the system, it can be subtracted from the first audio data (noise reduction processing). ), thereby weakening noise or background sound in the first audio data picked up by the first acquisition unit, and further achieving the purpose of eliminating noise.

FIG. 4B shows a schematic diagram of the internal sound system 408 of an exemplary display device according to an embodiment of the present disclosure.

In some embodiments, as shown in FIG. 4B, the internal sound system of the display device 300A, in addition to the sound pickup, noise cancellation and transmission mentioned above, also has a recording system, through the electronic table card itself or an additional MCU and Flash, record all the speeches in the whole meeting and store them in Flash. At the end of the meeting, the speaker signs and confirms through the screen of the table card. This signature label will be saved together with the recorded voice, and will be uploaded to the central bank after the meeting. control system 200.

The display device with directional sound pickup and recording functions provided by the embodiment of the present disclosure has a table card display function, and the display content can be updated through wireless data; the front shell is provided with a MIC array, which can generate a specific effective sound pickup area in front of the sign, and can Accurately identify the speaker in this area, pick up and record the speaker's voice, and the rear shell MIC can be used to eliminate the sound outside the effective identification area. In addition, the electronic table card has the function of self-organizing network, all the table cards share the wireless channel, the central control system uses the downlink channel to broadcast the message, the electronic table card completes the registration by monitoring the message, and uploads the voice in the order assigned in the message to realize the conference scene or similar Scene sound control.

The display device provided by the embodiment of the present disclosure can accurately and clearly pick up the voice of the speaker, and can eliminate the whistling of the conference scene, improve the conference quality, and at the same time, reduce the number of conference equipment. The display system provided by the embodiment of the present disclosure, the device self-organizing network, can reduce the difficulty of use and facilitate maintenance, and the display device uploads data by means of time division power control, which can reduce data delay.

Embodiments of the present disclosure also provide an audio collection and transmission method based on a display device. FIG. 5 shows a schematic flowchart of an exemplary method 500 provided by an embodiment of the present disclosure. The method 500 may be applied to the display devices 300A˜ 300C of FIG. 1 , and as shown in FIG. 5 , the method 500 may further include the following steps.

In step 502, a plurality of first audio data collected by the plurality of first collection units (eg, the first collection units 308A to 308D of FIG. 3A ) are acquired.

In step 504, according to the plurality of first audio data, it is determined whether the sound source position is within a preset position range (eg, the range 404 of FIG. 4A).

In some embodiments, the plurality of first collection units include two first collection units (eg, first collection units 308B and 308D of FIG. 3C ) arranged along the first direction; according to the plurality of first collection units For audio data, determining whether the sound source position is within a preset position range includes: according to the two first audio data corresponding to the two first collection units arranged along the first direction, determining the sound source position and the the first distance difference between the two first collection units arranged along the first direction; and determining whether the sound source position is within a preset position range according to the relationship between the first distance difference and the first distance threshold.

In some embodiments, according to the two pieces of the first audio data corresponding to the two first collection units arranged along the first direction, the position of the sound source and the two pieces arranged along the first direction are determined. The first distance difference between the first collection units includes: determining, according to the two pieces of first audio data corresponding to the two first collection units arranged along the first direction, the The two first collection units of the cloth collect the first time difference of the corresponding two first audio data; and determine the first distance difference according to the first time difference and the speed of sound.

In some embodiments, the plurality of first collection units include two first collection units (eg, first collection units 308A and 308C of FIG. 3C ) arranged along the second direction; according to the plurality of first collection units For audio data, determining whether the sound source position is within a preset position range includes: according to the two first audio data corresponding to the two first collection units arranged along the second direction, determining the sound source position and the second distance difference between the two first collection units arranged along the second direction; and determining whether the sound source position is within a preset position range according to the relationship between the second distance difference and the second distance threshold.

In some embodiments, according to the two pieces of the first audio data corresponding to the two first collection units arranged along the second direction, determine the position of the sound source and the two pieces arranged along the second direction. The second distance difference between the first collection units includes: determining, according to the two pieces of first audio data corresponding to the two first collection units arranged along the second direction, the The two first collection units of the cloth collect the second time difference of the corresponding two first audio data; and determine the second distance difference according to the second time difference and the speed of sound.

In some embodiments, the display device further includes a second acquisition unit (eg, the second acquisition unit 312 in FIG. 3B ) disposed on the second side of the display device; the method further includes: acquiring the first acquisition unit Second audio data collected by the collection unit; the determining target audio data according to the plurality of first audio data and sending it to the target device includes: determining according to the plurality of first audio data and the second audio data target audio data and send it to the target device.

In some embodiments, the angle between the orientation of the first side (eg, the side where the first display screen 306 is located) and the orientation of the second side (eg, the side where the second display screen 310 is located) is Greater than or equal to 90° and less than 180°.

In step 506 , in response to determining that the sound source position is within a preset position range, target audio data is determined according to the plurality of first audio data and sent to a target device (eg, the central control system 200 of FIG. 1 ).

In the audio collection and transmission method based on a display device provided by the embodiment of the present disclosure, the front shell of the display device is provided with a MIC array, which can generate a specific effective pickup area in front of the sign, accurately identify the speaker in this area, pick up and record Speaker sound, while the rear shell MIC can be used to eliminate the sound outside the effective recognition area. In addition, the electronic table card has the function of self-organizing network, all the table cards share the wireless channel, the central control system uses the downlink channel to broadcast the message, the electronic table card completes the registration by monitoring the message, and uploads the voice in the order assigned in the message to realize the conference scene or similar Scene sound control.

It should be noted that, the methods of the embodiments of the present disclosure may be executed by a single device, such as a computer or a server. The method in this embodiment can also be applied in a distributed scenario, and is completed by the cooperation of multiple devices. In the case of such a distributed scenario, one device among the multiple devices may only perform one or more steps in the method of the embodiment of the present disclosure, and the multiple devices will interact with each other to complete all the steps. method described.

It should be noted that some embodiments of the present disclosure are described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that in the above-described embodiments and still achieve desirable results. Additionally, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Embodiments of the present disclosure also provide a computer device for implementing the above method 500 . FIG. 6 shows a schematic diagram of the hardware structure of an exemplary computer device 600 provided by an embodiment of the present disclosure. The computer apparatus 600 may be used to implement the display apparatuses 300A to 300C of FIG. 1 . In some scenarios, the computer device 600 may also be used to implement the central control system 200 of FIG. 1 .

As shown in FIG. 6 , computer device 600 may include: processor 602 , memory 604 , network module 606 , peripheral interface 608 , and bus 610 . Among them, the processor 602 , the memory 604 , the network module 606 and the peripheral interface 608 realize the communication connection among each other in the computer device 600 through the bus 610 .

The processor 602 may be a central processing unit (Central Processing Unit, CPU), an image processor, a neural network processor (NPU), a microcontroller (MCU), a programmable logic device, a digital signal processor (DSP), an application-specific An integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits. The processor 602 may be used to perform functions related to the techniques described in this disclosure. In some embodiments, processor 602 may also include multiple processors integrated into a single logical component. For example, as shown in FIG. 6, the processor 602 may include a plurality of processors 602a, 602b, and 602c.

Memory 604 may be configured to store data (eg, instructions, computer code, etc.). As shown in FIG. 6 , the data stored in the memory 604 may include program instructions (eg, program instructions for implementing methods of embodiments of the present disclosure) and data to be processed (eg, the memory may store configuration files of other modules, etc.). The processor 602 may also access program instructions and data stored by the memory 604 and execute the program instructions to operate on the data to be processed. Memory 604 may include volatile storage or non-volatile storage. In some embodiments, memory 604 may include random access memory (RAM), read only memory (ROM), optical disks, magnetic disks, hard disks, solid state drives (SSDs), flash memory, memory sticks, and the like.

Network interface 606 may be configured to provide computer device 600 with communication with other external devices via a network. The network can be any wired or wireless network capable of transmitting and receiving data. For example, the network may be a wired network, a local wireless network (eg, Bluetooth, WiFi, Near Field Communication (NFC), etc.), a cellular network, the Internet, or a combination thereof. It can be understood that the type of network is not limited to the above specific examples.

The peripheral interface 608 may be configured to connect the computer device 600 with one or more peripheral devices for information input and output. For example, peripheral devices may include input devices such as keyboards, mice, touch pads, touch screens, microphones, and various sensors, and output devices such as displays, speakers, vibrators, and indicator lights.

Bus 610 may be configured to transfer information between various components of computer device 600 (eg, processor 602, memory 604, network interface 606, and peripheral interface 608), such as an internal bus (eg, a processor-memory bus), an external bus (USB port, PCI-E bus), etc.

It should be noted that although the architecture of the computer device 600 above only shows the processor 602, the memory 604, the network interface 606, the peripheral interface 608 and the bus 610, in a specific implementation process, the architecture of the computer device 600 may also include Other components necessary for proper operation. In addition, those skilled in the art can understand that the architecture of the above-mentioned computer device 600 may only include components necessary to implement the solutions of the embodiments of the present disclosure, rather than all the components shown in the figures.

Based on the same inventive concept and corresponding to any of the above-mentioned embodiments, the present disclosure also provides a non-transitory computer-readable storage medium, where the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions use In order to cause the computer to execute the method 500 described in any of the above embodiments.

The computer readable medium of this embodiment includes both permanent and non-permanent, removable and non-removable media and can be implemented by any method or technology for information storage. Information may be computer readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Flash Memory or other memory technology, Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the method 500 described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

Based on the same inventive concept, and corresponding to the method 500 in any of the foregoing embodiments, the present disclosure also provides a computer program product, which includes a computer program. In some embodiments, the computer program is executable by one or more processors to cause the processors to perform the method 500 described. Corresponding to the execution body corresponding to each step in each embodiment of the method 500, the processor that executes the corresponding step may belong to the corresponding execution body.

The computer program product of the above embodiment is used to cause the processor to execute the method 500 described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

It should be understood by those of ordinary skill in the art that the discussion of any of the above embodiments is only exemplary, and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples; under the spirit of the present disclosure, the above embodiments or Technical features in different embodiments may also be combined, steps may be implemented in any order, and there are many other variations of the different aspects of the disclosed embodiments as described above, which are not provided in detail for the sake of brevity.

In addition, to simplify illustration and discussion, and in order not to obscure the embodiments of the present disclosure, well-known power/power sources associated with integrated circuit (IC) chips and other components may or may not be shown in the figures provided in the figures provided. ground connection. Furthermore, devices may be shown in block diagram form in order to avoid obscuring the disclosed embodiments, and this also takes into account the fact that details regarding the implementation of these block diagram devices are highly dependent on the implementation of the disclosed embodiments platform (ie, these details should be well within the understanding of those skilled in the art). Where specific details (eg, circuits) are set forth to describe exemplary embodiments of the present disclosure, it will be apparent to those skilled in the art that these specific details may be used without or with variations The embodiments of the present disclosure are implemented as follows. Accordingly, these descriptions are to be considered illustrative rather than restrictive.

Although the present disclosure has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations to these embodiments will be apparent to those of ordinary skill in the art from the foregoing description. For example, other memory architectures (eg, dynamic RAM (DRAM)) may use the discussed embodiments.

The disclosed embodiments are intended to cover all such alternatives, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omission, modification, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiments of the present disclosure should be included within the protection scope of the present disclosure.

Claims (15)

1. A display device based audio capture and transmission method, wherein the display device comprises a plurality of first capture units disposed on a first side of the display device, the method comprising:

acquiring a plurality of first audio data acquired by the plurality of first acquisition units;

determining whether the sound source position is in a preset position range or not according to the plurality of first audio data; and

and in response to determining that the sound source position is in a preset position range, determining target audio data according to the plurality of first audio data and transmitting the target audio data to the target device.

2. The method of claim 1, wherein the plurality of first acquisition units comprises two first acquisition units arranged along a first direction; determining whether the sound source position is in a preset position range according to the plurality of first audio data, wherein the determining comprises the following steps:

determining a first distance difference between the sound source position and the two first acquisition units arranged along the first direction according to the two first audio data corresponding to the two first acquisition units arranged along the first direction; and

and determining whether the sound source position is in a preset position range according to the relation between the first distance difference and a first distance threshold.

3. The method of claim 1 or 2, wherein the plurality of first acquisition units comprises two first acquisition units arranged along the second direction; determining whether the sound source position is in a preset position range according to the plurality of first audio data, wherein the determining comprises the following steps:

determining a second distance difference between the sound source position and the two first acquisition units arranged along the second direction according to the two first audio data corresponding to the two first acquisition units arranged along the second direction; and

and determining whether the sound source position is in a preset position range or not according to the relation between the second distance difference and a second distance threshold.

4. The method of claim 2, wherein determining a first distance difference between the sound source position and two first acquisition units arranged in the first direction from two of the first audio data corresponding to the two first acquisition units arranged in the first direction comprises:

determining a first time difference between the two first audio data acquired by the two first acquisition units arranged along the first direction according to the two first audio data corresponding to the two first acquisition units arranged along the first direction; and

and determining the first distance difference according to the first time difference and the speed of sound.

5. The method of claim 3, wherein determining a second distance difference between the sound source position and two first acquisition units arranged in the second direction according to two first audio data corresponding to the two first acquisition units arranged in the second direction comprises:

determining a second time difference between the two first audio data collected by the two first collecting units arranged along the second direction according to the two first audio data corresponding to the two first collecting units arranged along the second direction; and

and determining the second distance difference according to the second time difference and the speed of sound.

6. The method of claim 1, wherein the display device further comprises a second acquisition unit disposed on a second side of the display device; the method further comprises the following steps: acquiring second audio data acquired by the second acquisition unit;

the determining and sending the target audio data to the target device according to the plurality of first audio data comprises: and determining target audio data according to the plurality of first audio data and the second audio data and sending the target audio data to the target equipment.

7. The method of claim 6, wherein an angle between orientations of the first side and the second side is greater than or equal to 90 ° and less than 180 °.

8. A display device comprising one or more processors, memory; and one or more programs, wherein the one or more programs are stored in the memory and executed by the one or more processors, the programs comprising instructions for performing the method of any of claims 1-7.

9. The display device of claim 8, wherein the display device is an electronic table card.

10. A display system, comprising:

a plurality of display devices as claimed in claim 8 or 9; and

a central control system configured to: registering the display device and collecting target audio data from the display device.

11. The display system of claim 10, wherein the central control system is configured to: broadcasting a registration message including an ID of the central control system and the number of display devices that have been successfully registered to the plurality of display devices including a target display device;

the target display apparatus configured to: receiving the registration message and determining whether to store the ID of the central control system; and in response to determining that the ID of the central control system is not stored, sending a registration request to the central control system, the registration request including the ID of the target display device;

the central control system is further configured to: receiving the registration request and distributing an identification code for the target display device; and sending the identification code together with the ID of the target display device to the target display device;

the target display apparatus is further configured to: receiving the identification code and the ID of the target display device; and storing the identification code and sending a registration success message to the central control system.

12. The display system of claim 11, wherein the target display device is further configured to: and after the registration success message is sent, closing a transmission channel and entering a monitoring state.

13. The display system of claim 11, wherein the central control system is further configured to: broadcasting data acquisition messages in sequence according to the sequence of the identification codes, wherein the data acquisition messages comprise the identification codes in the current sequence position;

the target display apparatus is further configured to: receiving the data acquisition message and determining whether the identification code of the current sequence bit is the identification code of the target display device; and in response to determining that the identification code of the current sequence bit is the identification code of the target display device, sending the acquired target audio data to the central control system.

14. A non-transitory computer-readable storage medium containing a computer program which, when executed by one or more processors, causes the processors to perform the method of any one of claims 1-7.

15. A computer program product comprising computer program instructions which, when run on a computer, cause the computer to perform the method of any one of claims 1-7.

Images