CN108803859A - Information processing method, device, terminal, earphone and readable storage medium - Google Patents

Abstract

The application relates to an information processing method, an information processing device, a terminal, an earphone and a readable storage medium. The method comprises the following steps: based on the audio signal played by the audio processing device, acquiring an acoustic feature model associated with the internal structure of the ear of the current user; acquiring the use state of the audio processing device according to the acoustic feature model of the current user; generating a corresponding control instruction according to the use state of the audio processing device; and controlling a terminal communicating with the audio processing device according to the control instruction, and finishing instruction interactive operation corresponding to the control instruction. By the method, intelligent audio playing control can be realized according to the using state of the audio processing device, terminal power consumption can be saved when a user does not need to use the audio processing device, complicated playing control operation is reduced for the user, and user experience is improved.

Description

Information processing method, device, terminal, earphone and readable storage medium

technical field

The present application relates to the field of communication technologies, and in particular to an information processing method, device, terminal, earphone and readable storage medium.

Background technique

With the intelligent development of communication equipment, people use smart terminals more and more frequently in their daily life. Through smart terminals, various activities such as video, call, voice, listening to music and video playback can be performed. Users usually wear earphones to receive the sound on the terminal. When users use earphones to listen to music and answer calls, some control operations are involved, such as playing, pausing, adjusting volume, switching songs, opening contacts, etc. It can be completed by operating on the relevant page of the corresponding application on the Internet, or by operating the preset control buttons on the headset.

Existing earphones cannot intelligently recognize the user to play songs automatically. Every time the user puts on the earphones, he has to manually turn on the music player to select the song he wants to play, and every time he takes off the earphones, he has to manually turn off the music player. The user experience is not good.

Contents of the invention

Embodiments of the present application provide an information processing method, device, terminal, earphone, and readable storage medium, which can control audio playback according to the usage status of the audio processing device, and improve human-computer interaction experience.

An information processing method, comprising:

Acquiring an acoustic feature model associated with the internal structure of the current user's ear based on the audio signal played by the audio processing device;

Obtaining the use status of the audio processing device according to the current user's acoustic feature model;

generating corresponding control instructions according to the usage state of the audio processing device;

The terminal communicating with the audio processing device is controlled according to the control instruction, and an instruction interaction operation corresponding to the control instruction is completed.

An information processing device, comprising:

A feature model acquisition module, configured to acquire an acoustic feature model associated with the internal structure of the current user's ear based on the audio signal played by the audio processing device;

A use status acquisition module, configured to acquire the use status of the audio processing device according to the current user's acoustic feature model;

A control command generating module, configured to generate corresponding control commands according to the usage state of the audio processing device;

The control instruction execution module is configured to control the terminal communicating with the audio processing device according to the control instruction, and complete the instruction interaction operation corresponding to the control instruction.

A terminal includes a memory and a processor, where computer-readable instructions are stored in the memory, and when the instructions are executed by the processor, the processor is made to execute the steps of the above method.

A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above method are realized.

An earphone comprising an electroacoustic transducer, a memory, a processor and a computer program stored on the memory and operable on the processor, the processor is electrically connected to the electroacoustic transducer and the memory, and the processor executes The computer program realizes the steps of the above method.

The above information processing method, device, terminal, earphone, and computer-readable storage medium obtain an acoustic feature model associated with the internal structure of the current user's ear based on the audio signal played by the audio processing device; obtain the acoustic feature model based on the current user's acoustic feature model The use state of the audio processing device; generate a corresponding control instruction according to the use state of the audio processing device; control the terminal communicating with the audio processing device according to the control instruction, and complete the corresponding control instruction command interaction. Through the above method, intelligent audio playback control can be realized according to the use state of the audio processing device, and the power consumption of the terminal can be saved when the user does not need to use the audio processing device, and the user can reduce cumbersome playback control operations and improve user experience.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present application. Those skilled in the art can also obtain other drawings based on these drawings without creative work.

Fig. 1 is a schematic diagram of an application environment of an information processing method in an embodiment;

Fig. 2 is a schematic diagram of the internal structure of a terminal in an embodiment;

Fig. 3 is a schematic flow chart of an information processing method in an embodiment;

Fig. 4 is a schematic flow chart of an information processing method in another embodiment;

Fig. 5 is a schematic flow chart of an information processing method in another embodiment;

Fig. 6 is a schematic diagram of the change curve of the acoustic feature model in an embodiment;

Fig. 7 is a schematic flowchart of an information processing method in another embodiment;

Fig. 8 is a structural block diagram of an information processing device in an embodiment;

FIG. 9 is a block diagram of a partial structure of a mobile phone related to the terminal provided by the embodiment of the present application.

Detailed ways

In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, and are not intended to limit the present application.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this application belongs. The terms used herein in the description of the application are only for the purpose of describing specific embodiments, and are not intended to limit the application. It can be understood that the terms "first", "second" and the like used in this application may be used to describe various elements herein, but these elements are not limited by these terms. These terms are only used to distinguish one element from another element. For example, a first application could be termed a second application, and, similarly, a second application could be termed the first application, without departing from the scope of the present application. Both the first application and the second application are applications, but they are not the same application.

Fig. 1 is a schematic diagram of an application environment of a data processing method in an embodiment. As shown in Figure 1, the application environment includes a terminal 110 and an audio processing device 120 communicating with the terminal 110, wherein the audio processing device 120 can be an earphone, and it can be understood that the audio processing device 120 can also be other functional devices.

Wherein, an audio signal is played on the terminal 110 , the audio signal includes but not limited to songs, video audio, call audio, etc., and the terminal 110 communicates with the audio processing device 120 . The type of the audio processing device 120 may be an earphone, an earphone, a headset, etc., and the terminal 110 and the audio processing device 120 may communicate in a wired or wireless manner to realize data transmission.

The audio processing device 120 includes an acoustic-electric transducer 121 positioned at a tip portion of the audio processing device 120, and when the tip portion of the audio processing device 120 is positioned in the user's ear canal, the acoustic-electric transducer 121 will The audio signal played by the terminal 110 is output to the user's ear canal. The acoustic-electric transducer 121 includes a speaker and a microphone. The speaker is used to play the audio signal sent by the terminal 110, and the microphone is used to record the audio signal around the audio processing device 120. Optionally, the microphone can also collect the audio signal and reflect it through the ear canal. Acoustic echo signal formed by vibration. In the embodiment of the present application, the loudspeaker and the microphone are integrated.

An application program (Application, APP) is installed on the terminal 110. The application program refers to a computer program for completing one or more specific tasks. It runs in a user mode, can interact with the user, and has a visual user interface. The terminal 110 interacts with the user through the application program. For example, the terminal 110 can manage the user's music playing information through the music playing application, and play the user's favorite songs and recommend songs to the user.

Fig. 2 is a schematic diagram of the internal structure of the earphone in one embodiment. The headset 120 includes a processor, a memory, and a communication interface connected through a system bus. Wherein, the processor is used to provide computing and control capabilities to support the operation of the entire earphone 120 . The memory is used to store data, programs, and/or instruction codes, etc., and at least one computer program is stored on the memory, and the computer program can be executed by the processor to implement the audio playback method suitable for the earphone 120 provided in the embodiment of the present application. The memory may include a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM), or a random-access-memory (Random-Access-Memory, RAM). For example, in one embodiment, the memory includes non-volatile storage media and internal memory. Nonvolatile storage media store operating systems, databases, and computer programs. The database stores data related to implementing an audio playing method provided by the above embodiments. The computer program can be executed by a processor, so as to implement an audio playing method provided by various embodiments of the present application. Internal memory provides a cached operating environment for operating systems, databases, and computer programs in non-volatile storage media. The communication interface includes a mobile communication interface, a wifi communication interface, a bluetooth communication interface, etc., and the headset 120 communicates with the server 110 through the communication interface to obtain audio information on the server 110 .

Those skilled in the art can understand that the structure shown in FIG. 2 is only a block diagram of a part of the structure related to the solution of this application, and does not constitute a limitation on the earphone 120 to which the solution of this application is applied. The specific earphone 120 can be More or fewer components than shown in the figures may be included, or some components may be combined, or have a different arrangement of components.

The embodiment of the present application provides an information processing method, which is applied to an audio processing device with an audio signal communication function. The audio processing device is specifically described by taking earphones as an example. It can be understood that the audio processing device can also be other devices with audio processing functions. Functional devices will not be described one by one in the embodiment of the present application.

As shown in FIG. 3 , it is a flowchart of an information processing method in an embodiment. The information processing method in this embodiment is described by taking the terminal and/or audio processing device running on the terminal and/or audio processing device in FIG. 1 as an example. The information processing method includes the following steps 302 to 308:

Step 302: Based on the audio signal played by the audio processing device, acquire an acoustic feature model associated with the internal structure of the current user's ear.

An acoustic feature model associated with the internal structure of the current user's ear may be acquired according to the audio signal currently played in the earphone. It can be understood that the audio signal can be a multimedia file currently being played, or a voice signal of the user or a contact during a call, or an audio signal (higher than 20KHz) outside the normal range of human hearing. audio signal), the user cannot hear it even if the headset is in the user's ear. The acoustic feature model can be understood as the spatial feature of the internal structure of the user's ear where the earphone is currently located, that is, different acoustic feature models can be used to represent different positions of the earphone in the user's ear.

Step 304: Obtain the use state of the audio processing device according to the acoustic feature model of the current user.

Specifically, according to the mapping relationship between the pre-stored acoustic feature model and the position of the ear canal of the user where the earphone is located, the position information of the ear canal of the user where the earphone is currently located can be acquired. When the user is wearing the headset, the current user's acoustic feature model change characteristics can be continuously obtained to identify the usage status of the headset. That is, the acoustic feature model of the current user acquired by the earphone represents the use state of the earphone.

Further, the use state of the earphone includes a wearing state and a non-wearing state. The wearing state can be understood as the state where the earpiece of the earphone is located in the user's ear canal, that is, the state where the user uses the earphone to listen to sound; the non-wearing state can be understood as the earpiece of the earphone The state away from the user's ear canal, that is, the state when the user removes the earphones.

When the user needs to use the earphone for functions such as listening to music, watching videos, or answering calls, the user will put the earphone into the ear to receive the sound transmitted on the earphone. Wherein, the earpiece of the earphone will send out an audio signal and collect the returned signal, and it can be judged whether the earpiece of the earphone is located in the user's ear canal by analyzing the returned signal. For example, an audio signal of a specific frequency (such as an ultrasonic signal) is sent out through the earpiece of the earphone. The audio signal will have different effects when passing through different obstacles. At this time, the signals around the earpiece are collected, and the collected signals are analyzed and compared. . If it is detected that the returned signal matches the human ear data, it is determined that the earphone has been inserted into the user's ear, that is, the earphone is in the wearing state at this time; if it is detected that the returned signal does not match the human ear data, it is determined that the earphone is far away from the user's ear , that is, the headset is in the non-wearing state at this time.

Step 306: Generate corresponding control instructions according to the use state of the audio processing device.

According to the pre-stored mapping relationship between the use state of the earphone and the control instruction, the corresponding control instruction can be generated according to the current use state of the earphone. Wherein, the control instruction is used to control the terminal communicating with the earphone to execute the corresponding instruction interaction operation.

Optionally, it is possible to detect whether the use state of the earphone changes; when the earphone is switched from the non-wearing state to the wearing state, the audio is played according to the preset playback strategy; when the earphone is switched from the wearing state to the non-wearing state, The earphone is then controlled to suspend playing audio, and if it is detected that the earphone remains in the non-wearing state within a preset period of time, the application program for playing audio on the terminal is closed.

Optionally, when the earphone is switched from the non-wearing state to the wearing state, the music player application on the terminal communicating with the earphone can also be started, and the music player application can be controlled to continue playing the audio specified in the previous play record .

Optionally, when the headset is switched from the non-wearing state to the wearing state, the identity information of the current wearing user can also be obtained; when the wearing user's identity information matches the preset identity information, according to the preset identity information Corresponding preset playlist for audio playback.

It can be understood that the control command generated according to the use status of the headset can also control the terminal to perform other command interaction operations, such as controlling the display page of the terminal to perform personalized display when the headset is in the wearing status. An example.

Step 308: Control the terminal communicating with the audio processing device according to the control instruction, and complete the instruction interaction operation corresponding to the control instruction.

Specifically, the acquired control instruction is transmitted to the terminal communicating with the earphone, and the terminal is instructed to perform an instruction interaction operation corresponding to the control instruction. Among them, the instruction interaction operation includes but is not limited to play/pause music, switch music songs, adjust the playback volume, open the preset application program, open the voice assistant, etc. Optionally, the instruction interaction operation can also be search, collection, download or Share the audio signal being played, it can also initiate a phone call, end a phone call, record, etc. Certainly, according to the usage habit of the user, the content of instruction interaction operation may also be added or deleted, which is not limited in this embodiment.

For example, the control instruction recognized by the earphone can control the preset application program on the terminal to perform corresponding operations, wherein the application program can be a music player, a video player, a radio player, a call, etc.; The application program can also be a voice assistant, and the terminal control operation is completed by enabling the voice assistant to recognize the user's voice commands. It can be understood that, the present embodiment is not limited to the above-mentioned illustrations, and will not be described one by one here.

The above information processing method, based on the audio signal played by the audio processing device, obtains the acoustic feature model associated with the internal structure of the current user's ear; obtains the use status of the audio processing device according to the current user's acoustic feature model; according to the The use status of the audio processing device generates a corresponding control instruction; according to the control instruction, the terminal communicating with the audio processing device is controlled, and the instruction interaction operation corresponding to the control instruction is completed. Through the above method, intelligent audio playback control can be realized according to the use state of the audio processing device, and the power consumption of the terminal can be saved when the user does not need to use the audio processing device, and the user can reduce cumbersome playback control operations and improve user experience.

In one embodiment, as shown in FIG. 4, based on the audio signal played by the audio processing device, the acoustic feature model associated with the internal structure of the current user's ear is acquired, that is, step 302 includes:

Step 402: Obtain an audio signal currently played by the audio processing device.

Obtain the audio signal currently played by the earphone. The audio signal can be the music or voice signal emitted by the multimedia file played by the preset application program, or the sound signal beyond the user's hearing range. The audio signal can also be the user's own or Voice signals of contacts, etc.

Step 404: Based on the electro-acoustic transducer playing the audio signal, collect the echo signal formed after the audio signal is reflected and vibrated by the inner structure of the ear.

The earphone includes an electro-acoustic transducer, and the electro-acoustic transducer can be used as a speaker to convert an electrical signal corresponding to an audio signal into an acoustic signal that can be heard by the user. At the same time, the electroacoustic transducer is very sensitive to the sound waves in the internal structure of the user's ear (ear canal), which can cause the vibration of the speaker paper cone, and drive the coil connected to the paper cone to cut the magnetic force line in the magnetic field of the permanent magnet. Generate a current that changes with the change of the sound wave (the phenomenon of generating current is called electromagnetic induction phenomenon in physics), and at the same time, the electromotive force of the audio frequency will be output at both ends of the coil. Therefore, the electro-acoustic converter can also collect the acoustic echo signal generated by the reflection and vibration of the internal structure of the ear. That is, the electro-acoustic transducer can also be used as a microphone. Wherein, the principle of the microphone is to convert the acoustic signal into mechanical vibration and then convert it into an electrical signal after the electro-acoustic transducer performs energy reverse conversion, so as to realize the function of collecting the echo signal.

Electro-acoustic transducers, although different in type, function or working state, they all include two basic components, namely, the electrical system and the mechanical vibration system. Inside the electro-acoustic transducer, the electrical system and the mechanical vibration system pass through Certain physical effects are interconnected to complete the energy conversion.

Based on the electroacoustic transducer playing the audio signal, the acoustic echo signal formed by the reflection and vibration of the audio signal through the internal structure of the ear does not need to be additionally provided with a microphone in the earphone to collect the acoustic echo signal, which saves costs , simplifying the internal structure of the earphone.

Optionally, the acoustic echo signal formed by the reflection and vibration of the internal structure of the ear through the audio signal may also be collected through a microphone arranged in the earphone. Wherein, when the earphone is worn in the user's ear, the microphone is arranged on the side where the earphone is in contact with the internal structure of the user's ear, that is, the microphone is arranged on the earphone housing with the speaker through hole.

Step 406: Determine an acoustic feature model associated with the internal structure of the current user's ear according to the audio signal and the echo signal.

Among them, the audio signal played by the earphone speaker is s(t), the acoustic echo signal in the ear canal collected by the microphone is r(t), and the acoustic feature model associated with the internal structure of the current user's ear is represented by w(t). , so the following expression can be obtained:

r(t)=s(t)*w(t) (1)

In the formula, w(t) is a parameter that can reflect the coupling between the earphone and the user's ear, and can be used to characterize the acoustic characteristics of the space where the earphone is placed in the user's ear. In formula (1), the audio signal s(t) and the acoustic echo signal r(t) can be obtained by monitoring the audio circuit set by the earphone or electronic equipment, and then the acoustic features associated with the internal structure of the current user's ear can be obtained Model w(t). The acoustic feature model w(t) can be understood as the spatial feature of the internal structure of the user's ear where the earphone is currently located, that is, different acoustic feature models w(t) can be used to represent different positions of the earphone in the user's ear. Correspondingly, the acoustic feature model w(t) can also be used to characterize the earprint features of each user.

Further, a noise factor e(t) can also be added to the above formula (1), wherein the noise factor e(t) includes environmental noise and circuit noise; the environmental noise is when the audio signal s(t) is not played, the recording The environmental noise generated during the process of the acoustic echo signal can be collected by an additional microphone; the circuit noise is the noise caused by the built-in circuit of the earphone, which is an inherent property of the earphone. Adding the noise factor e(t) as a known parameter, after considering the noise factor e(t), the formula (1) can be revised as:

r(t)=s(t)*w(t)+e(t) (2)

In formula (2), the newly added noise factor e(t), the audio signal s(t) and the acoustic echo signal r(t) are all known parameters, and then the parameters related to the internal structure of the current user's ear can be obtained. The acoustic feature model w(t) of .

In one embodiment, as shown in FIG. 5, the use status of the audio processing device is obtained according to the current user's acoustic feature model, that is, step 304 includes:

Step 502: Continuously acquire echo signals collected by the audio processing device according to the audio signal played by the audio processing device.

Since the position of the earphone inside the ear may change when the user is wearing the earphone, for example, the user may take off the earphone while using the earphone, and then put it on again after a period of time. The position of the headset changes. In this embodiment, according to the currently played audio signal, the echo signal collected by the earphone is continuously obtained. Since the echo signal is formed by the reflection and vibration of the internal structure of the ear through the audio signal, the function of the echo signal and the audio signal can be analyzed. The relational expression can obtain the position change of the earphone inside the user's ear.

Step 504: Construct a variation curve of an acoustic feature model according to the audio signal and the acquired echo signal.

Specifically, as shown in Figure 6, it is a schematic diagram of the change curve of the acoustic feature model during the user's use. It can be seen from the figure that when the user wears the earphone, as time t changes, when the user removes the earphone, through the earphone The acquired user's acoustic feature model w(t) presents a rising trend. Since the acoustic feature model w(t) represents the spatial characteristics of the internal structure of the user's ear where the earphone is currently located, it indicates that the position of the earphone inside the user's ear appears outward continuously. The phenomenon of movement means that the headset is not in place at this time.

Optionally, when the user uses the earphones, when the user takes off the earphones and puts on the earphones again, the user's acoustic feature model w(t) obtained through the earphones shows a downward trend as time t increases, indicating that At this time, the earphone is constantly approaching the user's ear canal, which means that the earphone is in the position at this time.

From the above analysis, it can be seen that the change curve of the user's acoustic feature model w(t) acquired by the earphone represents the position state of the earphone in the user's ear, and the position can be obtained after integral operation of the change curve of the acoustic feature model w(t) parameter, the function expression can be expressed as:

Further, according to the parameter value of the position parameter, the use state of the audio processing device as described in step 506 and step 508 can be obtained.

Step 506: When the parameter value of the acoustic feature model is smaller than a preset threshold, the audio processing device is in a wearing state.

Step 508: When the parameter value of the acoustic feature model is greater than a preset threshold, the audio processing device is in a non-wearing state.

In the information processing method provided in this embodiment, when the user wears the earphone, the echo signal collected by the earphone is continuously acquired according to the audio signal played by the earphone, and the change curve of the acoustic feature model is constructed according to the audio signal and the acquired echo signal, The use status of the earphone is obtained after integral calculation of the change curve of the acoustic characteristic model, and the in-position status of the earphone during the user's use of the earphone can be obtained in real time, so as to realize the monitoring of the in-position status of the earphone, so as to monitor the terminal connected to the earphone Perform control operations to achieve intelligent control effects and improve user experience.

In one embodiment, as shown in Figure 7, the information processing method further includes:

Step 702: Obtain the identity information of the currently wearing user.

According to the acoustic feature model, it can also be used to generate an acoustic signature to verify the user's identity, that is, to generate the user's earprint information. The earprint is a unique feature of people, just like fingerprints and facial appearances, unique.

For example, when the user uses the headset for the first time, the user's earprint information needs to be initialized and stored in the earprint database built by the preset system, so as to verify the user's identity. Specifically, the system prompts the user to identify and store the user's earprint information after wearing the headset to a preset standard position, where the preset standard position can be set and adjusted according to the user's wearing habits. When the user wears the earphones, the system obtains the current user's earprint features, and calls the earprint information in the earprint database for data matching; if the current user's earprint features can match the earprint information in the earprint database, the system prompts User "authentication successful". If the earprint feature of the current user does not match the earprint information of the preset user, after storing the earprint information of the current user, the user is prompted "identity registration is successful".

Step 704: When the identity information of the wearing user matches the preset identity information, perform audio playback according to the preset playlist corresponding to the preset identity information.

When the identity information of the wearing user matches the preset identity information, that is, the current user is the preset user of the headset, such as the owner. Then perform audio playback according to the preset playlist corresponding to the preset identity information. For example, music recommendations can be made according to the preset user's listening preferences, the frequency of song playback can be recorded, and songs with high playback frequency can be placed in the priority playlist. When a specific user is identified to wear headphones, the recorded priority playlist will be played. song.

Specifically, record the listening frequency of the audio file of the user corresponding to the preset identity information, generate a preset playlist according to the audio files whose listening frequency is higher than the preset frequency, and use the audio files in the preset playlist as Audio playback object.

For example, record the number of plays of each song corresponding to the user's ear canal information, set a song play threshold, for example, set the song play threshold to 10 times, and the song that has been played more than this threshold can be defaulted to the user's higher preference. If it is detected that the number of times the user plays song 1, song 2 and song 3 all exceeds the song playing threshold, then these three songs are added to the preset playlist. When it is detected that the user wears the earphone, the audio file in the preset playlist is used as the playback object.

In the method provided in this embodiment, by acquiring the identity information of the current wearing user, when the identity information of the wearing user matches the preset identity information, the audio is played according to the preset playlist corresponding to the preset identity information, The song can be played in priority according to the user's preference, so as to improve the user's listening experience.

It should be understood that although the steps in the flow chart corresponding to the above embodiments are displayed sequentially as indicated by the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least some of the steps in Figures 3-7 may include a plurality of sub-steps or stages, these sub-steps or stages are not necessarily executed at the same time, but may be executed at different times, these sub-steps or stages The order of execution is not necessarily performed sequentially, but may be performed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.

As shown in FIG. 8 , in one embodiment, an information processing device is provided, which includes: a feature model acquisition module 810 , a usage status acquisition module 820 , a control instruction generation module 830 and a control instruction execution module 840 .

A feature model acquisition module 810, configured to acquire an acoustic feature model associated with the internal structure of the current user's ear based on the audio signal played by the audio processing device;

The usage status acquisition module 820 is configured to acquire the usage status of the audio processing device according to the current user's acoustic feature model;

A control instruction generating module 830, configured to generate corresponding control instructions according to the usage state of the audio processing device;

The control instruction execution module 840 is configured to control the terminal communicating with the audio processing device according to the control instruction, and complete the instruction interaction operation corresponding to the control instruction.

The above information processing device, based on the audio signal played by the audio processing device, obtains the acoustic feature model associated with the internal structure of the current user's ear; obtains the use status of the audio processing device according to the current user's acoustic feature model; according to the The use status of the audio processing device generates a corresponding control instruction; according to the control instruction, the terminal communicating with the audio processing device is controlled, and the instruction interaction operation corresponding to the control instruction is completed. Through the above device, intelligent audio playback control can be realized according to the use state of the audio processing device, and the power consumption of the terminal can be saved when the user does not need to use the audio processing device, and the user can reduce cumbersome playback control operations and improve user experience.

In one embodiment, the feature model acquisition module 810 is also used to acquire the audio signal currently played by the audio processing device; based on the electro-acoustic transducer playing the audio signal, collect the reflection of the audio signal through the internal structure of the ear and the echo signal formed after the vibration; determine the acoustic feature model associated with the internal structure of the current user's ear according to the audio signal and the echo signal.

In one embodiment, the use status acquisition module 820 is also configured to continuously acquire the echo signal collected by the audio processing device according to the audio signal played by the audio processing device; construct the acoustic feature model according to the audio signal and the acquired echo signal Variation curve; when the parameter value of the acoustic feature model is less than a preset threshold, the audio processing device is in a wearing state; when the parameter value of the acoustic feature model is greater than a preset threshold, the audio processing device is in a non-wearing state state.

In one embodiment, the control instruction generation module 830 is also used to detect whether the use state of the audio processing device changes; Play; when the audio processing device is switched from the wearing state to the non-wearing state, the audio processing device is controlled to pause playing audio, and if it is detected that the audio processing device remains in the non-wearing state within a preset duration, all An application for playing audio on the above-mentioned terminal.

In one embodiment, the control instruction generating module 830 is further configured to start a music playing application on the terminal communicating with the audio processing device, and control the music playing application to continue playing the audio specified in the previous playing record.

In one embodiment, the information processing device further includes an information acquisition module, configured to acquire the identity information of the current wearing user; when the identity information of the wearing user matches the preset identity information, according to the preset identity information Corresponding preset playlist for audio playback.

In one embodiment, the control instruction execution module 840 is also used to record the listening frequency of the audio file of the user corresponding to the preset identity information; generate a preset playlist according to the audio file whose listening frequency is higher than the preset frequency; The audio files in the preset playlist are used as audio playback objects.

The division of each module in the above information processing device is only for illustration. In other embodiments, the information processing device can be divided into different modules according to needs, so as to complete all or part of the functions of the above information processing device.

For specific limitations on the information processing apparatus, refer to the above-mentioned limitations on the information processing method, which will not be repeated here. Each module in the above-mentioned information processing device can be fully or partially realized by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, and can also be stored in the memory of the computer device in the form of software, so that the processor can invoke and execute the corresponding operations of the above-mentioned modules.

The implementation of each module in the information processing device provided in the embodiment of the present application may be in the form of a computer program. The computer program can run on a terminal or a server. The program modules constituted by the computer program can be stored in the memory of the terminal or server. When the computer program is executed by the processor, the steps of the information processing method described in the embodiments of the present application are realized.

The embodiment of the present application also provides an earphone, which includes an electro-acoustic transducer, a memory, a processor, and a computer program stored in the memory and operable on the processor, the processor and the electro-acoustic transducer The processor is electrically connected to the memory, and when the processor executes the computer program, the information processing method as described in the foregoing embodiments is implemented.

The embodiment of the present application also provides a computer-readable storage medium. One or more non-transitory computer-readable storage media containing computer-executable instructions that, when executed by one or more processors, cause the processors to perform the Describe the information processing method.

The embodiment of the present application also provides a computer program product. A computer program product containing instructions, when running on a computer, causes the computer to execute the information processing methods described in the above embodiments.

The embodiment of the present application also provides a terminal device. As shown in FIG. 9 , for ease of description, only the parts related to the embodiment of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present application. The terminal device can be any terminal device including mobile phone, tablet computer, PDA (Personal Digital Assistant, personal digital assistant), POS (Point of Sales, sales of electronic equipment), vehicle-mounted computer, wearable device, etc. example:

FIG. 9 is a block diagram of a partial structure of a mobile phone related to a terminal device provided in an embodiment of the present application. 9, the mobile phone includes: a radio frequency (Radio Frequency, RF) circuit 910, a memory 920, an input unit 930, a display unit 940, a sensor 950, an audio circuit 960, a wireless fidelity (wireless fidelity, WiFi) module 970, a processor 980 , and power supply 990 and other components. Those skilled in the art can understand that the structure of the mobile phone shown in FIG. 9 does not constitute a limitation to the mobile phone, and may include more or less components than shown in the figure, or combine some components, or arrange different components.

Among them, the RF circuit 910 can be used for sending and receiving information or receiving and sending signals during a call. After receiving the downlink information from the base station, it can be processed by the processor 980; it can also send uplink data to the base station. Generally, an RF circuit includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (Low Noise Amplifier, LNA), a duplexer, and the like. In addition, RF circuitry 910 may also communicate with networks and other devices via wireless communications. The above-mentioned wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile Communication (Global System of Mobile communication, GSM), General Packet Radio Service (General Packet Radio Service, GPRS), Code Division Multiple Access (Code Division Multiple Access, CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), etc.

The memory 920 can be used to store software programs and modules, and the processor 980 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 920 . The memory 920 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as an application program for a sound playback function, an application program for an image playback function, etc.); The data storage area can store data created according to the use of the mobile phone (such as audio data, address book, etc.) and the like. In addition, the memory 920 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage devices.

The input unit 930 can be used to receive input numbers or character information, and generate key signal input related to user settings and function control of the mobile phone 900 . Specifically, the input unit 930 may include an operation panel 931 and other input devices 932 . The operation panel 931, which can also be referred to as a touch screen, can collect touch operations of the user on or near it (for example, the user uses any suitable object or accessory such as a finger or a stylus to operate on the operation panel 931 or near the operation panel 931) , and drive the corresponding connection device according to the preset program. In one embodiment, the operation panel 931 may include two parts, a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, and detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and sends it to the to the processor 980, and can receive and execute commands sent by the processor 980. In addition, the operation panel 931 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the operation panel 931 , the input unit 930 may further include other input devices 932 . Specifically, other input devices 932 may include, but are not limited to, one or more of physical keyboards, function keys (such as volume control keys, switch keys, etc.), and the like.

The display unit 940 may be used to display information input by or provided to the user and various menus of the mobile phone. The display unit 940 may include a display panel 941 . In one embodiment, the display panel 941 may be configured in the form of a liquid crystal display (Liquid Crystal Display, LCD), an organic light-emitting diode (Organic Light-Emitting Diode, OLED), or the like. In one embodiment, the operation panel 931 can cover the display panel 941. When the operation panel 931 detects a touch operation on or near it, it sends it to the processor 980 to determine the type of the touch event, and then the processor 980 determines the type of the touch event according to the touch event The type of provides a corresponding visual output on the display panel 941 . Although in Fig. 9, the operation panel 931 and the display panel 941 are used as two independent components to realize the input and input functions of the mobile phone, but in some embodiments, the operation panel 931 and the display panel 941 can be integrated to realize the mobile phone. input and output functions.

Cell phone 900 may also include at least one sensor 950, such as a light sensor, motion sensor, and other sensors. Specifically, the light sensor can include an ambient light sensor and a distance sensor, wherein the ambient light sensor can adjust the brightness of the display panel 941 according to the brightness of the ambient light, and the distance sensor can turn off the display panel 941 and/or when the mobile phone is moved to the ear. or backlight. The motion sensor can include an acceleration sensor, through which the magnitude of acceleration in various directions can be detected, and the magnitude and direction of gravity can be detected when stationary, and can be used for applications that recognize the attitude of a mobile phone (such as switching between horizontal and vertical screens), vibration recognition related functions (such as pedometer, tap), etc.; in addition, the mobile phone can also be equipped with gyroscope, barometer, hygrometer, thermometer, infrared sensor and other sensors.

Audio circuitry 960, speaker 961 and microphone 962 may provide an audio interface between the user and the handset. The audio circuit 960 can transmit the electrical signal converted from the received audio data to the speaker 961, and the speaker 961 converts it into an audio signal for output; After being received, it is converted into audio data, and after being processed by the output processor 980, the audio data can be sent to another mobile phone through the RF circuit 910, or the audio data can be output to the memory 920 for subsequent processing.

WiFi is a short-distance wireless transmission technology. The mobile phone can help users send and receive emails, browse web pages, and access streaming media through the WiFi module 970. It provides users with wireless broadband Internet access. Although FIG. 9 shows a WiFi module 970, it can be understood that it is not an essential component of the mobile phone 900 and can be omitted as required.

The processor 980 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire mobile phone. By running or executing software programs and/or modules stored in the memory 920, and calling data stored in the memory 920, execution Various functions and processing data of the mobile phone, so as to monitor the mobile phone as a whole. In one embodiment, processor 980 may include one or more processing units. In one embodiment, the processor 980 may integrate an application processor and a modem, wherein the application processor mainly processes the operating system, user interface, application programs, etc.; the modem mainly processes wireless communication. It can be understood that the foregoing modem may not be integrated into the processor 980 . For example, the processor 980 may integrate an application processor and a baseband processor, and the baseband processor and other peripheral chips may form a modem. The mobile phone 900 also includes a power supply 990 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 980 through a power management system, so that functions such as charging, discharging, and power consumption management can be realized through the power management system.

In one embodiment, the mobile phone 900 may also include a camera, a Bluetooth module, and the like.

In the embodiment of the present application, the processor included in the mobile phone implements the information processing method described above when executing the computer program stored in the memory.

When the computer program running on the processor is executed, intelligent audio playback control can be realized according to the use status of the audio processing device, and the power consumption of the terminal can be saved when the user does not need to use the audio processing device, and the user can reduce the tedious playback Control operations and improve user experience.

Any reference to memory, storage, database, or other medium as used herein may include non-volatile and/or volatile memory. Suitable nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM), which acts as external cache memory. By way of illustration and not limitation, RAM is available in many forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Chain Synchlink DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

The above-mentioned embodiments only express several implementation modes of the present application, and the description thereof is relatively specific and detailed, but should not be construed as limiting the patent scope of the present application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the scope of protection of the patent application should be based on the appended claims.

Claims (14)

1. a kind of information processing method, which is characterized in that including：

Based on the audio signal that apparatus for processing audio plays, acoustics associated with the ear internal structure of active user is obtained Characteristic model；

The use state of the apparatus for processing audio is obtained according to the acoustic feature model of active user；

Corresponding control instruction is generated according to the use state of the apparatus for processing audio；

The terminal communicated with the apparatus for processing audio is controlled according to the control instruction, and is completed and the control instruction Corresponding instruction interaction operation.

2. according to the method described in claim 1, it is characterized in that, it is described based on apparatus for processing audio play audio signal, Acoustic feature model associated with the ear internal structure of active user is obtained, including：

Obtain the currently playing audio signal of the apparatus for processing audio；

Based on the electroacoustic transducer for playing the audio signal, acquire reflection of the audio signal Jing Guo ear internal structure with The echo signal formed after vibration；

Acoustic feature mould associated with the ear internal structure of active user is determined according to the audio signal, echo signal Type.

3. according to the method described in claim 1, it is characterized in that, the use state of the apparatus for processing audio includes wearing shape State and non-wearing state；The use state that the apparatus for processing audio is obtained according to the acoustic feature model of active user, Including：

According to the audio signal that apparatus for processing audio plays, the echo signal of the apparatus for processing audio acquisition is persistently obtained；

The change curve of acoustic feature model is built according to the audio signal and the echo signal of acquisition；

When the parameter value of the acoustic feature model is less than predetermined threshold value, the apparatus for processing audio is in wearing state；

When the parameter value of the acoustic feature model is more than predetermined threshold value, the apparatus for processing audio is in non-wearing state.

4. according to the method described in claim 3, it is characterized in that, described give birth to according to the use state of the apparatus for processing audio At corresponding control instruction, including：

Whether the use state for detecting the apparatus for processing audio changes；

When the apparatus for processing audio is switched to wearing state by non-wearing state, audio is carried out according to default broadcast strategy and is broadcast It puts；

When the apparatus for processing audio is switched to non-wearing state by wearing state, then the apparatus for processing audio pause is controlled Audio is played, the apparatus for processing audio keeps non-wearing state if being detected in preset duration, closes in the terminal Application program for playing audio.

5. according to the method described in claim 4, it is characterized in that, described when the apparatus for processing audio is cut by non-wearing state When being changed to wearing state, audio broadcasting is carried out according to default broadcast strategy, including：

The music application in the terminal communicated with the apparatus for processing audio is opened, and controls the music and answers Specified audio is recorded with primary play before playing is continued.

6. according to the method described in claim 4, it is characterized in that, described when the apparatus for processing audio is cut by non-wearing state When being changed to wearing state, audio broadcasting is carried out according to default broadcast strategy, further includes：

Obtain the current identity information for wearing user；

When the identity information for wearing user is matched with default identity information, according to corresponding with the default identity information Default playlist carries out audio broadcasting.

7. according to the method described in claim 6, it is characterized in that, the basis is corresponding with the default identity information default Playlist carries out audio broadcasting, including：

It records the default identity information corresponding user and frequency is listened to audio file；

According to the frequency of listening to default playlist is generated more than the audio file of predeterminated frequency；

Object is played using the audio file in the default playlist as audio.

8. a kind of information processing unit, which is characterized in that including：

Characteristic model acquisition module, the audio signal for being played based on apparatus for processing audio obtain the ear with active user Piece associated acoustic feature model of internal structure；

Use state acquisition module, the use for obtaining the apparatus for processing audio according to the acoustic feature model of active user State；

Control instruction generation module, for generating corresponding control instruction according to the use state of the apparatus for processing audio；

Control instruction execution module, for controlling the end communicated with the apparatus for processing audio according to the control instruction End, and complete instruction interaction operation corresponding with the control instruction.

9. a kind of terminal, including memory and processor, computer-readable instruction, described instruction quilt are stored in the memory When the processor executes so that the processor executes the step of method as described in any one of claim 1 to 7.

10. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the computer program The step of method as described in any one of claim 1 to 7 is realized when being executed by processor.

11. a kind of earphone, which is characterized in that the earphone includes electroacoustic transducer, memory, processor and is stored in memory Computer program that is upper and can running on a processor, the processor are electrically connected with the electroacoustic transducer and the memory It connects, when the processor executes the computer program the step of any one of realization claim 1 to 7 the method.

12. earphone according to claim 11, which is characterized in that the electroacoustic transducer is for playing the audio letter Number, and for acquiring the acoustic echo signal generated due to pressure change in user ear canal.

13. earphone according to claim 12, which is characterized in that the electroacoustic transducer includes loud speaker and microphone, The loud speaker is generated for acquiring in user ear canal due to pressure change for playing the audio signal, the microphone Acoustic echo signal.

14. earphone according to claim 13, which is characterized in that the loud speaker and the microphone are integral type knot Structure.