CN118505830A - Display processing method, device, electronic device and storage medium - Google Patents

Abstract

The present disclosure provides a display processing method, device, electronic device and storage medium. A display processing method, comprising: determining the pixel coordinates of the target pixel of the display device at the corresponding position on the target image; determining the partial derivative of the pixel coordinates in the direction of the screen coordinate system; determining n sampling points on the target image according to the pixel coordinates and the partial derivatives, where n is not less than 2; determining the display color of the target pixel according to the colors of the n sampling points on the target image. The method proposed in the present disclosure can significantly improve the clarity of the rendering of images with a resolution higher than the screen resolution of the display device, and the aliasing is basically eliminated, and the problem of flickering artifacts is also greatly improved.

Description

Display processing method, device, electronic device and storage medium

Technical Field

The present disclosure relates to the field of computer technology, and in particular to a display processing method, device, electronic device and storage medium.

Background Art

The image has a resolution, and the display device that displays the image also has a resolution (screen resolution). One pixel of the display device that displays the image can only display one color, and the image resolution may be higher than the resolution of the display device, which results in the pixels of the display device not corresponding one-to-one with the pixels of the image. Therefore, processing is required during rendering.

Summary of the invention

The present disclosure provides a display processing method, device, electronic device and storage medium.

The present disclosure adopts the following technical solutions.

In some embodiments, the present disclosure provides a display processing method, including:

Determine the pixel coordinates of the target pixel of the display device at the corresponding position on the target image;

Determine the partial derivative of the pixel coordinate in the direction of the screen coordinate system;

Determine n sampling points on the target image according to pixel coordinates and partial derivatives, where n is not less than 2;

The display color of the target pixel is determined according to the colors of the n sampling points on the target image.

In some embodiments, the present disclosure provides a display processing device, including:

A determination unit, used to determine the pixel coordinates of a target pixel of a display device at a corresponding position on a target image;

A calculation unit, used to determine the partial derivative of the pixel coordinate in the direction of the screen coordinate system;

The calculation unit is further used to determine n sampling points on the target image according to pixel coordinates and partial derivatives, where n is not less than 2;

The calculation unit is used to determine the display color of the target pixel according to the colors of the n sampling points on the target image.

In some embodiments, the present disclosure provides an electronic device, comprising: at least one memory and at least one processor;

The memory is used to store program codes, and the processor is used to call the program codes stored in the memory to execute the above method.

In some embodiments, the present disclosure provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store program code, and when the program code is executed by a processor, the processor is prompted to perform the above method.

The display processing method provided by the embodiment of the present disclosure takes into account the distortion factors of the display and determines the positions of n sampling points on the target image according to the first partial derivative dFdx and the second partial derivative dFdy for the distortion of pixel coordinates, thereby determining the display color of the target pixel according to the color of the sampling points. For the rendering of images higher than the screen resolution, the clarity of the rendering is significantly improved, the jagged edges are basically eliminated, and the problem of flickering artifacts is also greatly improved.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

FIG. 1 is a flow chart of a graphics rendering pipeline according to an embodiment of the present disclosure.

FIG. 2 is a schematic diagram of an image displaying a triangle according to an embodiment of the present disclosure.

FIG. 3 is a schematic diagram of a display state of a pixel of a display device according to an embodiment of the present disclosure.

FIG. 4 is a schematic diagram of an image sampling point according to an embodiment of the present disclosure.

FIG. 5 is a schematic diagram of image distortion according to an embodiment of the present disclosure.

FIG. 6 is a schematic diagram of a distorted image, a convex lens, and a displayed image according to an embodiment of the present disclosure.

FIG. 7 is a flow chart of a display processing method according to an embodiment of the present disclosure.

FIG. 8 is a schematic diagram of the locations of sampling points according to an embodiment of the present disclosure.

FIG. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.

DETAILED DESCRIPTION

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

It should be understood that the various steps described in the method embodiments of the present disclosure can be performed in sequence and/or in parallel. In addition, the method embodiments may include additional steps and/or omit the steps shown. The scope of the present disclosure is not limited in this respect.

The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

It should be noted that the modification of “one” mentioned in the present disclosure is illustrative rather than restrictive, and those skilled in the art should understand that it should be understood as “one or more” unless otherwise clearly indicated in the context.

The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

The solution provided by the embodiment of the present disclosure will be described in detail below with reference to the accompanying drawings.

In the related art, the rendering of images, such as the rendering of virtual reality images, uses the graphics rendering pipeline of OpenGL, and its main process is shown in Figure 1. Before starting to draw graphics, you need to input vertex data to OpenGL. For example, when you want to render a triangle, you need to input the coordinates of three vertices. Each vertex has a 3D position, and the 3D coordinates are represented by an array, which is the vertex data. The vertex shader stage takes a single vertex as input. The function of the vertex shader is to convert the 3D coordinates into another 3D coordinate and perform some processing on the vertex attributes. The primitive assembly stage takes all the vertices output by the vertex shader as input, and assembles all the points into the shape of the specified primitive. The output of the primitive assembly stage will be passed to the geometry shader. The geometry shader takes a collection of a series of vertices in the form of primitives as input. The output of the geometry shader will be passed to the rasterization stage, which maps the primitives to the corresponding pixels on the final screen and generates fragments for the fragment shader. Clipping is performed before the fragment shader runs. Clipping discards all pixels outside the view. The main function of the fragment shader is to calculate the final color of a pixel. After all the corresponding color values are determined, the final object will be passed to the test and blend stage. This stage detects the corresponding depth values of the fragments, uses them to determine the relationship between this pixel and other objects, and determines whether to discard it. Then it will enter the FrameBuffer stage, store it in the display buffer, and then display it.

The display device has a resolution, and the target image to be displayed also has a resolution. When the resolution of the target image is greater than the resolution of the display device, a pixel of the display device cannot correspond to a pixel of the target image one-to-one. In some embodiments, a specific method is used to map the target image to a pixel on the final display device. Taking GPU rendering of the raster as an example, this operation is completed in the rasterization stage, and a fragment for the fragment shader is generated. The color sampled by the fragment shader from the fragment is used as the color displayed by the pixel of the display device. If the resolution of the target image to be displayed is greater than the resolution of the display device, the fragment shader only samples one point in the fragment, which may cause the image to have jagged problems. Taking the triangle shown in Figure 2 as an example, Figure 2 shows a schematic diagram of the target image. The grid is a fragment (fragment) after the pixel of the display device is mapped on the target image. Each fragment center contains a sampling point. The color of each pixel at the sampling point covered by the triangle is sampled as the color of the pixel. By collecting the color of the center of the corresponding position of the pixel on the target image, it is used as the color of the pixel on the display device. The result is shown in Figure 3, which shows the state of the pixel on the display device. It can be seen that the triangle actually displayed has jagged edges, one square is one pixel, and the displayed color is easily distorted. In some embodiments, a multi-sampling algorithm is used, that is, the color of each pixel is no longer determined solely by the color of the sampling point at the center of a segment, but is jointly determined by the colors of multiple sampling points. For example, as shown in FIG. 4 , four sampling points are set in a segment, and the final color of the pixel is determined by the colors of the four sampling points, thereby being able to restore the image more realistically and solve the problem of aliasing.

However, in some display devices, non-planar lenses such as convex lenses or concave lenses are used. Due to refraction, the image will be distorted when displayed through the lens. For example, when a convex lens is used, a barrel distortion as shown in Figure 5 will be generated, and the original straight line will become a curve. In order to correct this distortion, the target image to be displayed is generally distorted in advance. For example, as shown in Figure 6, the left side of Figure 6 is a distorted target image, and the lens of the display device is a convex lens. The final effect is shown on the right side of Figure 6. Since the distortion was performed in advance, it can be normally displayed as the image shown on the right side of Figure 6 after passing through the convex lens, and the image displayed by the human eye is normal. Because the target image is distorted first, the original primitive size corresponding to the fragment used for fragment shader sampling is different, and the UV coordinate (pixel coordinate) of each pixel sampled does not change linearly. For the case where the lens is a plane lens and no distortion is required, the position of the sampling point of each pixel can be the same, but for the case of a convex lens or a concave lens, the sampling effect of each pixel using the sampling point at the same distance from the center point is also different. If the sampling point with the same distance from the center point is selected, the pixel at the center position may not be much different, but it will no longer be suitable for the pixels in the edge area. Therefore, when sampling a distorted image, it is necessary to select appropriate sampling points to ensure the sampling effect.

As shown in FIG. 7 , FIG. 7 is a flowchart of a display processing method according to an embodiment of the present disclosure. The method proposed in the present disclosure may be used in the processing stage executed by a fragment shader (as shown in FIG. 1 ), and may specifically include the following steps.

S11. Determine the pixel coordinates of the target pixel of the display device at the corresponding position on the target image.

In some embodiments, the method proposed in the present disclosure can be used for a display device, and the display device can be, for example, a virtual reality device, an extended reality device, an augmented reality device, etc., without limitation. The display device can be a device with a convex lens or a concave lens. In some embodiments, the target image is an image that needs to be displayed on the display device. The display device has a resolution, and fragments can be generated at the rasterization stage shown in Figure 1 according to the resolution of the display device. The target pixel of the display device can be any pixel, and the pixel coordinates (u, v) corresponding to the target pixel can be the coordinates of the center point of the corresponding fragment (see Figure 2). The pixel coordinates are texture coordinates, that is, the coordinates of the image itself. One fragment corresponds to one pixel of the display device, and one fragment can cover multiple pixels of the target image itself. The target pixel (which can be denoted as F) can be the pixel currently being processed on the display device.

S12. Determine the partial derivative of the pixel coordinate in the screen coordinate system.

In some embodiments, the pixel coordinates (u, v) are coordinates of a texture coordinate system, and the texture coordinate system is a different coordinate system from the screen coordinate system. The direction of the screen coordinate system may include a first direction x and a second direction y, and the first direction x and the second direction y intersect. The partial derivatives of the pixel coordinates (u, v) in the first direction x and the second direction y of the screen coordinate system may be determined to obtain a first partial derivative dFdx and a second partial derivative dFdy. Specifically, the first direction x may be the horizontal direction in the target image, and the second direction y may be the vertical direction in the target image, and the two are perpendicular to each other. The pixel coordinates may be expressed as (u, v). The first partial derivative dFdx and the second partial derivative dFdy may be calculated by dFdxFine, or by dFdxCoarse, or by using dFdx(). The partial derivatives are calculated by the rate of change of the pixel coordinates in the first direction x and the second direction y.

S13. Determine n sampling points on the target image according to pixel coordinates and partial derivatives, where n is not less than 2.

In some embodiments, the resolution of the target image itself is different from the resolution of the display device, so the color can be better represented by n sampling points. More importantly, the setting position of the sampling points in this embodiment needs to take into account the partial derivatives of the pixel coordinates in the direction of the screen coordinate system. The n sampling points on the target image can be determined according to the pixel coordinates (u, v), the first partial derivative dFdx and the second partial derivative dFdy, and the partial derivatives include the first partial derivative dFdx and the second partial derivative dFdy. This is because when the lens of the display device is a convex lens or a concave lens, distortion will occur (refer to Figure 6), and the fragments used for fragment shader sampling correspond to the original image primitive size. The coordinates of each pixel sampling point do not change linearly, so the positions of the n sampling points on the target image are determined according to the partial derivatives in the present disclosure. The partial derivatives reflect the degree of distortion, so that the sampling points are determined according to the partial derivatives to adapt the distorted image. In addition, the method proposed in the embodiment of the present disclosure can also adapt to undistorted images.

S14. Determine the display color of the target pixel according to the colors of the n sampling points on the target image.

In some embodiments, the color displayed by the target pixel of the final display device is determined by the color synthesis of n sampling points. One pixel on the display device displays one color. The fragment corresponding to the target pixel can cover more than one pixel on the target image. Therefore, the target pixel cannot directly use the color of the center point of the fragment. The display color of the target pixel needs to be determined based on the colors of at least two sampling points.

In some embodiments of the present disclosure, partial derivatives of pixel coordinates (u, v) in a first direction x and a second direction y of a screen coordinate system are determined to obtain a first partial derivative dFdx and a second partial derivative dFdy; positions of n sampling points of a target image are determined according to the pixel coordinates (u, v), the first partial derivative dFdx and the second partial derivative dFdy, so that the selection of the positions of the sampling points changes with the change of the partial derivatives of the pixel coordinates. The first partial derivative dFdx and the second partial derivative dFdy can reflect the degree of distortion. For different pixels on a display device, the positions of corresponding sampling points may be different rather than fixed, so that the selection of the positions of the sampling points is more reasonable, thereby improving the display effect of the display color presented by the display, and being applicable to distorted images. For rendering of images with a resolution higher than the screen resolution, the clarity of the rendering is significantly improved, the jagged edges are basically eliminated, and the problem of flickering artifacts is also greatly improved.

In some embodiments of the present disclosure, determining n sampling points on the target image based on the pixel coordinates and the partial derivatives includes: determining a sampling distance in the direction of the screen coordinate system based on the partial derivatives; and determining the n sampling points on the target image based on the sampling distance in the direction of the pixel coordinates and the screen coordinate system.

In some embodiments, the partial derivative reflects the degree of distortion of the target image, so the sampling distance is determined by the partial derivative, and the sampling distance may be the distance from the pixel coordinates, or the sampling distance may be a reference for determining the distance from the pixel coordinates. According to the sampling distance, the interval or difference between the sampling point and the pixel coordinates is determined, thereby determining the position of the sampling point. In some embodiments, when the number of sampling points is different, the position of the sampling point may be different, and the position of the sampling point relative to the pixel coordinates may be determined according to the sampling distance and the number of sampling points n.

In some embodiments of the present disclosure, n sampling points on the target image are determined according to the pixel coordinates (u, v), the first partial derivative dFdx and the second partial derivative dFdy, including: determining a first distance dx in the first direction x according to the first partial derivative dFdx; determining a second distance dy in the second direction y according to the second partial derivative dFdy; and determining n sampling points on the target image according to the pixel coordinates (u, v), the first distance dx and the second distance dy.

In some embodiments, when determining the position of the sampling point, the magnitude of the rate of change needs to be considered. The first partial derivative dFdx represents the rate of change of the pixel coordinates in the first direction x, and the second partial derivative dFdy represents the rate of change of the pixel coordinates in the second direction y. These two rates of change also affect the intervals of the sampling points in the first direction x and the second direction y. The first distance dx and the second distance dy are sampling distances, which respectively represent the distances from the pixel coordinates that need to be referenced in the first direction x and the second direction y.

In some embodiments of the present disclosure, n sampling points on the target image are determined according to the pixel coordinates (u, v), the first distance dx and the second distance dy, including: the coordinates of the pixel coordinates (u, v) in the screen coordinate system are (x ₀ , y ₀ ); the number of the sampling points is 4, which are (x ₀ +dx, y ₀ ), (x ₀ -dx, y ₀ ), (x ₀ , y 0 +dy) and (x 0 , y ₀ -dy) in the coordinate system with the first direction and the second direction as axes, respectively; or, the number of the sampling points is 4, which are (x ₀ +dx, y ₀ +dy), (x ₀ +dx, y ₀ -dy), (x ₀ -dx, y ₀ +dy) and (x ₀ -dx, y ₀ -dy) in the coordinate system with the first direction and the second direction as axes _{, respectively} .

In some embodiments, as shown in FIG8 , the box in FIG8 is a fragment corresponding to the target pixel, and the coordinates of the center of the target fragment are (x ₀ , y ₀ ), which shows four sampling points obtained by extending the first distance dx and the second distance dy from the center point to the surrounding areas. In other embodiments, a rectangle may be made with 2 times dx and 2 times dy as the length and width, the center of the rectangle coincides with the center point of the fragment, and the four vertices of the rectangle are sampling points.

In some embodiments of the present disclosure, determining the second distance in the second direction y according to the second partial derivative dFdy includes: setting the second distance dy equal to one quarter of the second partial derivative dFdy. In some embodiments, determining the first distance dx in the first direction x according to the first partial derivative dFdx includes: setting the first distance dx equal to one quarter of the first partial derivative dFdx.

In some embodiments, the first partial derivative dFdx represents the rate of change from the pixel coordinates along the first direction x, so half of the first partial derivative dFdx is close to the boundary from the current segment along the first direction to the adjacent segment, and a quarter of the first partial derivative dFdx is close to the half of the current segment from the center point to the boundary, but they may not coincide. The second partial derivative dFdx similarly represents the rate of change from the pixel coordinates along the second direction y, so half of the second partial derivative dFdy is close to the boundary from the current segment along the second direction to the adjacent segment, and a quarter of the second partial derivative dFdy is close to the half of the current segment from the center point to the boundary, but they may not coincide.

In some embodiments of the present disclosure, determining the display color of the target pixel according to the colors of the n sampling points on the target image includes: using the average value of the colors of the n sampling points as the display color of the target pixel. In some embodiments, using the average value of the colors of the n sampling points as the display color is more accurate and reasonable than directly using the color of the center point of the fragment.

In some embodiments of the present disclosure, the target image is a distorted image. In some embodiments, the lens of the display device is a convex lens or a concave lens. In order to ensure that the image viewed by the user is displayed correctly, the target image is a distorted image, for example, a four-sided concave pillow-shaped image adapted to the convex lens as shown in FIG6 , and the fragment on the target image corresponding to the pixel of the display device may be a non-rectangular area, and may be a distorted quadrilateral as shown in FIG6 .

In some embodiments, the resolution of the target image is greater than the resolution of the display device displaying the target image. Therefore, the pixels in the target image are larger than the pixels that can be displayed by the display device, so a fragment can cover more than one pixel of the target image itself.

In order to better illustrate the method proposed in the present disclosure, a specific embodiment is proposed below. In this embodiment, in order to realize the automatic selection of appropriate sampling points. It is necessary to achieve that the sampling effects of the sampling points corresponding to the pixels at different positions of the display device are equivalent. Since glsl (GL Shader Language) provides the functions dFdx()/dFdy() for calculating partial derivatives, these functions can be used to calculate the partial derivatives of (u, v) (pixel coordinates) in the first direction x and the second direction y respectively. If the difference between u/v between two pixels is relatively large/small, the calculated partial derivatives will also be relatively large/small, so that the corresponding value can be obtained according to the rate of change of u/v, and the position of the sampling point from the center point can be calculated based on this value. For example, 1/4 of the partial derivatives of (u, v) in the first direction x and the second direction y are used as the distance of the sampling point from the center point in the first direction x and the second direction y, that is, at the center of the fragment corresponding to the current target pixel on the target image, the four points of the upper left, upper right, lower right and lower left are taken as sub-sampling points. Among them, dx and dy can be regarded as 1/4 of the difference between the coordinate values of two adjacent pixels in the first direction and the second direction.

The present disclosure further provides a display processing device, which may be a display device, such as a virtual reality device, an augmented reality device, or an extended reality device, and includes:

A determination unit, used to determine the pixel coordinates of a target pixel of a display device at a corresponding position on a target image;

A calculation unit, used to determine the partial derivative of the pixel coordinate in the direction of the screen coordinate system;

The calculation unit is further used to determine n sampling points on the target image according to the pixel coordinates and the partial derivatives, where n is not less than 2;

The calculation unit is used to determine the display color of the target pixel according to the colors of the n sampling points on the target image.

In some embodiments, determining n sampling points on the target image according to the pixel coordinates and the partial derivatives includes:

Determine a sampling distance in the direction of the screen coordinate system according to the partial derivative;

n sampling points on the target image are determined based on the pixel coordinates and the sampling distance in the direction of the screen coordinate system.

In some embodiments, determining the partial derivatives of the pixel coordinates in the direction of the screen coordinate system includes: determining the partial derivatives of the pixel coordinates (u, v) in the first direction x and the second direction y of the screen coordinate system to obtain a first partial derivative dFdx and a second partial derivative dFdy;

Determining n sampling points on the target image according to the pixel coordinates and the partial derivatives includes: determining n sampling points on the target image according to the pixel coordinates (u, v), the first partial derivative dFdx and the second partial derivative dFdy.

In some embodiments, determining n sampling points on the target image according to the pixel coordinates (u, v), the first partial derivative dFdx and the second partial derivative dFdy comprises:

Determine a first distance dx in the first direction x according to the first partial derivative dFdx;

Determine a second distance dy in the second direction y according to the second partial derivative dFdy;

Determine n sampling points on the target image according to the pixel coordinates (u, v), the first distance dx and the second distance dy.

In some embodiments, determining n sampling points on the target image according to the pixel coordinates (u, v), the first distance dx and the second distance dy includes:

The coordinates of the pixel coordinate (u, v) in the screen coordinate system are (x ₀ , y ₀ );

The number of the sampling points is 4, namely (x ₀ +dx, y ₀ ), (x ₀ -dx, y ₀ ), (x ₀ , y ₀ +dy) and (x ₀ , y ₀ -dy), or,

The number of the sampling points is 4, namely (x ₀ +dx, y ₀ +dy), (x ₀ +dx, y ₀ -dy), (x ₀ -dx, y ₀ +dy) and (x ₀ -dx, y ₀ -dy).

In some embodiments, determining the first distance dx in the first direction x according to the first partial derivative dFdx includes: setting the first distance dx equal to one quarter of the first partial derivative dFdx; and/or,

Determining the second distance in the second direction y according to the second partial derivative dFdy includes: setting the second distance dy equal to one quarter of the second partial derivative dFdy.

In some embodiments, determining the display color of the target pixel according to the colors of the n sampling points on the target image includes:

The average value of the colors of the n sampling points is used as the display color of the target pixel.

In some embodiments, the target image is a non-rectangular distorted image; and/or,

The resolution of the target image is greater than the resolution of a display device that displays the target image.

For the embodiments of the device, since they basically correspond to the method embodiments, the relevant parts refer to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the modules described as separation modules may or may not be separated. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present embodiment. Those of ordinary skill in the art can understand and implement without paying creative work.

The method and device of the present disclosure are described above based on the embodiments and application examples. In addition, the present disclosure also provides an electronic device and a computer-readable storage medium, which are described below.

Referring to FIG9 below, it shows a schematic diagram of the structure of an electronic device (e.g., a terminal device or a server) 800 suitable for implementing the embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (e.g., vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. The electronic device shown in the figure is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

The electronic device 800 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 are also stored. The processing device 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input/output (I/O) interface 805 is also connected to the bus 804.

Typically, the following devices may be connected to the I/O interface 805: input devices 806 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 808 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 809. The communication device 809 may allow the electronic device 800 to communicate with other devices wirelessly or by wire to exchange data. Although the electronic device 800 with various devices is shown in the figure, it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have alternatively.

In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 809, or installed from a storage device 808, or installed from a ROM 802. When the computer program is executed by the processing device 801, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

The computer-readable medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method of the present disclosure.

Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and/or flow chart, and the combination of the square boxes in the block diagram and/or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, constitute a limitation on the unit itself.

The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

According to one or more embodiments of the present disclosure, a display processing method is provided, including:

Determine the pixel coordinates (u, v) of the corresponding position of the target pixel of the display device on the target image;

Determine the partial derivative of the pixel coordinate in the direction of the screen coordinate system;

Determine n sampling points on the target image according to pixel coordinates and partial derivatives, where n is not less than 2;

Determine the display color of the target pixel based on the colors of n sampling points on the target image.

According to one or more embodiments of the present disclosure, a display processing method is provided, which determines n sampling points on the target image according to the pixel coordinates and the partial derivatives, including:

Determine a sampling distance in the direction of the screen coordinate system according to the partial derivative;

n sampling points on the target image are determined based on the pixel coordinates and the sampling distance in the direction of the screen coordinate system.

According to one or more embodiments of the present disclosure, a display processing method is provided, which determines the partial derivatives of the pixel coordinates in the direction of the screen coordinate system, including: determining the partial derivatives of the pixel coordinates (u, v) in the first direction x and the second direction y of the screen coordinate system to obtain a first partial derivative dFdx and a second partial derivative dFdy;

Determining n sampling points on the target image according to the pixel coordinates and the partial derivatives includes: determining n sampling points on the target image according to the pixel coordinates (u, v), the first partial derivative dFdx and the second partial derivative dFdy.

According to one or more embodiments of the present disclosure, a display processing method is provided, which determines n sampling points on a target image according to pixel coordinates (u, v), a first partial derivative dFdx, and a second partial derivative dFdy, including:

Determine a first distance dx in a first direction x according to the first partial derivative dFdx;

Determine a second distance dy in the second direction y according to the second partial derivative dFdy;

Determine n sampling points on the target image according to the pixel coordinates (u, v), the first distance dx and the second distance dy.

According to one or more embodiments of the present disclosure, a display processing method is provided, which determines n sampling points on a target image according to pixel coordinates (u, v), a first distance dx, and a second distance dy, including:

The coordinates of the pixel coordinate (u, v) in the coordinate system with the first direction and the second direction as axes are (x ₀ ,y ₀ );

The number of sampling points is 4, namely (x ₀ +dx,y ₀ ), (x ₀ -dx,y ₀ ), (x ₀ ,y ₀ +dy) and (x ₀ ,y ₀ -dy), or,

The number of sampling points is 4, namely (x ₀ +dx, y ₀ +dy), (x ₀ +dx, y ₀ -dy), (x ₀ -dx, y ₀ +dy) and (x ₀ -dx, y ₀ -dy).

According to one or more embodiments of the present disclosure, a display processing method is provided, which determines a first distance dx in a first direction x according to a first partial derivative dFdx, including: setting the first distance dx equal to one quarter of the first partial derivative dFdx; and/or,

Determining a second distance in the second direction y according to the second partial derivative dFdy includes: setting the second distance dy equal to one quarter of the second partial derivative dFdy.

According to one or more embodiments of the present disclosure, a display processing method is provided, which determines the display color of a target pixel according to the colors of n sampling points on a target image, including:

Use the average color of n sampling points as the display color of the target pixel.

According to one or more embodiments of the present disclosure, a display processing method is provided, wherein a target image is a non-rectangular distorted image.

According to one or more embodiments of the present disclosure, a display processing method is provided, in which the resolution of a target image is greater than the resolution of a display device that displays the target image.

According to one or more embodiments of the present disclosure, there is provided a display processing device, including:

A determination unit, used to determine the pixel coordinates of a target pixel of a display device at a corresponding position on a target image;

A calculation unit, used to determine the partial derivative of the pixel coordinate in the direction of the screen coordinate system;

The calculation unit is further used to determine n sampling points on the target image according to the pixel coordinates and the partial derivatives, where n is not less than 2;

The calculation unit is used to determine the display color of the target pixel according to the colors of the n sampling points on the target image. According to one or more embodiments of the present disclosure, there is provided an electronic device, including: at least one memory and at least one processor;

The at least one memory is used to store program codes, and the at least one processor is used to call the program codes stored in the at least one memory to execute any one of the above methods.

According to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium is used to store program code, and when the program code is executed by a processor, the processor is prompted to execute the above method.

The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other.

In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

Although the subject matter has been described in language specific to structural features and/or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims (11)

1. A display processing method, characterized by comprising:

determining pixel coordinates of a corresponding position of a target pixel of the display device on the target image;

Determining a partial derivative of the pixel coordinates in the direction of a screen coordinate system;

determining n sampling points on the target image according to the pixel coordinates and the partial derivative, wherein n is not less than 2;

And determining the display color of the target pixel according to the colors of the n sampling points on the target image.

2. The method of claim 1, wherein determining n sampling points on the target image from the pixel coordinates and the partial derivatives comprises:

determining a sampling distance in the direction of the screen coordinate system according to the partial derivative;

And determining n sampling points on the target image according to the pixel coordinates and the sampling distances in the direction of the screen coordinate system.

3. The method of claim 1, wherein the step of determining the position of the substrate comprises,

Determining the partial derivative of the pixel coordinates in the direction of the screen coordinate system comprises: determining partial derivatives of the pixel coordinates (u, v) in a first direction x and a second direction y of the screen coordinate system to obtain a first partial derivative dFdx and a second partial derivative dFdy;

Determining n sampling points on the target image according to the pixel coordinates and the partial derivative, including: n sampling points on the target image are determined from the pixel coordinates (u, v), the first partial derivative dFdx, and the second partial derivative dFdy.

4. A method according to claim 3, wherein determining n sampling points on the target image from the pixel coordinates (u, v), the first partial derivative dFdx and the second partial derivative dFdy comprises:

Determining a first distance dx in the first direction x from the first partial derivative dFdx;

determining a second distance dy in the second direction y from the second partial derivative dFdy;

N sampling points on the target image are determined from the pixel coordinates (u, v), the first distance dx and the second distance dy.

5. The method of claim 4, wherein determining n sampling points on the target image from the pixel coordinates (u, v), the first distance dx, and the second distance dy comprises:

The pixel coordinates (u, v) have coordinates (x ₀,y₀) in the screen coordinate system;

The number of the sampling points is 4, namely (x ₀+dx,y₀)、(x₀-dx,y₀)、(x₀,y₀ +dy) and (x ₀,y₀ -dy), or

The number of the sampling points is 4, and the sampling points are (x ₀+dx,y₀+dy)、(x₀+dx,y₀-dy)、(x₀-dx,y₀ +dy) and (x ₀-dx,y₀ -dy) respectively.

6. The method of claim 4, wherein determining a first distance dx in the first direction x from the first partial derivative dFdx comprises: setting the first distance dx equal to a quarter of the first partial derivative dFdx; and/or the number of the groups of groups,

Determining a second distance in the second direction y from the second partial derivative dFdy comprises: the second distance dy is set equal to one fourth of the second partial derivative dFdy.

7. The method of claim 1, wherein determining the display color of the target pixel based on the colors of the n sampling points on the target image comprises:

And using the average value of the colors of the n sampling points as the display color of the target pixel.

8. The method of claim 1, wherein the step of determining the position of the substrate comprises,

The target image is a non-rectangular distorted image;

And/or the number of the groups of groups,

The resolution of the target image is greater than the resolution of a display device displaying the target image.

9. A display processing apparatus, comprising:

A determining unit for determining pixel coordinates of a corresponding position of a target pixel of the display device on the target image;

A calculation unit for determining a partial derivative of the pixel coordinates in a direction of a screen coordinate system;

The computing unit is further used for determining n sampling points on the target image according to the pixel coordinates and the partial derivative, wherein n is not less than 2;

The computing unit is used for determining the display color of the target pixel according to the colors of the n sampling points on the target image.

10. An electronic device, comprising:

at least one memory and at least one processor;

Wherein the at least one memory is configured to store program code, and the at least one processor is configured to invoke the program code stored by the at least one memory to perform the method of any of claims 1 to 8.

11. A computer readable storage medium for storing program code which, when executed by a processor, causes the processor to perform the method of any one of claims 1 to 8.