How to insert a large amount of data in bulk by mybatis
This article mainly introduces how mybatis inserts a large amount of data in bulk, which has a certain reference value, and interested friends can refer to it. I hope you will gain a lot after reading this article.
When the database uses mybatis to insert a large amount of data, in order to improve efficiency, the circular insert is abandoned and the batch insert is changed. The mapper is as follows:
Package com.lcy.service.mapper;import com.lcy.service.pojo.TestVO;import org.apache.ibatis.annotations.Insert;import java.util.List;public interface TestMapper {@ Insert (") Integer testBatchInsert (List list);}
Entity class:
Package com.lcy.service.pojo;import lombok.AllArgsConstructor;import lombok.Data;import lombok.NoArgsConstructor;@Data@NoArgsConstructor@AllArgsConstructorpublic class TestVO {private String T1; private String T2; private String T3; private String T4; private String T5;}
The test classes are as follows:
Import com.lcy.service.TestApplication;import com.lcy.service.mapper.TestMapper;import com.lcy.service.pojo.TestVO;import org.junit.Test;import org.junit.runner.RunWith;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.boot.test.context.SpringBootTest;import org.springframework.test.context.junit4.SpringRunner;import java.util.ArrayList;import java.util.List;@SpringBootTest (classes = TestApplication.class) @ RunWith (SpringRunner.class) public class TestDemo {@ Autowired private TestMapper testMapper @ Test public void insert () {List list = new ArrayList (); for (int I = 0; I < 200000; ilevels +) {list.add (new TestVO (I + "," + I, I + "," + I));} System.out.println (testMapper.testBatchInsert (list));}}
In order to reproduce bug, I limited JVM memory:
The error report of the execution test class is as follows:
Java.lang.OutOfMemoryError: Java heap space
At java.base/java.util.Arrays.copyOf (Arrays.java:3746)
As you can see, Arrays caused stack memory overflow when requesting memory.
Improve the method and add it in batches:
Import com.lcy.service.TestApplication;import com.lcy.service.mapper.TestMapper;import com.lcy.service.pojo.TestVO;import org.junit.Test;import org.junit.runner.RunWith;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.boot.test.context.SpringBootTest;import org.springframework.test.context.junit4.SpringRunner;import javax.swing.*;import java.util.ArrayList;import java.util.List;import java.util.stream.Collectors @ SpringBootTest (classes = TestApplication.class) @ RunWith (SpringRunner.class) public class TestDemo {@ Autowired private TestMapper testMapper; @ Test public void insert () {List list = new ArrayList (); for (int I = 0; I < 200000; ilevels +) {list.add (new TestVO (I + "," + I, I + "," + I) } int index = list.size () / 10000; for (int istream 0 < index;i++) {/ / streamstream expression, skip means skipping the first 10000 records of the current stream, and limit means reading the first 10000 records of the current stream (list.stream (). Limit (10 000). Limit (10) (stream ());}
Another way is to increase the JVM memory, but it is not recommended, not only eat memory, but also too much data will cause the sql to report an error.
Thank you for reading this article carefully. I hope the article "how to insert a large amount of data in bulk in mybatis" shared by the editor will be helpful to everyone. At the same time, I also hope that you will support us and pay attention to the industry information channel. More related knowledge is waiting for you to learn!
